Project · Intermediate · about 20 hours
An LLM triage integration driven by helpdesk webhooks
Build a TypeScript service that receives signed helpdesk webhooks, triages each ticket with an LLM, writes the result back through the API, and survives retries, duplicates and outages.
Skills:LLMs and NLP, Monitoring and observability, Docker and containers, REST APIs and HTTP, TypeScript and JavaScript
Part of: Forward Deployed Engineer
Data and licences
- Bitext customer support dataset (synthetic, 26,872 messages) ↗CDLA-Sharing-1.0
Brief
Most forward deployed work is integration: the model is only useful once it sits inside the customer's tools. Here the customer's helpdesk sends a webhook for every new ticket. Your service must tag it, draft a reply as an internal note, and never make things worse when the helpdesk retries, the model API is slow or your service restarts.
What to build
A mock helpdesk that replays Bitext messages as new tickets, sends signed webhooks, retries on timeouts and sometimes sends the same event twice.
A webhook endpoint that verifies the HMAC signature, stores the event and returns 202 within 200 ms.
A worker that takes events from a queue (a database table is fine), calls the LLM with structured output, validates it, and updates the ticket through the API with an idempotency key.
Retries with backoff for transient failures, and a dead-letter table with the reason after the last attempt.
Metrics (events received, processed, failed, latency) exposed for Prometheus, and structured logs with the ticket id.
Docker Compose for the mock helpdesk, service, worker and database, plus a runbook.
Deliverables
Repository with
helpdesk-mock/,service/,worker/,tests/,compose.yamlandRUNBOOK.md
How it is graded
Reviewers replay 500 tickets with duplicates, a 30-second model outage and a worker restart. Every ticket should be updated exactly once or be in the dead-letter table with a reason.
Stretch
Add a feature flag so the customer can switch the integration to "suggest only" or "off" without a deploy.
Deliverables
- A mock helpdesk that sends signed webhooks for new tickets and accepts updates through its API
- A TypeScript service that verifies signatures, queues events, triages with an LLM and writes back tags and a draft
- Idempotent processing, retries with a dead-letter store, and metrics for throughput, latency and failures
- A Docker Compose setup and a README a customer's IT team could follow
How a reviewer grades it
| Criterion | Weight | What good looks like |
|---|---|---|
| Webhooks are verified, acknowledged quickly and processed asynchronously | 25% | |
| Duplicates and retries never produce double updates; failures land in a dead-letter store | 25% | |
| LLM output is validated before anything is written back | 20% | |
| Metrics, logs with request ids, and a runbook | 30% |