Support triage

Tickets arrive from the helpdesk all day. Millstream classifies each one, drafts a reply with an agent, and gives support staff an inbox where they approve, edit or reject the draft, with urgent tickets in their own queue and a short timeout.

The pipeline

flowchart LR
  W(["webhook: helpdesk"]) --> D{{dedupe}} --> C["classify (agent)"] --> R["draft reply (agent)"]
  R -->|priority p1| U[/urgent review/]
  R -->|other| N[/support review/]
  U & N --> S[send reply]

pipeline: support-triage
description: Classify tickets and draft replies for review

triggers:
  ticket: { webhook: { rate: 300/min } }

streams:
  tickets.unique: { retention: 180d, redact: [/customer/email] }

steps:
  dedupe:
    action: dedupe
    in: trigger.ticket
    out: tickets.unique
    config: { key: "item.ticketId + ':' + string(item.revision)", window: 7d }
  classify:
    action: lab/classify-ticket@1
    in: tickets.unique
    out: tickets.classified
    concurrency: 8
  draft:
    action: lab/draft-reply@1
    in: tickets.classified
    concurrency: 4
    out:
      - { when: "item.priority == 'p1'", out: replies.urgent }
      - { when: "true", out: replies.normal }
  urgent-review:
    action: review
    in: replies.urgent
    out: replies.approved
    config: { inbox: support-urgent, timeout: 30m, onTimeout: { escalate: support-leads }, editable: [/reply] }
  support-review:
    action: review
    in: replies.normal
    out: replies.approved
    config:
      inbox: support
      timeout: 8h
      onTimeout: { escalate: support-leads }
      editable: [/reply, /category]
      reasons: [wrongCategory, wrongFacts, wrongTone, needsEngineer]
  send:
    action: lab/helpdesk-reply@1
    in: replies.approved

budget: { dayUsd: 30 }

What Millstream does for it

Two agents, two jobs

• classify answers a small schema: category, priority, language. Its manifest can name a smaller model, such as claude-haiku-5-5, with maxTurns: 2.
• draft writes the reply and may read the help centre through the tools and network destinations its manifest declares, nothing else.
• Both are tested with recorded calls: millstream test pipeline and millstream eval replay them in CI without an API key.

Staff stay in charge

• Every reply is read by a person before send, an action declared effects: external.
• The urgent queue escalates after 30 minutes, the normal one after eight hours, each through a chat-compatible notify webhook if you add one.
• Rejections carry a reason code, and every edit to /reply is kept as a diff.

Ticket text is untrusted

• Ticket bodies reach the agents, so their output is validated against the output schema before it goes anywhere.
• millstream check warns when agent output can reach an external effect without a review step between them; this pipeline has one on every path.

How it learns

Edits to /reply are attributed through lineage to draft, and recurring ones become a lessons proposal for it: rules plus corrected examples, rendered into the prompt. Edits to /category point at classify instead. classify has a bounded output (enums only) and two turns at most, the shape the distillable insight looks for: after 500 deliveries, and with 95% agreement from the reviewers who check its category, improve can draft Go code that classifies the common tickets and abstains on the rest, which go to the agent as its fallback. The code must agree on at least 98% of the holdout items it covers, and cover at least 60% of them, before a person sees the proposal.

Figures to watch

Next

How agents run · Human review · Self-improvement