Human review
Some decisions belong to a person. A review step parks each item in an inbox until someone decides, records every
decision in full, and measures how often people had to step in, so you can see that number go down.
A step like any other
review is a builtin. Items arrive on its input stream, wait in an inbox, and leave on its output stream when
approved, edited or not.
analyst-review:
action: review
in: listings.selected
out: listings.approved
config:
inbox: analysts
timeout: 72h
onTimeout: { escalate: leads }
editable: [/marginPct, /notes]
reasons: [overpriced, badLocation, needsVisit]
notify: { secret: reviewWebhook }
autoApprove: "item.marginPct >= 25"
sampleRate: 0.05
| Key | Does |
|---|---|
inbox |
Names the inbox; the commercial edition controls who reaches it |
timeout, onTimeout |
How long an item may wait, then reject, approve or escalate to another inbox |
editable |
JSON pointers the reviewer may change; the edit must still match the schema downstream |
reasons |
Reason codes; a rejection then requires one |
notify |
A webhook, Slack-compatible, with a link to the item in the inbox |
autoApprove, sampleRate |
A CEL rule that skips the inbox, and the share of its approvals a person still checks |
A parked item holds no lease and no concurrency slot, so a review that takes three days blocks nothing else. Its run stays open and finishes on the revision it started on.
Decide fast, in the browser or the terminal
The inbox shows the item rendered for a person, the editable fields marked, the reason codes on reject and a diff
of any edit before confirming. It is keyboard-first (a approve, r reject, e edit), works on a tablet, and reads
in English and French. The same decisions are available from the CLI, for scripts and for people who live there:
millstream inbox list --inbox analysts
millstream inbox approve dl_8f3a2c --set /marginPct=18 --reason needsVisit
millstream inbox reject dl_91be07 --reason overpriced --text "Asking price 20% above the street"
Every decision records who, when, the reason and the diff of an edit. A decision names the version of the review the reviewer saw, so one escalated meanwhile is a conflict rather than a silent overwrite.
Rules that approve, people who check them
autoApprove passes the obvious items straight through and records them as approved by rule. A sample of them,
5% by default and never less than 1%, still reaches a person as a review sample. A sample decision only measures the
rule: it never blocks or recalls the item. When a rule stops being right, its sampled agreement drops and an insight
says so.
The number that shows learning
Review metrics come with the platform: decisions by outcome and reason code, time to decision, edit rate, auto-approval rate, sampled-rule agreement, and the intervention rate, the share of items reaching a review step that a person had to look at. It is the headline measure of whether a pipeline is learning.
Every decision teaches the pipeline
Each edit is attributed through lineage to the step that produced the value, never to a pass-through. Patterns turn into proposals:
| Pattern | Proposal |
|---|---|
| Rejections predictable from fields known upstream | A filter placed before the most expensive step, backtested to drop at most 1% of past approvals |
| Recurring edits to the same field | A lessons rule and examples, or a prompt change, on the step that produced it |
| Near-universal approval without edits | An autoApprove rule, still sampled |
| Free-text reasons no code covers | New reason codes for the step |
The drafting agent sees reason codes, reason text and edit diffs, never who reviewed.