CurrencyTransfer call copilot on Cloudflare + Jev: architecture
Status: final synthesis, 2026-09-24. Feeds the otto-plan PRD. Built from the three proposals in docs/architecture/ (poc-fastest as the spine; production-shaped's guarantees and data-first's engine grafted on) and the three judges' verdicts. Citations are absolute paths under /home/stevan/dev/jev/; guide §n = docs/jev-guide.md, dissection §n = docs/research/reference-copilot-dissection.md, brief §n = docs/research/ct-domain-brief.md, corpus §n = docs/research/corpus-and-curation.md, stt §n = docs/research/stt-options.md, prior-art §n = docs/research/prior-art.md. Anything not cited is marked ASSUMPTION; numbers marked est. are estimates, not measurements.
1. Summary
- Milestone 1 is a replay copilot: pick a historical Aircall call or upload a recording, press play, and watch a dashboard update utterance by utterance with CT signals, stage, must-say checklist, risk flags, open concern and a next-best-move card drawn from pre-approved lines.
- One TypeScript engine,
step(state, utterance, answers, policy), pure and side-effect-free, runs in a Durable Object per replay/live session in M1 and in a Queue consumer for corpus scoring in M2. Same state, same questions, same gates in both. - Jev only judges: one speculative fan-out request per decision point (
env.AI.run('typesafe/jev')), ~9.6k input tokensest.at the measured ~3.5 chars/token (§6.2), soft budget 11k with a drop order, hard cap 12k against Cloudflare's 32k limit. All arithmetic, thresholds, memory and hero metrics are code. - Historical calls are ingested from the call-coach exports with no STT:
role internal|externalalready gives rep/client.@cf/deepgram/nova-3is used only for new uploads; Aircall recordings are mono, so diarisation plus a code/Jev/admin speaker map is the path (stt"Live verification"). - Policy (question banks, playbooks, weights, thresholds, checklist, risk rules) is versioned, immutable JSON with three separate hashes. Raw Jev answers are stored keyed by
bank_hash+state_hash; move/phrasing answers separately byplaybook_hash; derived decisions bypolicy_hash. Re-weighting is free; a playbook line edit never invalidates judging answers. - Hero metrics per Stevan's decision: Onboarding = must-say checklist completeness (rep-turn locks only) + latched risk flags; Customer Success = resolution state machine (reported → owned → client-accepted) + a secondary call-health composite. No closing probability anywhere.
- Rep-facing text is a typed union: an approved playbook line addressed by
text_id+ policy version, or a tailored rewrite that passed the single-condition Jev verifier nouls inpolicy/src/verify.json(eight as of 2026-09-25, incl.improves_rate_or_offers_discountandexplains_spread) and a code denylist, delivered off the critical path on a droppable message. The policy loader hard-rejects any bundle containing[VERIFY]. - Curation starts in M1: the engine emits Jev-tagged moments (must-say locks, risk flags, concern episodes, uncertain-band decisions) and the replay page has a one-form model/acceptable/avoid mark. M2 adds corpus-wide scoring, the review queue ordered by information value, promotion to examples and playbook lines, and an enforced publish gate.
- Compliance: pseudonymise in code before D1, Jev or the browser (
[REP],[CLIENT],[IBAN], ...); LLM coach text never enters D1; dedicated AI Gateway with body logging off; Cloudflare Access from day one on the Worker and the docs site (email policy for Stevan, a service token for the scripts); audit log with the Access identity. Client-derived text goes totypesafe/jevonly for the four exported samples until Stevan answers the DPA question (§13). - Cost
est.$0.022 per replayed 7-minute call in Jev credits, ~$23 per 1,000 historical calls, ~$70-75 for the whole usable corpus; nova-3 adds ~$0.04 per uploaded 7-minute call. Latency per decision 0.44 s over REST measured today, ~0.8 s via the binding with cold start; target p95 utterance-to-card ≤ 1.5 s.
2. Design stance
The three proposals scored within three points of each other; the judges converged on the same synthesis: poc-fastest's M1 spine (replay first, browser-owned clock, minimal machinery, three non-negotiables), production-shaped's enforced guarantees (hash split, budget drop order, typed rep-facing text, [VERIFY]-rejecting loader, gated publish, reconnect history, audit log) and data-first's engine and curation shape (pure step(), rep-turn-only must-say, decomposed verifiers, moment kinds, information-value review ordering, noul-first hero metrics, calibration-derived thresholds).
What this document optimises for, in order:
- Stevan's milestone 1 as written: upload or pick a call, transcribe if needed, replay it in sync with a dashboard. Everything that is not on that path is a named slot in M2 or M3, never a prerequisite.
- No corner-painting. Four things are non-negotiable even in the PoC because retrofitting them is expensive: (a) every raw Jev answer is stored with its hashes, model id and the exact pseudonymised state; (b) banks, playbooks, weights and thresholds are versioned data, not code constants; (c) transcripts are pseudonymised before they reach D1, Jev or the browser; (d) the engine is a pure function so batch scoring is configuration, not a rewrite.
- Compliance by construction, not convention. Where the judges found a badge, a prose
not_for, or a "will not block on verifier error", this design uses a type, a code rule, or a hard drop. - Jev hygiene from
guide §7: literal single-condition nouls with alignedtrue/falsecriteria; no counting, arithmetic or dates asked of Jev; small filtered state; anoneoption on every Choice paired with an absolute noul; thresholds never carried between question types;response.modellogged on every answer because Cloudflare does not pin versions (guide §8).
What it trades away in M1: no admin console beyond the mark-moment form, no corpus-wide scoring, no CT-DB pre-call brief, no Vectorize, no live audio, no weights editor in the UI, rewrite path built but off. One piece of M2-serving infrastructure is paid for early on purpose: the answers/move_answers hash split (§5, D3) exists so that an M2 playbook-line edit does not invalidate judging answers; M1 has no playbook editing, so nothing in M1 exercises the split, and reviewers should read it as a retrofit-cost decision, not as something M1 validates.
3. System overview
flowchart LR
subgraph devbox
EXP[("call-coach exports<br/>call_records / call_turns .jsonl.gz")]
IMP["scripts/import-call-coach.mjs<br/>select · stitch v1 · redact v1"]
EVAL["scripts/eval.mjs<br/>L1 L4 L5 L6 → eval/reports"]
EXP --> IMP
end
UP["Browser upload<br/>mp3 / m4a / wav"]
subgraph CF["Cloudflare account 694e…9ce9 — Cloudflare Access on every hostname"]
W["Worker jev-copilot<br/>HTTP /api/* · WS upgrade · static dashboard"]
DO[("Durable Object CallSession<br/>step() per utterance · WS · seek cache")]
D1[("D1 copilot<br/>calls · utterances · answers · move_answers<br/>decisions · policy_versions · moments · marks<br/>labels · eval_runs · audit_log")]
R2[("R2 copilot-raw<br/>raw/ audio · aircall json · nova-3 json<br/>eval/ runs · golden/")]
AI[["Workers AI<br/>typesafe/jev · @cf/deepgram/nova-3"]]
GW["AI Gateway jev-copilot<br/>body logging off · spend limit"]
PG["Pages ct-copilot-docs<br/>architecture · demo script · eval reports"]
Q[["Queue score-calls<br/>M2: batch scorer = same step()"]]
end
IMP -->|"POST /api/calls/import"| W
UP -->|"PUT /api/calls/upload"| W
W -->|"audio → REST"| AI
W --> D1
W --> R2
W -.->|M2| Q
Q -.-> DO
DB["Browser dashboard<br/>owns the clock: audio element or virtual timer"] <-->|"WebSocket /ws/calls/:id"| DO
DB -->|"GET /api/calls/:id, /audio"| W
DO -->|"one request per decision point"| GW
GW --> AI
DO -->|"answers · decisions · moments"| D1
DO -->|"read policy + call once"| D1
EVAL -->|"POST /api/evaluate"| W
EVAL --> PG
Data flow for one replayed utterance: the browser passes utterances[i].t_end and sends {type:'utterance', i}; the DO enqueues it on its serial chain (§4), builds the filtered state, and looks up (call_id, i, bank_hash, state_hash) in answers and (call_id, i, playbook_hash, state_hash) in move_answers. Cache lookup is per hash. Both hit: no Jev call. Both miss: one fan-out request. Judging hit + move miss (the normal case after an M2 playbook edit): a smaller moves-only request carrying next_move plus the code-allowed phrasing:: Choices against the same state_json (~2k tokens est.), stored in move_answers. Move hit + judging miss is treated as a full miss, because allowedMoves() depends on the judging answers. Then: apply the token budget, store the raw answers, run step(), persist SessionState, the decision and any moments, and push one decision message. A rewrite, if enabled, is enqueued and arrives 1-3 s later as a separate rewrite message or not at all.
4. Components on Cloudflare
Worker jev-copilot. Stateless front door, TypeScript, plain fetch router (Hono optional). Responsibility: Cloudflare Access JWT check (Cf-Access-Jwt-Assertion validated against the team's certs and the application AUD; the identity, email for a person or common_name for a service token, becomes audit_log.actor and sessions.actor), POST /api/calls/import (canonical rows from the import script), PUT /api/calls/upload (raw binary body streamed to R2 → nova-3 → stitch → redact → D1, §7), GET /api/calls, GET /api/calls/:id (pseudonymised utterances from D1), GET /api/calls/:id/audio (from R2 with HTTP Range support: parse Range, call env.R2.get(key, { range }), answer 206 with Content-Range, Accept-Ranges: bytes, Content-Length and the stored httpMetadata.contentType, 416 on an unsatisfiable range, HEAD supported; a plain new Response(obj.body) is a 200 without ranges, which means no seeking in Chrome/Firefox and no playback at all in Safari), GET /ws/calls/:id (WebSocket upgrade forwarded to the DO), POST /api/evaluate (the existing jev-lab shape in src/index.ts, kept for the eval script), POST /api/moments, GET /api/policy/:scenario, and the dashboard as Worker static assets (assets binding). Why: the only compute primitive with the AI binding; one origin for HTTP, WebSocket and static files means one Access application and no cross-site cookie work in M1 (§15, D9). That Access application has two policies: Allow (email = Stevan) and Service Auth (service token jev-scripts), because scripts/import-call-coach.mjs and scripts/eval.mjs run from devbox as non-browser clients and an email-only policy answers them with the login page; the scripts send CF-Access-Client-Id / CF-Access-Client-Secret, and the Worker accepts the resulting service-token JWT (common_name, no email). The token pair lives in ~/.config/jev/cloudflare.env next to the API token, never in the repo. Authorisation in M1 is the Access gate alone: every Access identity can read the whole corpus, which is acceptable while the only identity is Stevan's. Before Access is extended to RMs or managers (§13 Q5), GET /api/calls, GET /api/calls/:id and GET /ws/calls/:id must filter on calls.rep_ref (already in the schema) against the identity, or Stevan records full-corpus read as the tenancy model; the route list has no per-call authorisation today and this document says so rather than implying one. Limits that matter: 100 MB request body bounds uploads (stt §3.3); 128 MB memory per isolate, which is why uploads stream and are never buffered (§7); 30 s CPU default per invocation, so nothing loops over Jev calls in the Worker; wrangler dev bills real AI usage (guide §4.2), so unit tests stub env.AI.
Durable Object CallSession (SQLite-backed). One per (call_id, session_nonce), so two tabs replaying the same call do not share state. Holds the rolling window, persisted facts and checklist, concern state, resolution machine, EMA'd health, stage hysteresis state, card state, the pinned policy version, and the WebSocket. Serialisation is explicit, not assumed. A Durable Object's input gate closes only while a storage operation is pending; while the object awaits env.AI.run or a fetch (0.4-0.8 s) further webSocketMessage events are delivered and their handlers interleave. At 5-20× the browser sends an utterance every 0.1-0.6 s, so without a queue step i+1 would build its state and state_hash before step i's answers and persisted facts existed, corrupting the feedback chain, the seek cache and the "catching up" count. The DO therefore keeps an in-object promise chain (this.tail = this.tail.then(() => this.process(msg))) and a pending counter: utterance, seek and set_weights are enqueued on the chain; ping, ack_card, dismiss_card, mark_moment and flag_false_positive bypass it; pending drives the queue message; seek clears the chain before it runs; nothing else in the DO awaits Jev outside the chain. (The reference is serial only because one asyncio loop awaits handle_message per received frame, server.py:236-243; poc-fastest's "processed serially" was not a DO guarantee.) Hibernation loses memory, so state is stored per step. With the WebSocket Hibernation API the object is evicted from memory whenever it is idle for more than a few seconds while the socket stays open, and the constructor runs again on the next message; none of the in-memory fields above survive that. SessionState (≤ ~50 KB) and the session's decision list are written to ctx.storage (SQLite) at the end of every step(), in the same turn as the D1 write is issued, and the constructor reloads them inside blockConcurrencyWhile; the session nonce, call_id and pinned policy version are attached to the socket with serializeAttachment so a hibernated socket resumes without a new load. Because the browser owns the clock there is no server-side timer chain and no alarms, so nothing conflicts with hibernation and it costs nothing (the judges flagged this conflict in production-shaped and data-first). On reconnect it replays the decision timeline from the same storage so a tab reload never loses the session (dissection §7a fixed). Limits: 10 GB SQLite per object (we keep < 5 MB), soft 1,000 req/s per object, 32 MiB received WS messages (corpus §5); irrelevant at one decision every 2-6 s.
Session load and resume (COPILOT-035). GET /ws/calls/:id?session=<nonce> (src/routes/ws.ts) runs Access, then forwards the upgrade to idFromName("<call_id>:<nonce>") with the verified actor in X-Copilot-Actor, a header the Worker always overwrites. The object accepts the socket through the Hibernation API with the attachment {call_id, nonce, policy_version, actor, session_id, eval_uid, debug} (session fields filled in by load). Frames are parsed by src/session/protocol.ts and dispatched through HANDLERS (src/session/handlers/index.ts, one file per type; a type with no handler gets error{not_implemented}). A fresh load reads the call (status not ready → error{scenario_unsupported|needs_review|processing}) and the published policy, or the named policy_version in any status. It then inserts sessions and audit_log{session.start} in one batch, ends the session it replaces, and resets storage. load{resume:true} on an object that already holds a session for the call keeps it, clears ended_at and re-reads only the call and the pinned version from D1. Both answer hello{…, history, last_i, transcript_rev}. ctx.storage (src/session/store.ts) holds meta (identity, pinned hashes, eval_uid, debug, transcript_rev, last_i, gen), and state (SessionState), written in one multi-key put after the D1 decisions row; since COPILOT-037 this is only a cache (see below). An utterance{i} with i ≤ last_i re-sends the stored decision, and i > last_i + 1 gets error{out_of_order}. webSocketClose sets sessions.ended_at once no other socket carries the session. Until COPILOT-039 the step is a placeholder: step() with no answers (one unknown decision per utterance), TEST_TASK_DELAY_MS, and with debug a debug{i, request:{state, state_hash}} echo. scripts/ws-probe.mjs drives it live or --local; later stories register flags in scripts/lib/probe-flags/index.ts.
Serial chain, generations and the revision guard (COPILOT-036). src/session/chain.ts SerialChain is the chain above made concrete, with one refinement to the list: ack_card and dismiss_card mutate SessionState (card state), so they are enqueued with utterance, seek, set_weights and load; only ping, mark_moment and flag_false_positive bypass it (their D1 writes need no ordering against a step; flag_false_positive queues its state change, see COPILOT-042 below). enqueue(task) captures gen at enqueue time and runs the task only if gen is still current (otherwise it counts as skipped); errors go to sendError, so the chain never stays rejected. pending (queued + running, in memory, so 0 after a rehydrate) is broadcast as queue{pending} on every change. A fresh load and a seek bump gen in webSocketMessage the moment they arrive, before their own task is queued: queued work is skipped, and the task in flight calls host.check(gen) after every await, which throws Abandoned (counted, never reported) before its next write. Every derived D1 write goes through derivedBatch(db, call_id, pinned_rev, build) (src/db/d1.ts): revProbe first, then statements that each append revGuard, in one batch. A probe with no row means a replacement committed first: the task throws SessionError('call_replaced'), does not touch storage, and the session sends error{call_replaced} once and is stale (further queued messages other than load are dropped; a resumed hello repeats the error) until a fresh load. A probe with a row and 0 changes is a legitimate no-op. The D1 batch is the commit; the storage put after it is a cache write. A reset that arrives after a batch was submitted cannot undo it, so the reset path truncates: a fresh load deletes the replaced session's decisions in its own batch (a seek deletes i > seek_i). There is no read-then-write revision check under src/session/ (grep test in tests/unit/session-chain.test.ts). scripts/ws-probe.mjs --burst N [--reset-after K] exercises it live.
Checkpoint and recovery: D1 is the commit point (COPILOT-037). src/session/checkpoint.ts persistDecision writes each decision with session_state_json (the post-step SessionState, ≤ 50 KB, a larger one is logged) in the guarded batch above; INSERT OR IGNORE on decision_id = session_id:i makes re-running the same i after a crash a no-op. The constructor's blockConcurrencyWhile reads only meta (the session's identity) from ctx.storage and calls rehydrate(env, meta): one D1 batch reads the latest decisions row (ORDER BY i DESC LIMIT 1, → SessionState and last_i), the session's rows in i order (→ hello.history), the session's card.ack/card.dismiss audit rows, the call's false_positive labels and revProbe. The acks (→ card.acked) and labels (→ dismissFalsePositive) are re-applied on top of the checkpoint. Re-applying is idempotent, so all of them are replayed rather than only those after the checkpoint, which would race a bypassing flag_false_positive against a step in flight. A label hits an item only as it stood at the label's i: a risk lit, a must-say locked or a concern raised after it keeps its newer, undismissed state. No probe row → the session is stale and a resumed hello is followed by error{call_replaced}. With no decision row the session starts empty (initialState, last_i = -1): the cache is never read for state, so a reset or a forced replacement that deleted the rows cannot resurrect it. The cache is then rewritten from the rebuilt session. hello.last_i is the last DECISION index; the browser resumes at last_i + 1 and re-sends the transcript-only utterances after it (they write nothing), so the window is rebuilt exactly. pong{rehydrated, last_i, attachment} reports rehydrated: true on the first message after a constructor run that restored a session. TEST_FAIL_AFTER_D1='1' (honoured only with JEV_MODE=stub) throws between the D1 commit and the cache write. scripts/ws-probe.mjs --idle S --then-ping --reconnect checks a real hibernation live.
Request execution: budget, hash, per-hash cache, one request (COPILOT-038). src/session/execute.ts executeDecision(ctx, u) takes an explicit ExecContext {env, judge, policy, session, state} (no socket, so it is testable alone). A transcript-only utterance returns {transcript_only: true}. A decision point runs allowedMoves(s, {}, p) on the pre-step state (no answers yet, so no features: a forced move comes from a held fact only), then planRequest (budget first; may drop questions or shrink the window to 8), then stateJson/stateHash of the plan's state (with uid for eval sessions). The planned questions split by isMoveQuestion (src/engine/request.ts): next_move, phrasing::, covered::, fits::, situation:: are cached in move_answers under playbook_hash, the rest in answers under bank_hash. A stored row is a hit only when its question_ids cover every id this request needs; the smallest superset wins, so a row stored after a budget drop is a miss for a later request that needs more, which adds a second immutable row beside it. Both hit: no request (cache:'hit'). Judging hit + move miss: a moves-only request with the planned move questions against the same state (the full request's plan, so the state hash cannot move). Otherwise the full request (a move hit alone is a full miss). The judge is sessionJudge(env) = makeJudge(env, d1Sink(env)), so every attempt writes a jev_requests row with session_id, call_id, i; cf-aig-metadata is {call_id, policy_version, mode:'replay'}. On success one derivedBatch writes INSERT OR IGNORE rows (qset_hash = sha256 of the sorted ids, question_ids, state_json and budget_json with request_chars and window on answers) under revGuard and re-reads them by their full key in the same transaction; the STORED answers are consumed, so two sessions racing on one key both use the winner's row. A failed judgement (incl. malformed, e.g. a missing id) is never cached and returns error with empty answers and null hashes. skip returns {skipped_budget} and sends nothing. ExecContext.stub (honoured only with JEV_MODE=stub) appends __stub controls after hashing; the stub records stateHash over the same JSON.stringify bytes.
Decisions, commit order and the error policy (COPILOT-039). src/session/handlers/utterance.ts: a transcript-only utterance (decision_point = 0) goes through observe (amounts + window), advances the in-memory cursor and the storage cache, writes nothing to D1 and is answered decision{d: {i, transcript_only: true}} (TranscriptOnly in protocol.ts; re-sent the same way for i ≤ last_i). A decision point runs executeDecision with sessionJudge(env), then step() and emitMoments(), then commits: (1) persistDecision (src/session/checkpoint.ts), ONE guarded batch = revProbe, INSERT OR REPLACE decisions (decision_json, session_state_json, shown_json = {text_ids, policy_hash, bank_hash, playbook_hash} from the card's approved lines, the four answer hashes from stateRef (all NULL for an unknown decision), window, uid), INSERT OR IGNORE moments, and UPDATE sessions (unknown_count recounted from the decision rows, so a re-run cannot double it; model_drift = 1 latched); (2) only then the storage cache. A crash between them is recovered by re-running the same i (its answers are now a cache hit, so the same state and moments are re-written; only the telemetry says hit). Decision.plan.decision_id = session_id:i; Snapshot.budget = the number of drop steps; Decision.jev = {model, latency_ms, input_tokens, output_tokens, unknown, cache: hit|moves-only|miss|skipped, requests, errors, cum_cost_usd, budget: {estimate, dropped, window}}, the three totals carried from the previous decision (so they survive a rehydrate; cost = Σ input tokens × $0.042/M). With load.debug the sent decision also carries jev.last_request/jev.last_response (JSON, each cut at 32 KB) and a debug frame precedes it; neither is persisted. Error policy (INV-COPILOT-010): credits_empty → alert{credits_empty}, nothing written, host.stopped set (every chained message but load is dropped, so no further request; the next load clears it); any other failure → an unknown decision holding the previous snapshot; skipped_budget → alert{budget_skipped} plus the same held decision (jev.cache: 'skipped', budget.window: null); a model other than the expected one → sessions.model_drift = 1 and one alert{model_drift} per session (derived from the history, so a woken object does not repeat it). Under JEV_MODE=stub the expected model is the stub's stub-1, and STUB_FAIL_AT/STUB_402_AT/STUB_MODEL/STUB_DELAY_MS env vars become the request's stub controls.
Card actions (COPILOT-041). ack_card{card_id} and dismiss_card{card_id} (src/session/handlers/card.ts, with ack_card.ts/dismiss_card.ts as the registered handlers) run on the serial chain because they change SessionState. Both hide the move for the rest of the call (ackCard adds it to card.acked; allowedMoves then drops it, so the next decision shows another card or listening). They differ only in the audit action. A card_id that is not a move of the pinned playbook gets error{unknown_card} and nothing is written. The commit order matches decisions. The durable record comes first: audit_log{action:'card.ack'|'card.dismiss', subject: card_id, actor: the socket's actor, detail_json:{session_id, card_id, i: last decision}}. Only after it are card.acked and the ctx.storage cache updated. A crash between the two is recovered by rehydrate, which re-applies the session's card rows (COPILOT-037). There is no generation check after the audit write: an ack holds across a seek, so the in-memory state takes it too. Each action writes one row, even a repeated one, and there is no reply frame. hello.acked carries card.acked, so a reconnected dashboard knows which cards are hidden. TEST_FAIL_AFTER_AUDIT='1' (only with JEV_MODE=stub) throws between the audit row and the cache write. scripts/ws-probe.mjs --ack-first-card checks the flow live.
Curation writes: marks and false-positive labels (COPILOT-042). src/session/marks.ts holds the writes that mark_moment, flag_false_positive and POST /api/moments share. Each is ONE guarded batch (derivedBatch on the transcript revision the caller read; a replaced transcript → call_replaced, nothing written) carrying its own audit_log row. Free text is pseudonymised before storage (INV-COPILOT-004). note and tag go through redact() with the rep first names and the allowlist. A value-shaped token that survives (assertRedactedText) or any unresolved capitalised token refuses the mark with error{note_rejected, detail:{unresolved_count, reason: 'value_pattern'|'unresolved'}}. The text is never echoed; the operator rewrites the note without names. mark_moment{start_i, end_i, verdict, tag, note?} (bypass) writes INSERT OR IGNORE moments + marks{admin: actor, use_as_example: 0} + audit_log{action:'moment.mark', subject: mark_id, detail_json:{moment_id, call_id, start_i, end_i, verdict, kind, session_id}} and replies marked{moment_id, mark_id}. The tag picks the moment: a tag whose prefix names a moment kind (must_say, risk:rep_made_guarantee) files under that kind with the part after the colon as topic; any other tag files under uncertain. moment_id is the engine's deterministic momentId(call_id, kind, topic, start_i, policy_hash), so a kind:topic mark on an engine moment's span attaches to that moment, and a second mark on the same span and tag lands on the same moment. flag_false_positive{kind, id, i} (bypass) checks id against the pinned policy (risk_flags for risk, checklist for must_say, anything for the single concern; otherwise error{unknown_target}). It then writes labels{question_id: id, gold_json:{value:false, kind:'false_positive', target, id, i}, admin} + audit_log{action:'label.false_positive', subject: label_id}. The state change goes on the chain through host.enqueue: applyDurableMutations with that one label, the same function rehydrate uses, so live and rebuilt states agree. Its chain slot is reserved synchronously when the message arrives, before any await, and the task waits there for the write. So an utterance sent after the flag is always stepped after the dismissal, while the label write itself never waits behind a Jev call. The label is the recovery input: a crash, an eviction or a reset that skips the queued task loses nothing. There is no reply frame; the next decision shows the risk flag or checklist lock dismissed, or the concern's dismissed_since, and a client re-raise clears it. src/routes/moments.ts: GET /api/moments?call_id= returns the call's moments in span order, each with its marks (read-only, jev_json parsed as jev). POST /api/moments takes the mark_moment payload plus call_id for scripts, validates it with the same parseClientBody('mark_moment', …) as the socket, and writes the same rows under the scenario's published policy (201 {moment_id, mark_id}; 400 bad payload, 404 unknown call, 409 no scenario/policy or call_replaced, 422 note_rejected/out_of_range).
Seek (COPILOT-040). seek{i} (src/session/handlers/seek.ts, on the chain) is a reset: the generation moves on receipt, so an utterance in flight abandons at its next check and its decision is never sent. The handler refuses i ≥ n_utterances (out_of_range) and a missing session (no_session). It then runs ONE guarded batch (derivedBatch): DELETE FROM decisions WHERE session_id = ? AND i > seek_i, sessions.gen and unknown_count updated, and the call's false_positive labels read. After that, SessionState restarts from initialState, keeping only card.acked (an ack holds for the rest of the call), last_i = -1, and utterances 0..seek_i run through the same runStep as utterance. Every utterance runs, not only the decision points, because the window needs the transcript-only turns too. Each one re-sends its decision in order, and decision points INSERT OR REPLACE their own rows from the answer caches, so a seek over played ground costs no Jev call. After each step the labels are re-applied with applyDurableMutations, exactly as rehydrate does (idempotent, and scoped to the item as it stood at the label's i). A dismissed risk flag, lock or concern is therefore dismissed again from the step that re-lights it: that step's own decision shows it as it first appeared, and the state and every later decision show it dismissed. A reconnect after a seek sees only 0..seek_i in hello.history. flag_false_positive's chain slot is queued acrossReset (SerialChain.enqueue(…, {acrossReset:true})), so a seek that arrives right after a flag cannot skip it: the seek, queued behind it, reads the labels only once that write has committed. scripts/ws-probe.mjs --burst N --seek K checks it live.
Replay CLI and per-event chain contexts (COPILOT-040b). scripts/replay-ws.mjs --call <id> --scenario <s> [--speed 20] [--eval-uid X] [--policy-version N] [--local] replays a call the way the dashboard's virtual clock does (scripts/lib/replay-client.ts, npm ws, the Access service token on the upgrade). It sends load, then every utterance{i} at max(0.15, min(6, gap)/speed) s without waiting for answers. It prints {i, cache, model, input_tokens, latency_ms} per decision and a summary {decision_points, requests, cache_hits, errors, models, max_input_tokens, p95_latency_ms, final_hero}. eval_uid enters the hashed state, so the same uid replays from the answer caches and a new one pays again. Because a fast replay keeps the chain from ever draining, SerialChain.enqueue starts each task on a gate promise created in the enqueuing message's own I/O context. Chaining directly on tail ran every task in the first message's invocation, because workerd resumes a continuation in the context that created the promise, and the live replay hit the per-invocation subrequest cap at i=121.
Wall-clock replay and the arrival log (COPILOT-118a). --wall-clock sends utterance k at the absolute deadline t0 + t_end_k / speed, t0 being the first hello: the first part goes at its own t_end, gaps are neither capped nor floored, and after a reconnect the re-sent utterances are already due, so they go at once and the rest keep their deadlines (no drift). The eval's 1× sample (3347356034, L6's e2e) always runs this way. Per decision point the client records {i, call_t, due_ms, sent_ms, arrived_ms} from t0; --log <file> writes them under a session header, the input scripts/presenter-sim.mjs --log reads. The summary's transport_after_part_end_p50_ms / _p95_ms are arrived_ms − due_ms (at 1×, receipt after the part's end in the call): receipt, not display. Real gaps are long enough for the object to hibernate between two utterances, and a woken session resumes at its last decision, so the client now handles error{out_of_order, detail:{last_i}} as the player does: it re-sends from last_i + 1 once per cursor, drops the frames about utterances already in flight, and counts resyncs in the summary. An out_of_order that the re-send does not cure is still an error. (The first live 1× run without this answered out_of_order to 42 of 44 utterances after an 11.7 s gap.)
Subrequest budget (COPILOT-058b). Root cause. On run l4l6-20260926 only the Jev binding call failed: session 148a3207-0ab9-422c-acc4-9af079c0a92e (CS-strong 3303259297 at 20×) has jev_requests rows for i 152-170 that read network: Too many subrequests by single Worker invocation, latency 0, three attempts each, while the same decisions' D1 writes, their jev_requests rows included, succeeded. The Worker runs on the Workers Free plan, whose limit is 50 subrequests per invocation (wrangler.jsonc sets no limits.subrequests, and Free cannot raise it). The invocation charged is not the message whose task makes the call. It is the most recently delivered event on the object (a WebSocket message, the upgrade or a close): each new message starts a fresh count, and everything the chain does after the client's LAST utterance arrived is charged to that one message, even the tasks earlier messages enqueued. COPILOT-040b's turn gate routes each task's JS continuations into its own message's context, but the platform's counter does not follow them. Two measurements show it. In run l4l6-20260926 the client sent its last utterance about 33 s into the session, and exactly 50 Jev calls succeeded after that before every attempt failed. The instrumented replay then used a build with this story's meter and the reconnect and guard switched off (commit 92a86fb, never merged), on synthetic-400 at 20× (session 28d05f11-4dda-461c-b99e-2fb05d12cd63). Up to i=353 every decision was charged to its own message (jev.subrequests.invocation = 2). From the last message's arrival on, one invocation climbed to 100 at i=418, which is 50 Jev calls and 50 D1 batches. The 51st Jev attempt (i=419) failed with the platform's error, as did every attempt after it (237 rows, i 419-521). The D1 batches of those same decisions succeeded up to a meter count of 416, so the platform held only the external subrequests (the AI binding) to the 50; D1 was not stopped by this limit. Cost of a decision. Before this story a fresh decision cost 5: a cache-lookup batch, the Jev call, a jev_requests insert per attempt (d1Sink), the guarded answers batch with its re-read, and the decision batch. A hit cost 2 (lookup, decision batch), and a decision that failed three attempts cost 8. After it, a fresh decision costs 2: the Jev call and ONE batch holding the guarded answers/move_answers inserts and their re-read, the decisions row with its checkpoint, moments, the sessions counters and the decision's jev_requests rows (the meter's buffered sink). A hit costs 2 (one read of the stored answers, the batch), and each retry costs one more (three failed attempts cost 4). A judging hit with a move miss (moves-only) costs 3: the judging read, the request and the batch. load, and rehydrate, read the call's cache keys once (CacheKeys: (i, state_hash, qset_hash) per bank and playbook hash, plus each qset's question ids, no answer text), so a miss makes no lookup read. In the batch the decision, its moments and the counters are guarded on the stored rows being the answers this session consumed (wonGuard). A session that loses the COPILOT-038 insert race therefore writes nothing from its own answers, and is re-stepped from the winner's rows in a second batch; that is the only third subrequest. Meter. src/session/subrequests.ts SubrequestMeter keeps the platform's books. delivered() on every event opens a new invocation, and every subrequest is charged to the latest one, per decision, per invocation, per connection and per object lifetime. meter.db is the only D1 handle in src/session/ and meter.judge() the only judge (tests/unit/session-subrequests-grep.test.ts). The call and policy reads get the same handle through meteredEnv. Every decision carries jev.subrequests: {decision, invocation, connection}, including its own commit batch, and the replay summary reports max_subrequests_per_decision and max_subrequests_per_invocation. The meter counts D1 as well as Jev against the guard, the conservative reading, because the platform's treatment of D1 is observed, not documented. The guard sits at limit − 4. A batch that would take the charged invocation past it logs console.warn({invocation, used, limit}) once and leaves the pending jev_requests rows for the next batch with room. webSocketClose flushes what is left, in the same batch as sessions.ended_at, and a row that still cannot be written is logged: rows are deferred, never dropped (INV-COPILOT-005). There is no alarm or timer (INV-COPILOT-009). Design taken: reconnect. Charging every subrequest to the message that caused it is not possible from the object, because the platform attributes to the latest delivered event, and only a new client message starts a new count. So before a decision point the utterance handler asks meter.hasRoom(2). When the charged invocation cannot take one more decision under the guard line, the object sends reconnect{after_i} (after_i = the last utterance it handled), retires the socket (its queued utterances are dropped unanswered) and stops at that boundary. The client (scripts/lib/replay-client.ts; web/src/session.ts + app.ts for the dashboard) opens a new socket on the same nonce, sends load{resume:true} and re-sends from hello.last_i + 1. Each re-sent message is a fresh invocation, so no decision is paid twice and none is skipped. A seek re-steps 0..i inside one message and cannot reconnect midway. For a seek the guard defers telemetry, and a late seek on a long call is the noted follow-up. At 1× no backlog forms, and every decision runs in its own message's invocation.
Decision latency (COPILOT-058c). Measured before changing. Each sent decision carries jev.timing: {queued_ms, lookup_ms, jev_ms, step_ms, commit_ms, sent_ms}, when each stage ended in ms after the object received the message. The stamps use Date.now() in src/session/ (never in src/engine/**, INV-COPILOT-001). They are delivery telemetry, sent only and never persisted (hello.history and a re-send carry none). scripts/lib/replay-client.ts adds e2e_p95_ms and a p95 per stage (stage_p95_ms: queued, lookup, jev, step, commit, send, server = receipt → sent, network = e2e − server) to the replay summary. The Workers clock advances only across I/O, so pure compute (lookup without a read, step, send) reads 0 ms, and the real cost of that compute is under a millisecond. Before the change, build 289d7d5 at 1× on 3347356034 (lat-before-20260927, 22 decisions, errors 0) gave e2e p95 940 ms, split as queued 219, lookup 0, jev 571, step 0, commit 311, send 0, server 827 and network 40. The 2,503 ms of run l4l6-20260926 was measured before COPILOT-058b, when a decision cost five subrequests. After 058b the D1 commit (one batch, p95 311 ms) was the one server stage besides Jev and queueing, and the client waited for it. Design taken: (b) commit behind. The utterance handler issues the decision's one guarded batch, advances the in-memory session, and sends the decision while the batch runs, with commit_ms: null. The task then awaits the batch and only after it writes the ctx.storage cache. The chain's next task starts after that, so D1 order, generations, the revision guard (COPILOT-036), checkpoint and recovery (COPILOT-037), the error policy (INV-COPILOT-010) and the two subrequests per decision (COPILOT-058b) are unchanged. No timer or alarm is added (INV-COPILOT-009). A seek or fresh load that arrives during the commit is queued behind it and deletes or replays rows only once the rows exist. (b) was chosen over (a) provisional/committed/retracted because it needs no new message type and no restore logic in either client: both already replace a re-sent i (withDecision; the replay client keeps the first e2e) and resume from hello.last_i + 1 after a reconnect. A batch that fails after the send puts the session back at its last commit and sends reconnect{after_i} on the retired socket. The client's resume re-sends the utterance, which is decided again (from the answer cache when its answers were committed, otherwise by a new request) and replaces what it showed. A second failure on the same i is reported as error{internal}, so a D1 outage cannot loop paid requests. A failed revision guard sends error{call_replaced} after the decision the client saw, and leaves the session stale with nothing written at the old revision. The COPILOT-038 race re-step is sent again as a replacement. Tests: tests/workers/session-latency.test.ts covers TEST_D1_DELAY_MS and TEST_FAIL_D1_AT, honoured only with JEV_MODE=stub. After. Build 90d5bcc at 1× on 3347356034 (lat-20260927, 22 decisions, errors 0) gave e2e p95 452 ms (target ≤ 1,500), split as queued 164, lookup 0, jev 326, step 0, commit off the path, send 0, server 427 and network 31. The same run at 20× on 3303259297 (lat20x-20260927) had errors 0 and a maximum of 2 subrequests per decision. Provider latency varies between runs (Jev p95 571 before, 326 after), so the e2e gain is the commit plus that variance. One provider stall still costs two samples on a 22-decision call, because the next decision queues behind it (COPILOT-058's note).
Weight experiments: set_weights → recomputed (COPILOT-043). src/session/handlers/set_weights.ts, on the chain. It builds a transient policy: the pinned one with weights and thresholds overridden key by key, and weights_hash/policy_hash recomputed. The policy is never stored. An unknown threshold key, a wrong-shaped value or a weight that is not {kind, w} with a finite w is error{invalid_message}. One D1 batch reads the session's plan.purpose audit rows and joins the session's decisions rows (i ≤ last_i; a seek already deleted the rest) to the exact answers/move_answers rows each consumed, through the pinned bank_hash/playbook_hash plus the decision's own state and qset hashes. Two subsets asked at the same state are two rows, and the qset hash picks the referenced one. recompute(utterances[0..last_i], rows, p') rebuilds the timeline with zero Jev calls. Each snapshot goes out as a Decision with the stored decision's decision_id, budget and jev telemetry. The handler writes audit_log{action:'weights.experiment', subject: session_id, detail_json:{session_id, i, policy_hash, weights_hash, state_drift}} and replies recomputed{timeline, policy_hash, state_drift}. SessionState, the cache and the runtime policy are untouched, so the next utterance runs on the pinned policy. The session's plan.purpose overrides are replayed (recompute events, applied before the utterance they were made on). Not replayed: false-positive labels and card acks. A label that dismissed a persisted fact or an open concern changed the state Jev saw, so the decisions after it can count as drift even under an unchanged policy. scripts/ws-probe.mjs --set-weights '<json>' exercises it live.
The plan on the wire and the purpose override: set_purpose (COPILOT-091). Every decision.d carries the call plan (Snapshot.plan) with plan.decision_id = session_id:i, in live decisions, seek replays, hello.history and recomputed timelines alike. src/session/handlers/set_purpose.ts, on the chain, corrects the purpose. An id outside the pinned policy's plan.purposes is error{unknown_purpose} and writes nothing. Otherwise it writes audit_log{action:'plan.purpose', subject: purpose_id, actor, detail_json:{session_id, purpose_id, i, t}} FIRST, with i the latest decision (its utterance's t is the override's evidence). It then rebuilds that decision from its held snapshot with repurpose (src/engine/step.ts): overridePurpose, then repurposePlan re-templates the plan memory and re-derives the rows on the same decision. allowedMoves gets its features from snap.signals, so there is no Jev call and no answers read. The rebuilt decision is written over that i's row in one revision-guarded batch: decision_json, and session_state_json when that decision is still the last utterance. This is an UPDATE rather than persistDecision's INSERT OR REPLACE, so the row keeps the answer refs, window and uid that set_weights joins on. Then the handler updates the storage cache and the in-memory decisions (a reconnect's hello.history and a re-sent utterance{i} reproduce the sent decision) and sends decision{d}. rehydrate re-applies the session's newest plan.purpose row (reapplyOverride, idempotent), which recovers a crash after the audit write; the next decision then re-templates the plan. seek keeps an override the way it keeps acks, so the replayed plan follows its template from decision 0. Its evidence still names the decision it was made on. Known approximation: after a seek below the override's i, set_weights re-applies it at its recorded i, not from 0. scripts/ws-probe.mjs --plan / --set-purpose <id> exercise it live.
D1 copilot. Pseudonymised system of record for everything queryable across calls (§5). Why: cross-call SQL for the M2 review queue, calibration, rankings and label joins. Limits: 10 GB per database, 2 MB per row (answer rows ~3-4 KB), 100 KB per statement, 100 bound parameters per statement, which means at most floor(100 / columns) rows per multi-row insert: 9 rows for the 11-column utterances and answers tables (all three proposals copied "12 rows" from an 8-column example; a helper chunkForD1(rows, cols) enforces the real limit). Use batch() for the rest.
R2 copilot-raw. Private bucket, write-once blobs: raw/{call_id}/aircall.json (un-redacted normalised transcript), raw/{call_id}/coach.json (the LLM coach output including summary and quotes[].reason, which name clients), raw/audio/{call_id}.{ext}, raw/stt/{call_id}.json (nova-3 output), eval/{run_id}/…, golden/{policy_version}.json. Lifecycle rule on raw/ with a retention Stevan sets (§13). Why: PII blobs do not belong in a query store; unbounded size; cheap.
KV. Not used in M1. Policy versions are D1 rows; the DO reads the published row once per session and pins it. M2 adds policy:{scenario}:v{n} (immutable) and policy:{scenario}:current (pointer) as a read-hot mirror when the batch scorer reads the policy per invocation; KV's up-to-60 s eventual consistency is harmless for version-addressed keys (corpus §5).
Queues. Not used in M1. M2 adds score-calls (one message per call, consumer runs the same step() sequentially, idempotent on (call_id, i, bank_hash, state_hash), concurrency ≤ 8 until the Workers AI rate limit for Jev is measured, dead-letter after three failures) and rewrite-jobs (keeps the generator off the DO's critical path). The judges flagged the unmeasured Queue-consumer CPU assumption (~70 sequential fetches per invocation); the fallback is one message per 20 utterances carrying the engine state, and M1's cost/latency report measures the numbers that decide it.
Pages ct-copilot-docs. Hosts the deliverables per Stevan's decision: this document, the demo script with utterance numbers, eval reports, the cost/latency report. The project sits behind its own Cloudflare Access application (same email policy) from the first deploy, confirmed before deliverable 9 publishes anything: the demo script and eval reports name calls and rep tokens, and a Pages project is public by default. Nothing published here carries hosts, paths, keys or credentials; operational access details live in an ops runbook kept outside the Pages build. The dashboard itself is served by the Worker in M1 (one origin for WebSocket + Access); the M2 admin console moves the SPA to Pages under a custom domain with a Worker route on /api/* and /ws/* once a CT hostname is chosen (§13 Q6). Pages Functions are not used: the AI binding cannot be declared in wrangler config for Pages (guide §4.2).
Workers AI models. typesafe/jev via the binding with { gateway: { id: 'jev-copilot' } } as the third argument (guide §4.3); verified REST 400-550 ms and 0.44 s again today, binding ~800 ms including cold start; REST evaluation sits at result.result, so the client unwraps defensively (guide §4.7). If the binding turns out not to honour gateway selection or the per-request cf-aig-* controls (binding reference gap, guide §4.3), the DO switches to REST with fetch and an account token provisioned with wrangler secret put CF_AI_TOKEN (never in wrangler.jsonc, .dev.vars committed to git, or source), which scripts/jev.mjs already does from ~/.config/jev/cloudflare.env; tested on day 1. @cf/deepgram/nova-3 over REST with binary body and ?diarize=true&utterances=true&punctuate=true&smart_format=true&language=en-GB&numerals=true (verified: results.utterances[] with speaker/start/end/confidence, 8.6 s for a 9:46 mono Aircall call, 94% speaker agreement with Aircall labels, ~473 neurons/min ≈ $0.31/h; stt "Live verification"); add keyterm (CurrencyTransfer, GBP, EUR, SWIFT, IBAN, mid-market, forward, drawdown, safeguarding, beneficiary) and mip_opt_out=true (effect on the Cloudflare proxy unverified). Whisper is not used: no speakers (stt §3.1). Rewrite generator: ASSUMPTION @cf/meta/llama-3.3-70b-instruct-fp8-fast as the CF-native default, a policy field, swappable for a Haiku-class model through AI Gateway; behind a flag, off in M1.
AI Gateway jev-copilot. Dedicated gateway, never default: request/response body logging off (the default gateway stores Jev bodies regardless of ZDR, guide §4.3, §4.11), caching off (identical states are rare and a cached answer hides version drift), per-gateway spend limit set before any batch run, cf-aig-metadata {call_id, policy_version, mode} on REST calls. Billing: Unified Billing prepaid credits, 5% purchase fee, $0.042/M input pass-through, output free (guide §4.6, §4.10; keySource: Unified confirmed today). HTTP 402 code 2021 = empty credits: surfaced as an operator alert on the dashboard and in the Worker log, never retried (guide §4.8).
5. Data model
Principle: raw judgments, move/phrasing picks and derived decisions are three tables keyed by three different hashes, so each kind of policy edit invalidates only what it must.
-- corpus (pseudonymised text only; nothing from the LLM coach's prose)
CREATE TABLE calls(call_id TEXT PRIMARY KEY, source TEXT CHECK(source IN ('call-coach','upload')),
scenario TEXT, scenario_source TEXT CHECK(scenario_source IN ('call-coach','jev','admin')), scenario_conf REAL,
rep_ref TEXT, direction TEXT, recorded_at TEXT, duration_s REAL, engine TEXT,
stitch_version INTEGER, redact_version INTEGER, n_utterances INTEGER, n_decision_points INTEGER,
role_map_json TEXT, role_map_conf REAL, audio_r2_key TEXT, raw_r2_key TEXT,
coach_scores_json TEXT, -- numeric only: six 0-5 dims, call_value_score, quotes[].turn_idx (no reason text)
pd_ct_id TEXT, -- opaque join key for the M2 outcome join; ASSUMPTION: acceptable in D1
created_at TEXT);
CREATE TABLE utterances(call_id TEXT, i INTEGER, t REAL, t_end REAL,
speaker TEXT CHECK(speaker IN ('rep','client','unknown')), kind TEXT CHECK(kind IN ('turn','backchannel')),
text TEXT, words INTEGER, member_ids TEXT, redactions_json TEXT, decision_point INTEGER,
PRIMARY KEY(call_id, i)); -- 11 columns → ≤ 9 rows per INSERT
-- policy: immutable bundles, one row per version per scenario
CREATE TABLE policy_versions(scenario TEXT, version INTEGER, policy_hash TEXT, bank_hash TEXT, playbook_hash TEXT,
weights_hash TEXT, policy_json TEXT, status TEXT CHECK(status IN ('draft','evaluated','published','retired')),
eval_run_id TEXT, model_expected TEXT, created_by TEXT, created_at TEXT, note TEXT, PRIMARY KEY(scenario, version));
CREATE TABLE policy_pointers(scenario TEXT PRIMARY KEY, published_version INTEGER, published_by TEXT, published_at TEXT);
-- what Jev said (free recompute), split by what invalidates it
CREATE TABLE answers(call_id TEXT, i INTEGER, bank_hash TEXT, state_hash TEXT, model TEXT,
state_json TEXT, answers_json TEXT, input_tokens INTEGER, output_tokens INTEGER, latency_ms INTEGER,
cf_request_id TEXT, budget_json TEXT, created_at TEXT, PRIMARY KEY(call_id, i, bank_hash, state_hash));
CREATE TABLE move_answers(call_id TEXT, i INTEGER, playbook_hash TEXT, state_hash TEXT, model TEXT,
answers_json TEXT, created_at TEXT, PRIMARY KEY(call_id, i, playbook_hash, state_hash));
-- what code did with it (audit + timeline)
CREATE TABLE sessions(session_id TEXT PRIMARY KEY, call_id TEXT, mode TEXT, policy_version INTEGER, actor TEXT,
started_at TEXT, ended_at TEXT, model_drift INTEGER DEFAULT 0);
CREATE TABLE decisions(session_id TEXT, call_id TEXT, i INTEGER, policy_hash TEXT, decision_json TEXT,
shown_json TEXT, created_at TEXT, PRIMARY KEY(session_id, i));
CREATE TABLE rewrites(rewrite_id TEXT PRIMARY KEY, session_id TEXT, call_id TEXT, i INTEGER, move_id TEXT,
candidate TEXT, shown INTEGER, rejected_reason TEXT, verification_json TEXT, llm_model TEXT, policy_hash TEXT, created_at TEXT);
-- curation (M1 creates and seeds; M2 fills)
CREATE TABLE moments(moment_id TEXT PRIMARY KEY, call_id TEXT, scenario TEXT,
kind TEXT CHECK(kind IN ('must_say','risk','concern','next_step','uncertain','trainer_pick','coach_seed')),
start_i INTEGER, end_i INTEGER, topic TEXT, jev_json TEXT, policy_hash TEXT, priority REAL,
review_state TEXT DEFAULT 'queued', created_at TEXT);
CREATE TABLE marks(mark_id TEXT PRIMARY KEY, moment_id TEXT, verdict TEXT CHECK(verdict IN ('model','acceptable','avoid')),
note TEXT, use_as_example INTEGER, admin TEXT, created_at TEXT);
CREATE TABLE labels(label_id TEXT PRIMARY KEY, call_id TEXT, i INTEGER, question_id TEXT, gold_json TEXT, admin TEXT, created_at TEXT);
CREATE TABLE eval_runs(run_id TEXT PRIMARY KEY, scenario TEXT, policy_version INTEGER, bank_hash TEXT, model TEXT,
passed INTEGER, l1_pass_rate REAL, l4_pass INTEGER, l5_max_std REAL, l6_max_tokens INTEGER, l6_p95_ms INTEGER,
cost_usd REAL, r2_key TEXT, created_at TEXT);
CREATE TABLE audit_log(id INTEGER PRIMARY KEY, at TEXT, actor TEXT, action TEXT, subject TEXT, detail_json TEXT);
The policy bundle (one per scenario):
interface Policy {
scenario: 'onboarding' | 'customer_success';
version: number;
model_expected: 'jev-1.13.0';
bank: Record<string, JevQuestion>; // judging questions only (nouls, scores, stage, concern type)
playbook: { moves: Move[] }; // {id, title, what, not_for, must_say?: string[], requires?: string[], blocked_by?: string[],
// rewrite_allowed: boolean, lines: [{text_id, text, status:'approved', approved_by, approved_at, source_moment_id?}]}
checklist: ChecklistItem[]; // {id: 'said_who_holds_funds', applies: 'always' | {when: 'client_type', is: 'corporate'} | {when_fact: 'docs_status_known'} | ...}
risk_flags: RiskRule[]; // {id, kind: 'noul'|'persisted'|'code', question_id?, min?, code_rule?, label}
weights: Record<string, { kind: 'noul'|'fact'|'score'|'code'; w: number }>; // CS health composite only
thresholds: { persist_fact: 0.70; signal_on: 0.60; concern_open: 0.60; concern_clear: 0.60; concern_max_age: 12;
concern_type_min_conf: 0.40; next_move_min_conf: 0.35; next_move_min_prob: 0.30; phrasing_min_conf: 0.30;
switch_margin: 0.12; confirm_updates: 2; card_cooldown_utts: 4; uncertain_band: [0.30, 0.70];
risk_flag: 0.60; stage_switch_margin: 0.05; stage_confirm: 2 };
ema_alpha: 0.40;
token_budget: { soft: 11000; hard: 12000; window: 16; chars_per_token: 3.5; // divisor measured, not assumed (§6.2)
drop_order: ['phrasing_beyond_top4', 'all_phrasing', 'window_to_8', 'skip'] };
ingest: { stitch_version: 2; redact_version: 1 }; // STITCH_V2 from COPILOT-106
}
Hashing rules (scripts/hash-policy.mjs, shared with the Worker):
bank_hash = sha256(canonical(bank))where canonical sorts top-level question ids but serialises everycriteriaobject and array in declared order, so a reordering of Choice options changes the hash (option order affects results,guide §3.5, §8). Changing one word of one criterion is a new hash (the consistency cookbooks'_rubric_fingerprintdiscipline,corpus §6b).playbook_hash = sha256(canonical(playbook)): covers move ids, order,what/not_for, line ids and text. Thenext_moveChoice and thephrasing::<move>Choices are generated from the playbook, so their answers live inmove_answersunder this hash. Editing an approved line, the main M2 curation action, invalidates onlymove_answers; every judging answer inanswersstays valid.weights_hashcovers weights, thresholds, checklist applicability rules (and, on a derived bundle,checklist_by_template, COPILOT-110h), risk rules, EMA alpha.policy_hash = sha256(bank_hash + playbook_hash + weights_hash + ingest versions).state_hash = sha256(state_json). Stored on everyanswersrow, together with the exactstate_jsonsent, so any answer is reproducible and the seek cache never serves an answer to a different state (the judges' fatal flaw in poc-fastest's cache).
Recompute semantics, stated honestly: recompute(policy') re-runs step() over stored answers with zero Jev calls. It is exact for weights, EMA, display thresholds, card gates and hysteresis. It is approximate for thresholds that feed back into the state Jev saw (persist_fact changes known_facts/checklist_done; concern_open changes open_concern), because the stored answers were given against the old state. The engine counts decisions where the rebuilt state differs from state_json; M2's weights editor shows it as "state drift: N of M decisions; replay to re-ask", and in M1 the count exists only in the engine's unit tests and the debug drawer. All three proposals claimed free recompute without this caveat; the reference has the same limitation in a worse form (it recomputes from features, dissection §3e).
6. Per-utterance decision loop
6.1 Utterance stitching from Aircall fragments
Evidence from data/samples/onboarding-strong-3339895706.{txt,json}: 159 fragments in 587 s, 118 overlapping the previous one (median gap −0.17 s), 71 of two words or fewer; shards like REP: to / REP: to that, / REP: It's it's not a pay even it's three like a to pay four. cut. interleaved with CLI: Mhmm.. The call-coach call_turns.data carries members and stitched_chain_len but chains are still short. A second, deterministic pass (stitch.ts, pure, versioned as stitch_version, unit-tested on the four samples) runs at import for historical calls and in the Worker for nova-3 output; it is never re-run live:
- Sort by
start; maprole internal → rep,external → client; anything else →unknown(masks speaker-specific nouls to 0 instead of mislabelling, dissection §10b.7). - Backchannel = ≤ 2 words after stripping punctuation, every word in
{yeah, yes, yep, no, mhmm, mm, uh-huh, okay, ok, right, sure, gotcha, exactly, absolutely, cool, fine}, sitting inside the other speaker's run (start < prev.t_end + 1.5 s). Stored askind='backchannel', shown dimmed in the transcript, counted on the enclosing utterance, kept in the state window (they carry acceptance signal), never a decision point. - Merge consecutive same-speaker non-backchannel fragments when
next.start − prev.t_end < 1.0 s(overlap counts), or< 2.5 swhenprev.texthas no terminal punctuation; merged length capped at 120 words;t_end = max(member ends);member_idskept for traceability. - Split anything > 120 words at sentence boundaries into sub-utterances sharing
t, sequentialt_end. - Decision point = a stitched
turnwith ≥ 4 words, or ending in?, or the last utterance of a speaker run. Everything else updates the transcript only. - Garbled cross-talk is a transcription artefact, left as is; L5 measures whether Jev is robust to it, and re-transcription with nova-3 from Aircall audio is the fallback.
STITCH_V1 = { mergeGapS: 1.0, mergeGapUnfinishedS: 2.5, backchannelMaxWords: 2, backchannelGapS: 1.5, maxWords: 120 } are ASSUMPTION, tuned by eye on the samples in M1 and on 20 calls in M2. Judge-verified expectation: a naive stitch of the strong sample yields ~94 turns (59% of fragments, ~18 words each), so 40-80 decision points for a 6-10 minute call is realistic.
STITCH_V2 (COPILOT-103, decision 1a) = v1 plus { splitMinWords: 6, splitMinS: 5, splitForceS: 12, splitLookaheadS: 3 }, so Jev is asked every 5-12 s during a monologue. Between steps 3 and 4 each merged turn is cut into sentence-sized parts by estimated time: a word's time is its share of its own fragment's real [start, end]. A part ends at a sentence end (. ? !) once it holds ≥ 6 words over ≥ 5 s. Past 12 s it ends at the first . ? ! ; , within 3 s more, else at the last word boundary inside that window, so no part exceeds 15 s (a single longer word excepted). A non-last part under 4 words joins its neighbour. The first part keeps the turn's t, each later part starts at the previous part's t_end, and the last ends at the turn's t_end. kind, member_ids (the turn's), the v1 maxWords split inside a part, decision points and output order are v1's. On the four samples it gives 128 decision points for 3339895706 (longest gap 14.97 s, p90 10.57 s), 164 for 3303259297, 70 for 3485591407 and 43 for 3347356034. tests/fixtures/stitched/<call>.v2.json is written by node scripts/make-fixtures.mjs --stitch-only, which re-stitches the committed samples fixtures without data/. The Worker and importers still use v1 until COPILOT-104d.
Curation export before a re-import (COPILOT-104a, INV-COPILOT-019). A forced import deletes every derived row of the call (DERIVED_TABLES), so node scripts/export-call-curation.mjs --call <id> (read-only SELECTs through with-cf-env + wrangler d1 execute copilot --remote --json; rules in scripts/lib/export-curation.ts) first writes tests/fixtures/curation/<call>.rev<transcript_rev>.json (kind curation): transcript_rev, stitch_version, pd_ct_id, the marked moments and their marks, the labels (gold_json parsed), every utterance's {i, t, t_end, speaker, kind, words, member_ids} without text, expected_curation: {marks, labels, last_mark_at, last_label_at} (the values COPILOT-104b's claim compares) and policy_at_export: {version, checklist_ids, risk_ids} of the scenario's published policy. It is taken from one consistent snapshot: the header (transcript_rev, content_hash, expected_curation) and the rows (hashed, rows_sha256) are read twice; the export is written only when both reads agree and each read's rows reproduce its header, else it retries (1 + 3 attempts, then exit 1). --sessions <sid,…> writes tests/fixtures/decisions/<call>.v<policy_version>.<sid8>.json (kind decisions): each decision's decision_json without its jev telemetry, plus the answers / move_answers rows it consumed, picked by the decision's hashes and its policy's bank_hash / playbook_hash (the answers key includes bank_hash, and the same state is often answered under two banks); state_json and request telemetry are left out.
The v1→v2 i remap (COPILOT-104f, INV-COPILOT-019). node scripts/remap-i.mjs --export tests/fixtures/curation/<call>.rev<N>.json --to tests/fixtures/stitched/<call>.v2.json prints, as JSON, the old-i → new-i map over every exported line, the unmatched old lines (none on the four committed exports) and the moments, marks and labels moved onto the v2 parts (pure core scripts/lib/remap-i.ts; --apply posts it, COPILOT-104c). v1 split parts of one turn share t and member_ids, so a later part's effective span starts at the previous same-turn part's t_end (3339895706 lines 89/90: [464.435, 500.045) and [500.045, 505.81]). A turn maps to the v2 turns of the same speaker whose member_ids intersect its own and whose [t, t_end) overlaps that span; a backchannel to the v2 backchannel with the same member_ids; a time belongs to the part whose [t, t_end) contains it and the last part owns its t_end. A moment's start_i/end_i go to the first part of its old start line and the last part of its old end line, moment_id is rebuilt by momentId() with the old policy-hash suffix (an id that momentId() does not rebuild from its own row is refused) and each mark follows its moment; a label goes to the matched part holding the old line's effective end, with labels.i and gold_json.i rewritten together and label_id kept, so applyDurableMutations still reaches a lock made on the v2 part.
The curation precondition on a forced re-import (COPILOT-104b, INV-COPILOT-019). A forced replacement of an existing call (force=1, expected_rev > 0) deletes its marks and labels, so when the call holds any, the body of POST /api/calls/import must carry expected_curation: {marks, labels, last_mark_at, last_label_at} as COPILOT-104a exported it, else 409 curation_required (with the stored values). The claim (UPDATE calls … WHERE call_id = ? AND transcript_rev = ?, the first statement of the one db.batch()) adds four (subquery) IS ? conditions: the count and the latest created_at of the call's marks (through its moments) and of its labels, equal to the quoted values, or to zero/null when none was quoted. A mark or label written after the export (or after the route's own empty check) makes the claim change 0 rows, every token-guarded statement is then a no-op, and the route answers 409 curation_changed with the expected and current values. A first import needs no expected_curation, and an unforced import is covered by the existing no-derived-rows condition. A replacement without pd_ct_id (or with null) keeps the stored CRM key (COALESCE). node scripts/import-fixture.mjs --call <id> [--stitch 1|2] --force --expected-rev N (default --stitch 2) wraps tests/fixtures/stitched/<id>.v<stitch>.json and adds pd_ct_id and expected_curation from tests/fixtures/curation/<id>.rev<N>.json; when that export is missing and the route answers curation_required, it exits 1 naming the file; --print-payload prints the body without sending it. scripts/import-call-coach.mjs --stitch 1|2 (default 2) picks the stitch rule of its plan (it quotes no curation, so it cannot force over a curated call), and scripts/demo-check.mjs --stitch N (default 2) checks the clock against tests/fixtures/stitched/<call>.v<N>.json.
The curation restore (COPILOT-104c, INV-COPILOT-019). POST /api/calls/:id/curation (src/routes/curation.ts, service token only: an email actor gets 403) re-inserts a call's marked moments, marks and labels after the forced re-import moved their i: body {expected_rev, moments[], marks[], labels[], policy_at_export?}, each row exactly its table's columns (jev_json, gold_json as the stored JSON text). Every row is validated before any write, and the first failure answers 422 invalid_row{table, index, id, field}: call_id is the URL's; moment_id is momentId() of its own row; mark_id/label_id are newId forms; each mark's moment is in the body; kind/verdict are the CHECK values; 0 ≤ start_i ≤ end_i < n and 0 ≤ labels.i < n at expected_rev; created_at is ISO 8601; admin is an email or a service-token common name; topic, note and every string in jev_json pass isPseudonymised; a key twice in the body is duplicate. gold_json must be exactly a FalsePositiveLabel whose i and id are the row's i and question_id; a must_say/risk id is a checklist/risk-flag id of the scenario's published policy, or, when the policy has retired it, one listed by the body's policy_at_export (restored unchanged and reported under retired in the reply and the audit row); a concern id is a concern type. A primary key already stored answers 409 conflict{table, id} unless the stored row belongs to this call and equals the posted one in every column, which is counted unchanged: a repeated post changes nothing. The writes are one derivedBatch guarded on expected_rev (stale: 409 stale_rev, nothing written) of plain INSERTs (a key written concurrently fails the whole batch: 409 conflict) plus audit_log{action:'call.remap_i', subject: call_id} with {inserted, unchanged, retired, policy_at_export_version}. node scripts/remap-i.mjs … --apply [--expected-rev M] [--local] posts restoreBody() of the remap (M defaults to the export's rev + 1, the re-import's); run it after the re-import, since before it the old rows still hold some of the new moment ids.
Uploads on STITCH_V2 (COPILOT-104d, INV-COPILOT-004). POST /api/calls/:id/ingest and POST /api/calls/:id/speaker-map stitch with STITCH_V2 and write stitch_version = 2, so a new upload and every speaker-map re-stitch (built from the stored, pseudonymised STT fragments, never the stored rows) store sentence-sized parts; an upload stitched earlier stays v1 until it is re-stitched. Both routes' forced paths delete marks and labels, so they carry COPILOT-104b's precondition through the shared helpers in src/routes/calls-import.ts (curationRefusal, curationCondition, lostToCuration): the body may quote expected_curation; a call holding curation with no quote answers 409 curation_required before any work, a differing quote 409 curation_changed, and the claim asserts the quote (or zero marks and labels) in-statement, so a mark written after the check makes the claim change 0 rows and the route answers 409 curation_changed with nothing written. The review view's speaker swap sends no quote, so on a curated call it now toasts "Speakers not swapped (HTTP 409: curation required)" instead of deleting the curation.
Pre-call facts (COPILOT-112, decision 2a, INV-COPILOT-020). migrations/0004_precall.sql adds precall_json TEXT to calls (what is known before the call) and to sessions (the snapshot taken at session start, 113). The shape is closed and checked in one place, parsePrecall(json, policy) (src/ingest/precall.ts, pure; object or stored JSON text): source in {signup, crm, demo} is required; client_type, residency, transfer_purpose, frequency, documents_status, current_provider, funding_account and timing take a label id of that slot of the pinned onboarding policy (residency's labels are the country ids); bank_country a fact_lists.countries id, only once the policy has that slot (110b); currency_pair two different fact_lists.currencies codes; amount {digits, code} (digits without separators or a leading zero). Anything else, any other key included, throws PrecallError (invalid_precall) naming the field, never the value: there is no free-text field. isPseudonymised is not the guard here: it reads SA as an unresolved name and 25000 as a digit run. The import payload takes an optional precall, checked by validateImportPayload(…, precallPolicy) and refused (invalid_precall, detail no_policy) when the caller passes no policy; the import route passes the published onboarding policy.
Pre-call write paths (COPILOT-112b). Two paths write calls.precall_json, and both take the service token only (an email actor gets 403 service_token_required) and source demo or crm only (signup is reserved for the M2 sign-up join: 422 precall_source_reserved). PUT /api/calls/:id/precall (src/routes/precall.ts) validates the body against the published onboarding policy (400 invalid_precall naming the field; 409 no_published_policy), replaces the stored value and writes audit_log{call.precall_set} with {policy_version, precall} in the same batch (404 for an unknown call, with no audit row). The import payload's precall follows the same rules, but an invalid one is the import's 422. A replacement without precall keeps the stored value (COALESCE, like pd_ct_id), and one with it replaces it; the import's audit detail gains precall_source. The precall is not in the content hash, so an unchanged, unforced import writes nothing: use the PUT. A session that has already started keeps its own snapshot (113). node scripts/precall-payload.mjs --call <id> --source demo prints a call's demo payload from the table in scripts/lib/precall-payload.ts, validated against the onboarding policy built from policy/src. At session start seedFromPrecall (113) turns the stored facts into the seed: every value at i: -1, and each seeded slot with a known_fact persists it, so a seeded purpose establishes purpose_known like frequency and the pair (COPILOT-116b), and a call purpose at ≥ 0.9 does the same at its evidence i. On a call seeded with a purpose, open_with_agenda and ask_about_the_transfer are off the card from the first decision, and pin_down_timing_and_amount and agree_next_step are eligible.
6.2 State sent to Jev and the token budget
Small, filtered JSON object; only fields a question names by backticked path (guide §7). Numbers that only code needs (talk ratio, minute, word counts) stay out of the state; they are code signals, not Jev inputs.
interface JevState {
call_facts: {
scenario: 'onboarding' | 'customer_success';
client_type: 'personal' | 'corporate' | 'unknown'; // M1: 'unknown' unless obvious; M2: from CT DB pre_call
stage_history: string[]; // last 6 distinct, code-compressed
known_facts: string[]; // persisted *_known ids
checklist_done: string[]; // persisted said_* ids
open_concern: { open: false } | { open: true; type: string };
resolution?: 'none' | 'reported' | 'owned' | 'resolved'; // CS only
};
recent_transcript: { t: string; speaker: 'rep' | 'client' | 'unknown'; text: string }[]; // last token_budget.window (16) utterances incl. backchannels
latest_utterance: { t: string; speaker: 'rep' | 'client' | 'unknown'; text: string };
uid?: string; // eval/stability runs only (cache buster, guide §4.3)
}
Token estimate at 3.5 chars/token, not 4: on Cloudflare the reference's 37-question request averages ~27.9k chars for ~7.9k usage.input_tokens (eval/cf-parity.json: 380,577 tokens over 48 requests), and guide §10 records the same ratio (~25 KB → ~7.5k). Over compact JSON (no pretty-printing; the judges measured the brief's pretty-printed onboarding bank at ~9.2k on its own), plus the ~300-token fixed overhead the guide infers (guide §4.10):
| Part | Onboarding est. |
CS est. |
|---|---|---|
| Overhead | 300 | 300 |
call_facts |
170 | 170 |
recent_transcript (12 × ~35 words) + latest_utterance |
800 | 800 |
| Shared nouls (18) + scores (3, levels flattened to strings) | 2,400 | 2,400 |
| Scenario nouls (23 / 12) | 2,630 | 1,490 |
| Stage Choice (10 + none / 9 + none) | 690 | 630 |
| Concern-type Choice (8 + none / 7 + none) | 400 | 340 |
next_move Choice (14 / 13 moves, full fixed list) |
1,140 | 1,090 |
phrasing::<move> for code-allowed moves only (≤ 8 × 3 lines) |
1,030 | 970 |
| Total (fan-out request) | ~9,600 | ~8,200 |
moves-only request (judging hit + move miss, §3): overhead + call_facts + transcript + next_move + phrasing:: |
~2,100 | ~2,000 |
Question gating comes first (gates.ts, COPILOT-081): policy question_groups (rules-weights_hash) leave a group's bank questions out of the request unless the group is active. The seeded rate group (the 16 rate questions of docs/policy/rate-objections.md) is asked only while an undismissed rate, fees or alternative_provider concern is open, or, from COPILOT-088, while a rate: plan item is current. step() reads an omitted question's answer as missing (null), never 0. The drop order below then runs on the gated request as a backstop. Budget enforcement in code before every request (budget.ts): estimate = ceil(JSON.stringify(input).length / policy.token_budget.chars_per_token) + 300, with chars_per_token = 3.5 in policy v1 and re-derived from Σ chars / Σ usage.input_tokens of the first replays (deliverable 9); a divisor of 4 under-reports by ~14%, so a request the estimator passed at 11.9k would really be ~13.5k and the "hard cap never exceeded" claim below would rest on a miscalibrated instrument, which is why the divisor is policy data and L6 checks measured usage.input_tokens, not the estimate. Soft budget 11,000: over it, apply the drop order (1) drop phrasing:: questions for allowed moves beyond the top 4 by prior/stage, (2) drop all phrasing:: questions, (3) shrink the window 16 → 8, (4) mark the decision skipped_budget, hold the previous snapshot and log. Code rules sit around it: the pre-judge questions of the item about to open go first (prejudge, COPILOT-108b; the policy turns the pre-judge on by listing prejudge in drop_order, and code drops it first wherever it is listed), then the fact value questions (fact_values, COPILOT-085), the covered:: nouls right after (1) (covered, COPILOT-089), and the situation gate's fits:: and situation:: questions together after (2) and before (3) (fits, COPILOT-095b; a dropped gate is unknown, so the item shows "Listening"). Hard cap 12,000 is never exceeded; a 4xx from an oversized request is a bug, not a retry (guide §2.5). The degradation is written to answers.budget_json and the decision message. Dropping whole phrasing:: questions is hash-safe: each such Choice has a fixed option set, questions are evaluated in isolation (guide §1), and omitting one changes no other answer. The next_move Choice always carries the full move list in fixed order, so its probabilities are comparable turn to turn and its answers all live under one playbook_hash (this repairs data-first's per-turn shortlist, which broke its own hash discipline).
6.3 Question-bank structure per scenario
The bank is the DRAFT in brief §4.3-4.5, kept in Stevan's format, assembled at build time by scripts/build-policy.mjs from four files: shared.json, onboarding.json or customer-success.json, the scenario playbook, and verify.json (rewrite verifiers, sent only in the verification request). One transformation is applied at build time: every Score level is flattened to one string ("Disengaged: one-word or one-line answers; deflecting ('just email me'); trying to end the call"), exactly as cf_safe_questions in scripts/replay_via_cf.py does, because Cloudflare's typesafe/jev wrapper returns HTTP 500 code 2002 ("Failed to parse model output") when Score criteria are {summary, signals[]} objects, and the brief's three Scores (engagement, urgency, trust, brief §4.3) are all written that way (guide §10). Nouls and Choices keep their structured instructions/criteria. bank_hash is computed over the flattened bank, and a unit test asserts that no built bank contains an object-valued Score level, so the first full request does not fail and the hash does not change on day 1. Nothing in the engine knows question names except through a signals map of id → role:
type Role = 'client_turn' | 'rep_turn' | 'either' | 'window_fact' | 'must_say' | 'risk' | 'score' | 'stage' | 'concern_type' | 'next_move' | 'phrasing';
interface SignalSpec { id: string; role: Role; lock_on?: 'rep' | 'client' | 'any'; weight?: number }
| Group | Onboarding ids | CS ids | Rule in code |
|---|---|---|---|
| Shared client-turn nouls | client_objecting, client_accepts, client_disengaging, client_confused, client_ready_to_book, next_step_agreed |
same | masked to 0 on rep turns |
| Shared either-speaker nouls | client_asked_about_{rate,safety,timing,fees,documents}, client_mentioned_alternative_provider, client_mentioned_deadline, rep_proposed_next_step |
same | unmasked |
| Shared rep-turn nouls | rep_asked_open_question, rep_explaining_at_length |
same | masked on client turns |
| Risk nouls | rep_made_rate_prediction, rep_made_guarantee (rep-turn) + window funding_from_third_party, jurisdiction_concern |
rep_made_rate_prediction, rep_made_guarantee |
≥ 0.60 on the turn / ≥ 0.70 persisted; latched |
| Window facts | purpose_known, pair_known, timing_known, frequency_known, funding_source_known, beneficiary_known, docs_status_known, current_provider_known, decision_maker_known |
issue_reported, upcoming_payment_known |
persist ≥ 0.70 on any turn |
| Must-say checklist nouls | said_who_holds_funds, said_rate_transparency, said_quote_lifetime, said_settlement_timing, said_fund_from_own_account, said_booking_is_binding, said_minimum, said_docs_needed, said_payment_reason, said_rm_contact |
issue_resolved_or_owned, rep_checked_status_with_specifics, said_forward_deposit_and_liability (conditional) |
persist ≥ 0.70 on rep turns only |
| CS-only nouls | — | no_upcoming_need, client_wants_to_wait_for_rate, client_asked_about_{forwards,alerts_or_orders,recurring,access}, rep_offered_tool |
per role |
| Scores (3 levels) | engagement, urgency, trust |
same | centred, health only |
| Stage Choice | onboarding_stage (10 + none: "Too little context yet") |
cs_stage (9 + none) |
hysteresis |
| Concern-type Choice | onboarding_objection_type (8 + none) |
cs_objection_type (7 + none) |
consulted only when client_objecting ≥ 0.60 |
next_move Choice |
14 moves, {what, not_for} |
13 moves | full list, code filter after |
phrasing::<move> |
≤ 8 × 3 lines | ≤ 8 × 3 lines | options are line ids with null description + the text, so choice is a verbatim id |
Two rules the judges insisted on, both now in code and in the L1 suite: said_* items lock only when latest_utterance.speaker === 'rep' (a client paraphrase such as "so you never hold my money?" reads the same window and would otherwise tick a compliance item; a one-question REST probe today returned 0.02 for that utterance when the instruction names the rep as speaker, but the code mask is the guarantee, not the wording); and the objection-type Choice is only read when the absolute noul fires, because a Choice always crowns a winner (guide §3.1). Every stage Choice gains an explicit none option absent from the draft. Amounts, dates and counts are never asked of Jev (guide §7); a regex amount extractor in code feeds the checklist's said_minimum applicability rule.
6.4 The step() engine and hero metrics in code
// engine/step.ts — pure; identical in the DO (M1) and the Queue consumer (M2). No I/O, no clock, no randomness.
export function step(s: SessionState, u: Utterance, a: Answers, m: MoveAnswers | null, p: Policy): { s: SessionState; snap: Snapshot; moments: Moment[] } {
const f = extractFeatures(a, u.speaker, p); // speaker masks; score/(levels-1); persisted facts clamp to 1
s = persistFacts(s, f, u, p); // *_known on any turn; said_* only if u.speaker === 'rep'; never un-persist
s = updateConcern(s, f, a, u, p); // open ≥ .60 on client turn; type if conf ≥ .40 else parent bucket ('money'|'process'|'unknown');
// `since` refreshed on every re-raise; clear on accept ≥ .60 && disengaging < .60, or 8 decisions after last raise
s = updateStage(s, a[p.stageQuestion], p); // switch only if challenger leads by stage_switch_margin for stage_confirm decisions
s = updateResolution(s, f, u, p); // CS: none → reported (client, issue_reported ≥ .60) → owned (rep, issue_resolved_or_owned ≥ .70) → resolved (client, client_accepts ≥ .60)
const risks = riskFlags(s, f, u, p); // latched booleans with the utterance index and value
const hero = p.scenario === 'onboarding' ? onboardingHero(s, f, p, risks) : csHero(s, f, p);
const card = pickCard(s, a, m, p); // allowedMoves() in code, then Jev's relative pick within them, then hysteresis
const plan = updatePlan(s, u, p, card, moves, a, m); // COPILOT-090: Snapshot.plan = { purpose, facts: factChips(s, p), items, caution }
s = pushWindow(s, u);
return { s, snap: { i: u.i, hero, risks, stage: s.stage, concern: s.concern, checklist: checklistView(s, f, p), card, plan, signals: f, health: s.health, budget: a.__budget }, moments: emitMoments(s, f, u, p) };
}
// Onboarding hero: a fraction and a list, never a probability.
function onboardingHero(s: SessionState, f: Features, p: Policy, risks: RiskFlag[]): Hero {
const applicable = p.checklist.filter(item => applies(item, s)); // e.g. said_docs_needed only while docs incomplete; said_minimum only if amount unknown or < 5k mentioned
const done = applicable.filter(item => s.persisted[item.id] !== undefined);
const uncertain = applicable.filter(item => !s.persisted[item.id] && inBand(f[item.id], p.thresholds.uncertain_band));
return { kind: 'onboarding', completeness: done.length / Math.max(1, applicable.length), done, uncertain, applicable, risks };
}
// CS hero: resolution state + secondary call-health composite with EMA (reference maths, CT weights; brief §4.6 draft).
function csHero(s: SessionState, f: Features, p: Policy): Hero {
const inst = clamp(0.5 + Σ(p.weights, f), 0.03, 0.97); // scores centred by (x − 0.5)·2; nouls raw; code signals: concern_open, talk_penalty
s.health = s.health === null ? inst : p.ema_alpha * inst + (1 - p.ema_alpha) * s.health;
return { kind: 'customer_success', resolution: s.resolution, resolution_at: s.resolution_at, health: s.health, raw: inst, moved: topDeltas(s, 3) };
}
Both hero functions are noul-first: nouls calibrate best on the independent test (ECE 0.012 vs 0.086 Choice vs 0.254 Score, prior-art §5), so no hero number is derived from a Score magnitude; Scores only enter the secondary health bar, centred so the middle level is neutral (guide §3.4).
6.5 Gates, thresholds, anti-flicker
All values live in policy.thresholds; defaults are the reference's, which are starting points, not tuned values (dissection §5a, §9d). M2 replaces them per question from calibration at a target precision (§9).
| Decision | Rule | Source |
|---|---|---|
| Persist a fact / tick a checklist item | window noul ≥ 0.70; said_* only on rep turns; never un-persists in a session |
reference FACT_PERSIST_THRESHOLD; data-first §6.3 |
| Light a signal | noul ≥ 0.60; 0.30-0.70 rendered "uncertain", not off | consistency cookbook band (guide §6) |
| Open concern | client_objecting ≥ 0.60 on a client turn; type from the Choice if confidence ≥ 0.40, else the parent bucket (money for rate/fees, process for documents/funding, unknown); since refreshed on every re-raise |
dissection §4b; classification-with-confidence cookbook |
| Clear concern | max(client_accepts, client_ready_to_book, next_step_agreed) ≥ 0.60 AND client_disengaging < 0.60; or 8 decisions since the last raise |
reference; "a disengaging turn never clears" |
| Stage | switch only if the challenger leads the current stage by ≥ 0.05 for 2 consecutive decisions; none never displaces a real stage |
call-coach-ai/decide.js via prior-art §2.1 |
| Show next-move card | next_move.confidence ≥ 0.35 and max(prob) ≥ 0.30, else "listening… (leaning X)" |
reference; 0.35 on a 14-option Choice ≈ 0.39 peak |
| Switch the card | challenger must lead by ≥ 0.12 or stay on top for 2 decisions; 3-decision cooldown after a switch; a rep-acked card is hidden for the rest of the call | decide.js |
Suppress a move (code, not not_for prose) |
move must_say already persisted; requires facts missing; blocked_by facts persisted; agree_next_step while a concern is open; check_serviceability_first forced to the top when jurisdiction_concern ≥ 0.60; concern open → shortlist restricted to that topic's moves. A suppressed top pick is shown greyed with "rule: X hidden because Y" |
dissection §5b gap; prior-art §2.1 |
| Highlight a line | phrasing::<move>.confidence ≥ 0.30 else the move's first approved line |
reference |
Points from a situation (COPILOT-083, COPILOT-095b, situation.ts) |
a move with situations[] (own or situation_refs) shows the approved points of the situation the current plan item's gate shows (pointsOf(move, shownSituation(s, p), p)), nothing when none shows or it is not one of the move's; only a move without situations shows its approved flat lines (≤ 3, phrasing:: highlight). All of the situation's points in order, no phrasing:: Choice (phrasingQuestions skips the move), highlight = the first point not yet covered |
docs/policy/rate-objections.md, docs/policy/like-for-like.md §3 |
Situation fit gate (COPILOT-095b, situation.ts, plan.ts) |
PlanMemory.fit = {item, move, candidates, lead?, fits, choice, asked_at?} for the current item. candidateSituations(item, s, p): the move's own situations (template points.move, ask:<move>, a concern item's move: the card's move on its topic, else the first allowed one) whose applies holds (planApplies: an unknown slot keeps it, an unknown fact does not), minus plan.rate_pressure.ask_first_situation / concession_situation; the rate item: the options of the rate Choice. fitsToAsk(s, u) (from s.plan only): on a client turn, and on the first decision after the current item changes (asked_at unset; also after a rate re-type, a new lead), up to FITS_MAX = 3 fits::<id> (all when ≤ 3, else the episode's situation, then a holding prior's ranked ones, then the previous Choice's probabilities, then playbook order) plus situation::<move> with 2 or more candidates (not the rate item, which keeps rate_objection_type). assembleQuestions(…, covered, fits) adds them last in full and moves-only. recordFits (in step() before pickCard, again in updatePlan, idempotent) replaces the item's answers when asked; retargetFit keeps them for the same item, clears them on a switch. shownSituationId: one candidate at fits ≥ thresholds.situation_fit (0.6); two or more: the Choice's pick (the rate item: the open episode's situation and conf) when not none, confidence ≥ situation_min_conf (0.4) and its own fits ≥ 0.6; else "Listening". Hysteresis (COPILOT-108a, decision 17, only with the optional thresholds.situation_fit_hard 0.3): SituationFit.shown / misses / miss_i; recordFits counts at most one miss per decision (an answer in [0.3, 0.6) or a dropped fits::), clears misses at ≥ 0.6, and clears shown at once under 0.3 into CallPlan.dropped_hard; shownSituationId keeps shown while misses < 2 unless rules 1-2 let another in. Pre-judge (COPILOT-108b, decision 3 R6, only while token_budget.drop_order lists prejudge): PlanMemory.prejudge is a SituationFit for the item that would become current next (the ranking's challenger, else the next open item; not the rate item); prejudgeToAsk asks its fits:: (≤ 3) and situation::<move> on client turns, a list of its own in planRequest (the prejudge drop step, first); recordPrejudge keeps the answers for that decision and the next; on a switch to that item retargetFit seeds its fits and choice from them and seededFit marks the situation they let in as shown, so the item opens with points. Segment priors (COPILOT-116, decision 2h, optional plan.priors, absent = M1): priorHolds(when, s, p) over knownValue (sell_in: the known sell leg is listed; residency_in_sell_country: the known residency is the sell code's first two letters); priorRanks(move, candidates, s, p) sets SituationFit.ranked on the gate (targetFit) and the pre-judge (targetPrejudge) via withRanks; fitIds orders the lead, then ranked, then the Choice's probabilities, then playbook order, before the cap of 3. It writes no slot, chip or answer. Only the current item shows a situation; PlanItem.headline / situation carry the shown one; the rate item keeps its do-not flags, its "Ask first" and concession terms show only with its situation. coveredToAsk asks on-screen points only; delivered counts only points that were shown. Stub: STUB_FITS rules over the latest client turn (0.9, else 0.05); situation:: picks the first option whose rule fires at 0.8, else none at 0.3 |
docs/policy/like-for-like.md §3 |
Rate episode (COPILOT-086, rate.ts) |
rate_objection_type is read only while an undismissed rate, fees or alternative_provider concern is open and was open before this decision (the rate group is gated on the state before the decision, so the episode opens one decision after its concern); a confident (≥ 0.40) answer other than none opens SessionState.rate = {situation, conf, start_i, t, open, concession}, a later confident different answer re-types it in place; a rep turn with rep_obtained_something_for_concession or rep_improved_rate_unasked ≥ 0.60 sets concession; it closes (open: false, end_i) when the concern clears, expires or is dismissed. rateView gives the plan item {situation, do_not_flags, qualify_first, concession}; the five do-nots latch through riskFlags on rep turns only |
docs/policy/rate-objections.md; DESIGN.md §Call plan contract "Rate pressure" |
Call plan (COPILOT-088, plan.ts) |
updatePlan(s, u, p, card, moves) keeps SessionState.plan = {template, items[{id, origin, state, raised_at?, done_at?, interrupted?, situation?, template?}], current?, challenger?, streak} and returns the PlanView. Template = plan.templates[purpose ?? 'other']; a purpose change keeps same-id items, done items the new template lacks and client items. A template item is done when every done_when holds (must_say/fact lock, slot filled, move_suppressed by a must_say: rule; delivered, COPILOT-089: every point covered, or one covered and the next client turn opens no concern on the move's topics, done_at = the latest covering rep turn), done_at = the latest condition's turn, never reopened; otherwise not_needed while applies fails (checklist grammar, when_slot, and when_amount_below_gbp also over the amount slot's GBP equivalent). An undismissed concern of a known type inserts concern:<type>:<start_i> as current where the call is (the interrupted item is next and reopens when it closes; done at the close; removed if dismissed); a rate episode turns it into rate:<start_i> in place. A client-turn lock of a plan.asks fact inserts ask:<move> after the current item. Current = inserted item, else the card's shown move, else the stage, else the previous current, else the first open item; a challenger needs confirm_updates (2) decisions in a row; inserts, closes and a done current switch at once; one change per decision. activeGroups also reads plan_item_prefix against the current item |
DESIGN.md §Call plan contract "Definitions" |
Call purpose (COPILOT-084, purpose.ts) |
the first plan.purpose_question answer (onboarding_purpose / cs_purpose) with confidence ≥ plan.purpose_min_conf (0.55) locks SessionState.purpose = {id, conf, i, t, source: 'jev'}, any option including other; Jev never re-locks it; overridePurpose (set_purpose) replaces it with source: 'override', never replaced by Jev; the Choice is left out of the request (omittedQuestions) once locked or after plan.purpose_max_decisions (12) decisions, and a stale answer to it is masked |
DESIGN.md §Call plan contract "Call purpose" |
Client fact values (COPILOT-085, fact-values.ts) |
a closed set of 12 plan.fact_slots, plus the optional bank_country (COPILOT-110b, onboarding only) (group, importance rank, source, trigger, known_fact?, known_at_conf?, approved labels or template); values come only from closed-set bank Choices (fact_<slot>, the currency legs over fact_lists.currencies) or span Choices (span_amount over the client's extractAmounts candidates ≥ 100: client turns' own and rep amounts the client's next turn confirms, COPILOT-058d; span_residency over the fact_lists.countries gazetteer; span_bank_country over the same gazetteer, only on a turn that also says bank/banks/banking, its chip the template "Banks in {country}" with the country id as a typed param rendered through display) whose request id is span_<slot>@<8 hex of sha256(candidate texts)> with c0…c5 taken from the post-budget window; asked only on client turns when triggered or pending (cap plan.fact_questions_max 3 by importance; left out by the cap or the budget → SessionState.fact_pending, asked while the evidence is in the window; planRequest drops them first, fact_values); an answer ≥ plan.fact_value_min_conf (0.60) and not none sets the slot, a different one replaces it with updated_at, the same keeps its first evidence; client_type sets SessionState.client_type; a value ≥ the slot's known_at_conf persists its known_fact at the value's evidence i (COPILOT-058e, documents_status 0.9 → docs_status_known; COPILOT-116b, transfer_purpose 0.9 → purpose_known); factChips orders by group, importance, recency, labels are approved or approved_template with typed params |
DESIGN.md §Call plan contract "Client fact values" |
Per-point said detection (COPILOT-089, plan.ts, build.ts) |
coveredQuestions(playbook, textIds) generates one noul per approved line or situation point, covered::<text_id> ("In the rep's turns in recent_transcript, did the rep make this point, in any wording?" with the point's text), like phrasing:: (answers under playbook_hash; mergeBanks refuses bank ids covered::…). coveredToAsk(s, u) reads only s.plan: rep turns only, the current item's uncovered points as shown (PlanMemory.shown, "Ask first" first) then, for COVERED_DWELL = 6 decisions after a switch, the previous item's (PlanMemory.dwell); at most COVERED_MAX = 3, least recently answered first (PlanMemory.asked). assembleQuestions(…, covered) adds them after phrasing:: in full and moves-only; planRequest drops them (code rule covered) right after phrasing_beyond_top4. updatePlan(…, a, m) records a point covered (PlanMemory.covered[text_id] = {i, t}) on the first answer ≥ persist_fact, never unset; a budget-dropped point (no answer) is not counted as asked. PlanPoint.covered_at / fresh in the view. Stub: 0.9 when the rep turn holds ≥ 60% of the point's content words, else 0.05. With the rate group asked the bank is ~10,975 estimated tokens at an empty window, so on rate-item turns the budget usually drops covered:: (open) |
DESIGN.md §Call plan contract "Per-point said detection", "Delivered" |
Situation data contract and fit questions (COPILOT-095, types.ts, build.ts) |
Situation carries client_examples: {text, source}[] (harvest, drafted, ct-rate-objection.pdf) and applies?: PlanApplies; SituationPoint carries slot_requirements? and rep_fills? (enforced by COPILOT-097, row "Slot requirements"); thresholds.situation_fit 0.6 and situation_min_conf 0.4. buildPolicy (assertSituations) refuses fewer than 2 client examples, an empty or unknown one, one failing assertRedactedText (INV-COPILOT-004), a headline over 8 words, an applies outside the plan grammar (when_fact persisted or in UNSOURCED_FACTS), and a point text_id used twice; assertPlanTemplates accepts a points.move whose only text is in situations. fitsQuestions(playbook, ids) (fits::<situation>, noul over what / not_for) and situationQuestions(playbook, moveIds, exclude) (situation::<move>, Choice over 2 or more situations then none) are generated like covered:: (answers under playbook_hash; mergeBanks refuses both prefixes). phrasingQuestions skips a move with situations. The engine gate that asks them is COPILOT-095b (row "Situation fit gate") |
docs/policy/like-for-like.md §2.1, §3.1 |
Slot requirements (COPILOT-097, fact-slots.ts, build.ts, situation.ts, plan.ts) |
assertPointPlaceholders: every distinct {…}/[…] token of an approved situation point is covered exactly once by slot_requirements[].placeholder or rep_fills; slot is in POINT_PARAM_SLOTS (currency_pair → currency, residency → country) or UNSOURCED_SLOTS (settlement_partner, last_transfer_month); leg sell/buy for currency_pair only; residency fills {country} only; fact_lists.countries[].display is required. pointText(point, s, p): no requirements → approved; all slots known → approved_template with params keyed by placeholder name (a leg's ISO code from pairLegs, the residency's country id), built by pointTemplate after checkPointParams; otherwise null and the point is left out (a situation with none left shows "Listening"). PlanPoint.line may be an approved_template; the card never holds one and leaves such points off. covered:: keeps the approved text with its placeholders. where_is_my_money is gated on the unsourced trade_status_known |
docs/policy/like-for-like.md §2.1, §3.8 |
The plan on every snapshot (COPILOT-090, step.ts, recompute.ts) |
step() runs updatePlan after pickCard (card → plan → pushWindow) and sets Snapshot.plan: CallPlan = { decision_id: null, purpose, facts: factChips(s, p), items, caution }; caution = the newest undismissed risk flag whose rule has a live_note ({risk_id, note, at: {i, t: t_end}}; RiskFlag.t = the lighting turn's t_end), null before any. An unknown decision holds the previous plan and leaves s.plan untouched (INV-COPILOT-010); the blank first snapshot shows the other template, every item later. recompute(…, events) re-applies {i, kind: 'purpose', id} through overridePurpose before utterance i; otherwise the plan replays from the stored answers (a covered:: or group question not asked replays as not answered). A template edit changes the plan timeline with state_drift 0 |
DESIGN.md §Call plan contract "Shape" |
| Plan templates (COPILOT-087, policy data) | plan.templates.<purpose id> (every purpose, other included; 3 to 10 items) of `{id, title, hint, applies, not_needed_reason, points: {move, lines?} |
null, stages, done_when}, appliesin the checklist grammar pluswhen_slot in/not_in; plan.asks ({fact, move}, each fact persisted on client turns; customer success only) and plan.rate_pressure (title, ask_first_situation: qualify_the_quote, concession_situation: concession_terms); buildPolicy refuses an unknown move, line (retired lines included), checklist id, persisted fact, slot or slot value, stage option or situation, a missing reason on a non-alwaysitem and a purpose without a template; texts areplan.template. |
| Risk flag | noul ≥ 0.60 on that turn (guarantee, rate prediction) or persisted ≥ 0.70 (third-party funding, jurisdiction); code rule: booking discussed while said_booking_is_binding unlocked; flags never auto-clear; admin can mark a false positive, which writes a label |
brief §4.6 |
| Jev error, timeout (10 s race) or 5xx after 2 retries | hold the previous snapshot, mark the decision unknown, never render zeros as signal |
guide §4.8 |
| 402 code 2021 | stop the session with an operator alert; no retry | guide §4.6 |
Retry policy mirrors the SDK defaults: 408/429/5xx (529 included), max 2 retries, 500 ms doubling to 5 s with 25% jitter, honour Retry-After ≤ 60 s; fail fast on 400/401/402/403/422 (guide §4.8).
6.6 Tailored-rewrite path (off the critical path, droppable, flag per scenario, off in M1)
Specified and built behind policy.playbook.moves[].rewrite_allowed plus a global REWRITE_ENABLED flag; enabled only after Stevan and compliance accept §10's policy.
- Trigger (reference
personalize.pyrules): card un-gated; moverewrite_allowed; ≥ 2 decisions since the last rewrite; and (move changed, or a new fact locked, or 8 decisions on the same move). Moves whosemust_saycovers safeguarding, binding or fees are neverrewrite_allowed: those lines stay verbatim-approved (data-first §6.6). - Generator (
ctx.waitUntilin M1,rewrite-jobsQueue in M2; 3 s timeout; ≤ 40 words): prompt = move title/what, its approved lines as tone anchors, the last 6 pseudonymised utterances,known_facts; instruction "use only details in the transcript; never state numbers, rates, dates, partner names, schemes or promises that were not said". Code cleans and length-checks the line. - Code denylist before spending a Jev call:
/guarantee|always cheaper|will (go|come) (up|down|back)|FSCS|risk[- ]free|definitely|protected by/i→ drop, reasondenylist. - Jev verification: a second, small request (state
{candidate_line, move:{title,what}, recent_transcript, known_facts}, ~1.5k tokens) with the single-condition verifier nouls inpolicy/src/verify.json(eight as of 2026-09-25),true= the bad case, aggregated with max (guide §7, SDE-cascade rule):invents_fact(drop ≥ 0.50),off_move(drop ≥ 0.50),makes_promise_or_guarantee(≥ 0.30),predicts_rate_direction(≥ 0.30),implies_scheme_protection("states or implies that funds are protected by a named compensation scheme such as FSCS", ≥ 0.30),names_payment_partner(≥ 0.30 until Stevan says which partners reps may name,brief §6 Q10), and, since the rate encoding,improves_rate_or_offers_discount(≥ 0.30) andexplains_spread(≥ 0.30). Verification unavailable → drop (the reference passes on error, dissection §6d; inverted here because the line is a compliance surface). - Stale check (move changed meanwhile → drop), then a
rewriteWebSocket message carrying{kind:'rewrite', rewrite_id, verified:true}; the card renders it under the approved lines, labelled "tailored (verified)". Every candidate, shown or dropped, is arewritesrow with its verifier values (one per verifier noul inpolicy/src/verify.json, eight as of 2026-09-25); the dashboard's debug drawer shows dropped candidates struck through with the reason so Stevan can audit the filter.
As built (COPILOT-075, steps 1-3). src/session/handlers/utterance.ts, after a live utterance decision (never on a seek replay), calls maybeRewrite inside ctx.waitUntil only when env.REWRITE_ENABLED === 'true' and the pinned policy's rewrite.enabled. src/session/rewrite/generate.ts decides synchronously: shouldRewrite (card shown, not unknown; move rewrite_allowed; ≥ rewrite.cooldown_decisions since the last rewrite, counted in SessionState.decisions; and the move changed, a held window fact is new, or rewrite.refresh_decisions passed on the same move). Its memory lives on the host (host.rewrite, keyed by session and chain generation, in memory only: a woken object starts afresh). A rewrite also needs room for its own two subrequests plus the next decision's. The generator (src/session/rewrite/generatorClient.ts, reached only through SubrequestMeter.generator) is one AI.run(rewrite.model, {messages, max_tokens:120}, {gateway:{id: JEV_GATEWAY_ID}}) raced against 3 s with no retry. The prompt is the move title/what, the approved lines, the last 6 utterances of the Jev state and known_facts. cleanLine ports personalize.py:104-118 with a strict ≤ max_words. One D1 batch then writes the jev_requests{kind:'generate'} row (usage from response.usage, else NULL) and, when there was text, one rewrites row: rejected_reason is unusable (no sentence) or denylist (rewrite.denylist, case-insensitive; a pattern that does not compile denies all), or NULL pending verification. The row always has shown=0 and carries gen, transcript_rev, policy_hash and decision_id. A failed or timed-out generation writes only its telemetry row. Local probe: npm run seed:local -- --sql --rewrite, REWRITE_ENABLED=true and JEV_MODE=binding in .dev.vars, then node scripts/with-cf-env.mjs node scripts/dev-remote-ai.mjs -- --port 8788. This writes the gitignored wrangler.dev-remote-ai.jsonc, where only AI is remote.
As built (COPILOT-076, steps 4-5 up to the message). src/session/rewrite/verify.ts verifyCandidate(row, s, p, judge) runs right after the generation batch, inside the same ctx.waitUntil, for every candidate still pending (rejected_reason IS NULL). s is the session host and judge is the session's metered makeJudge. It sends one kind:'verify' request with p.rewrite.verify_bank as the questions over {candidate_line, move:{title,what}, recent_transcript, known_facts}. That state is the pseudonymised context the generator saw, not the session's later one. The request gets one attempt with a 5 s deadline, and only when the meter has room for it plus the next decision. Outcomes, in order:
- no room, or an error or timeout →
verification_unavailable; - a missing verifier, a non-finite value or one outside [0,1] (by the judge's own validation or the re-check here) →
verification_malformed; - any verifier at or above its
p.rewrite.thresholdsvalue → that verifier's id, the highest value winning (aggregate by max); - otherwise
shown=1.
The binding is checked before the request (a candidate already stale spends nothing) and again before the write. The chain generation must be current, and the session_id, transcript_rev and pinned policy_hash unchanged. The card decision must also be the same: the first decision of the trailing run showing the same move. A later decision on the same card keeps the candidate, while a switch (A→B→A included) or a reset makes it stale. One metered batch updates the row: verification_json = {nouls: {one per verifier}, over, max, model, request_id} (or {error}), plus shown, rejected_reason and llm_model. The same batch carries the verify attempt's jev_requests row. A stale drop after the request keeps the verifier values. Under JEV_MODE=stub the stub answers invents_fact = 0.86 when candidate_line quotes a number that is in neither the transcript nor known_facts. The rewrite message and the card slot are COPILOT-077.
As built (COPILOT-077, step 5's message and the dashboard). src/session/rewrite/deliver.ts deliverRewrite(host, ws, row, outcome) runs after verifyCandidate, still inside the same ctx.waitUntil, and sends one rewrite message on the socket whose utterance triggered the candidate. A shown candidate's binding (bound) is checked once more right before the send, because the verify write awaited D1. If the context moved on in that window, the message goes out as stale and the row is updated to shown=0, rejected_reason='stale', so the audit row never claims a line the rep did not get. The message is {i, move_id, decision_id, rewrite, verification} plus: candidate only when rewrite is {kind:'rewrite', rewrite_id, verified:true}; rejected on every drop (a verifier id, stale, verification_*, or the generator's denylist/unusable); debug_candidate (the dropped text) only on a session loaded with debug:true. verification is the verifier's value per id when it answered, else {}. In the dashboard, web/src/session.ts keeps the latest verified rewrite as tailored (with its text in rewrites). web/src/plan.ts tailoredLine draws it as one more point on the current item, with "tailored (verified)" under it, only while the presented plan's decision_id equals the rewrite's: the next decision drops it. The review view's debug drawer (web/src/debug.ts, COPILOT-053) lists debug_candidate drops struck through with the reason and value. scripts/ws-probe.mjs --debug --wait-rewrite is the live probe; scripts/design/shoot-rewrite.mjs takes the evidence screenshots.
7. Ingestion & STT
Historical Aircall calls (no STT). scripts/import-call-coach.mjs reads the three exports in data/call-coach/*.jsonl.gz (psql COPY escaping: unescape backslashes before JSON.parse), selects category ∈ {Onboarding, Customer Service}, duration ≥ 180 s, ≥ 20 turns (1,895 + 1,226 usable calls), and for each: maps role internal → rep, external → client; drops external_phone and all Aircall metadata; runs stitch() then redact() (§10); writes the un-redacted normalised transcript and the coach record to R2 raw/{call_id}/; writes calls + utterances rows through POST /api/calls/import in ≤ 9-row statements; keeps only the coach record's numeric fields in calls.coach_scores_json (six dims, call_value_score, quotes[].turn_idx as hints for where to look, not as labels). scenario = category with scenario_source='call-coach' (LLM-assigned; M2 adds a Jev scenario Choice with disagreement flags, corpus §1e). M1 imports the four exemplars; further calls follow the DPA answer (§13 Q1).
New audio uploads (nova-3). Browser → PUT /api/calls/upload with a raw binary body and Content-Type/Content-Length (no multipart: request.formData() buffers the whole file and forwarding it makes a second copy against the 128 MB isolate limit; Aircall's 32 kb/s MP3 is harmless at ~14 MB per hour, but a 40-minute WAV is ~77 MB and would OOM) streamed to R2 raw/audio/{call_id}.{ext} with env.R2.put(key, request.body, { httpMetadata: { contentType } }) → the Worker reads the first 16 bytes back (R2.get with a range) and sniffs the container: ID3 or an MPEG frame sync for mp3, ftyp at offset 4 for m4a, RIFF…WAVE for wav; anything else, or a mismatch with the declared type, deletes the object and returns 415 rather than forwarding an unvalidated blob to a third party → the object body is streamed from R2 to nova-3 over REST with the verified query string plus keyterm and mip_opt_out=true → raw JSON to R2 raw/stt/ → results.utterances[] mapped to fragments {start, end, speaker: 0|1, text} → speaker map → stitch() → redact() → D1. Accepted content types in M1: audio/mpeg, the only one verified through the proxy (a 2.3 MB file, Content-Type: audio/mpeg, stt "Live verification"); audio/mp4 (m4a) and audio/wav are advertised only after the day-1 probe shows they survive the proxy (§13), and WAV above 25 MB is rejected with "convert to mp3". Size cap 100 MB (request body limit). Verified on a real 9:46 mono Aircall MP3: 8.6 s, 111 utterances, speaker fields survive the proxy. The body-size ceiling on the Cloudflare proxy beyond ~2.3 MB is unverified; M1 measures it with a 30-40 min file and, if it fails, returns a clear error rather than chunking (chunking at silence boundaries is M2).
Mapping diarised speakers to rep/client. Three layers, all in the Worker: (1) code heuristic: the speaker whose first 60 s contain "Currency Transfer"/"CurrencyTransfer", "calling from", or a known rep first name is rep (the strong sample shows the client answering first, so "first speaker" alone is insufficient); (2) one Jev request with one Choice per diarised speaker over that speaker's first 8 turns, options {rep, client, other} with contrastive descriptions, accepted at confidence ≥ 0.80 (corpus §1b); (3) a swap toggle on the replay page that rewrites role_map_json, re-labels utterances.speaker and writes an audit_log row. Below 0.80 the call is flagged role_map_conf low and speakers stay unknown until the toggle is used. Aircall recordings are mono (verified), so multichannel=true is not available; M3 prefers per-leg audio over diarisation wherever the transport allows.
Audio for historical calls. Recordings are fetchable with GET https://api.aircall.io/v1/calls/{id} → recording signed URL (verified, stt "Live verification"). Where the Aircall credentials live and how the call-coach host is reached are recorded in the ops runbook, which is not published; this document is (§4 Pages) and deliberately carries no hosts, paths, key or credential details. data/samples/call-3339895706.mp3 already exists, so the audio-synced demo runs on a real call with zero nova-3 cost: the Aircall transcript is the text, the audio only drives the clock.
Aircall match of an upload (COPILOT-098, P7 free STT path). An upload may carry ?aircall_call_id=<6-20 digits> and ?recorded_on=YYYY-MM-DD (a real calendar date), validated before any byte reaches R2 (else 422 {error:'invalid_query', field}) and stored in calls.aircall_call_id / calls.recorded_on (migrations/0002_upload_match.sql; recorded_at stays the real call timestamp). GET /api/calls and GET /api/calls/:id expose has_stt (stt_ref IS NOT NULL, like has_audio; the ref stays private), and the detail returns both hints. scripts/import-call-coach.mjs --ids <aircall_id> --into <up_…> writes a matched Aircall transcript into the upload: the same plan, posted with call.source='upload', call.call_id=<upload> and call.aircall_call_id at the upload's transcript_rev. Its own pre-check (source=upload, status=processing, has_stt:false) is advisory; the import route's claim UPDATE … WHERE call_id=? AND transcript_rev=? AND source='upload' AND status='processing' AND stt_ref IS NULL is the real guard (0 rows → 409 upload_not_importable), because setting stt_ref does not bump transcript_rev. An upload import never creates a row and never touches audio_r2_key. Every call-coach import sets aircall_call_id to its own id.
Devbox STT contract (COPILOT-099, P7 free STT path). env.STT_ENGINE (wrangler var, devbox by default, reported by GET /api/health as stt_engine) picks the one engine that transcribes uploads. Under devbox: GET /api/stt/pending lists up to five of the oldest untranscribed uploads (source='upload' AND status='processing' AND stt_ref IS NULL, job state not failed/needs_names) as {call_id, audio_ext, audio_bytes, aircall_call_id, recorded_on, imported_at, stt_job} (audio_bytes from an R2 head; never a key or text); POST /api/calls/:id/stt-status {state, code?, attempt} writes calls.stt_job_json (migrations/0003_stt_job.sql; closed states and codes, numbers, a timestamp) under the same untranscribed guard (else 409 call_not_pending; done is refused, only a segments post sets it) with audit_log{call.stt_status}; POST /api/calls/:id/segments validates the faster-whisper body (≤ 5 MB else 413; 1-20,000 segments, 0 ≤ start ≤ end ≤ audio_s + 1, non-decreasing start, text ≤ 4,000 chars, speaker 0-7; else 422 {error:'invalid_body', field, index?} without the text), stores only the validated fields as format:'devbox-whisper-v1' at raw/stt/<call_id>/<stt_ref>.json and promotes it with the guarded first-transcription update (0 rows → object deleted, 409 stt_already_set), setting coach_scores_json={stt_ms, rtf}, stt_job_json.state='done' and audit_log{call.transcribe, engine:'devbox', n_segments}. fromWhisperSegments (src/ingest/normalize.ts) turns the stored object into fragments (role:'unknown', or speaker_<n> when diarised). Under nova3 the queue is [] and segments are 409 stt_engine_nova3; POST /api/calls/:id/transcribe answers 409 stt_engine_devbox unless STT_ENGINE === 'nova3'. GET /api/calls/:id returns the parsed stt_job.
Devbox STT job (COPILOT-099b). scripts/lib/transcribe-job.ts runs on the devbox under a systemd user timer (scripts/devbox/systemd/), from a bundle pinned to a commit by scripts/devbox/install-stt-job.mjs (~/.local/share/jev-stt/<sha>/, current symlink; the checkout's scripts/devbox/transcribe-job.mjs runs the same code as revision dev). Per run: jev-stt rev <sha>, an exclusive pid lock (~/.local/state/jev-stt/job.lock; a second run logs busy), GET /api/health must say stt_engine: devbox, local files of calls no longer pending are reconciled against GET /api/calls/:id, then the queue is worked oldest first, one call at a time: stt-status running → download (~/.cache/jev-stt/, size = Content-Length) → the Aircall match (matchAndImport, COPILOT-098c; a review stop is needs_names) → whisper.py (large-v3-turbo, int8, 16 threads, VAD, no conditioning) → POST …/segments (a 409 stt_already_set/call_not_processing counts as done). A per-call state file ({phase, attempt, audio_sha256}, rewritten atomically) makes it resumable, so a transcribed call is posted without being transcribed again. Three failed attempts post stt-status failed with the code, and --retry re-enqueues the call. Local audio and whisper output are deleted once posted (a failed call's after 24 h), and the journal carries ids, codes and numbers only (INV-COPILOT-004). The bundle is self-contained: scripts with a main guard (with-cf-env.mjs, deploy.mjs, the devbox CLIs) are kept out of it, so loadEnvFile lives in scripts/lib/cf-env.mjs, the importer runs in-process as runImportCli() (scripts/lib/import-call-coach-cli.ts, with the policy lists bundled), and the real side effects are in scripts/lib/devbox-io.ts.
Devbox diarisation (COPILOT-101). After whisper the job runs scripts/devbox/diarize.py (sherpa-onnx: pyannote segmentation-3.0 + TitaNet-small embeddings on CPU, three clusters with the two holding the most speech kept; --no-diarize skips it). scripts/lib/diarization.ts gives each segment the speaker whose turns overlap it longest, or the nearest turn's speaker when none does, and applies the balance check: exactly two speakers, the smaller holding ≥ 15% of diarised speech. A balanced run posts segments[].speaker (0/1, no role) plus diarization: {engine, embedding, speaker_seconds, rtf, balanced: true}, so the ingest route maps speakers with COPILOT-066's mapSpeakers and the swap toggle (COPILOT-070) exactly as for nova-3. An unbalanced run posts the same summary with balanced: false and no speaker (COPILOT-100 labels the segments); a failed diarize.py posts neither. The journal line gains diarize <s>s and diarization_unbalanced / diarization_failed.
Role labelling of undiarised segments (COPILOT-100). When a devbox transcript has no diarised speaker, src/ingest/segmentRoles.ts labelSegments labels each segment, keyed s<i>, in COPILOT-066's speaker-map shape (map, conf, source) plus guesses, confs and n_review. It runs after redact() and the review gate and before stitch() (COPILOT-069's ingest route), and throws before any Jev call on text that fails assertRedactedText. The first layer is code, at confidence 1: a self-introduction, "calling from Currency Transfer" or a greeting of the client by name marks the rep; a greeting of the rep by name, "[CLIENT] speaking" or a bare "Hello?" among the first two segments marks the client; a segment with signals for both sides gets no label. Every other segment is asked of Jev as one segment_role Choice {rep, client} (policy/src/classify.json, the probe's variant E), packed several to a request over transcript.{s<i>} with three segments of context on each side. Packs are sized so each request's estimate (the engine's chars/3.5 + 300) stays ≤ 11,000 tokens, under the 12,000 cap. They are sent one after another, and the run makes at most ctx.maxAttempts attempts, retries included (JudgeOptions.maxAttempts), because the Free plan allows 50 subrequests per invocation. The jev_requests rows go through bufferedSink and are flushed as one db.batch(). A label sticks at confidence ≥ 0.80; a backchannel keeps its guess at any confidence; everything else stays unknown and counts in n_review for the admin toggle (COPILOT-070). conf is the share of speech time labelled by a heuristic or at ≥ 0.80.
8. Dashboard and Admin console
8.1 Dashboard (Worker static assets; TypeScript, no framework; Chart.js as in the reference)
Shell (COPILOT-045). Two views on one page, both routes: ?view=live (default; design G) and ?view=review (D's review view on C's grid), plus ?call=<id>. web/src/boot.ts (public/boot.js, a blocking <head> script) sets <html data-view> and data-theme before paint; the theme follows the OS, ?theme=light|dark overrides and is remembered in localStorage (?theme=auto forgets). web/app.css holds both token sets scoped by [data-view] (copied verbatim from docs/design/shotgun/G/live.css and docs/design/mockups/src/tokens.css, checked by tests/unit/web-shell.test.ts), the two layouts and the picker; fonts are self-hosted under public/fonts/ with their OFL files. Without a replayable call the page is the call picker (GET /api/calls, scenario and status badges, non-ready calls disabled with the reason). With one, web/src/shell.ts draws the view's frame with fixed slots ([data-slot], class .slot-<name>): live transcript, plan-head, chips, plan; review review-transcript, mark, seen, review-plan, resolution (CS only), risks, stage, concerns, signals, usage, alerts, conn; both player. Panel stories call registerPanel(name, slot, render) in web/src/app.ts. All DOM goes through h() (web/src/dom.ts), never HTML strings (INV-COPILOT-014). Evidence screenshots: node scripts/design/shoot-dashboard.mjs <dir> '<shots>' --base <wrangler dev url>. State flags (COPILOT-102a) take the shot in the page's own session and fail when the state is absent: --at <mm:ss> (20×, pause at the clock time, wait for the session queue to drain), --until '<sel>' (20×, pause on the first frame the selector matches; exit 1 if the call ends first), --expect '<sel>' (exit 1 on no match within 10 s) and --expect-none '<sel>' (exit 1 on any match); the cores are pure functions in scripts/design/lib/shoot-flags.mjs.
Session connection (COPILOT-045b). web/src/app.ts owns the one DashState and the socket ws(s)://<host>/ws/calls/:id?session=<nonce> (scheme from the page). The nonce is per tab and call in sessionStorage['copilot:session:<id>'] (web/src/session.ts sessionNonce, storage errors tolerated), so a reload reaches the same Durable Object with load{resume:true} and continues from hello.last_i + 1 with its acks and purpose override; a new tab gets its own; choosing a call in the picker and Reload call (after error{call_replaced}, load without resume) call newSession. A closed socket reconnects with backoff (1 s doubling to 10 s) unless the session is stale. Every frame goes through dispatch(state, msg) → {state, effects}, a pure function covering every ServerMsg type (unknown or malformed → console.warn, never a throw): hello replays history, decision keeps d.plan for the live view (a re-sent i, as set_purpose does, replaces its decision), queue sets pending, error{call_replaced} sets stale. Panels read the state through subscribe(fn); send(body) and sendSetPurpose(id) frame client messages. The browser imports the real protocol types: SocketAttachment lives in src/session/protocol.ts and the call read shapes in src/routes/calls-shape.ts, so neither pulls Worker globals into web/.
Policy by version, text by id (COPILOT-045b). After hello the page fetches GET /api/policy/:scenario?version=<hello.policy.version> and verifyPolicy (web/src/policy.ts) recomputes bank, playbook and weights hashes and policy_hash with the bundled src/policy/canonical.ts; a mismatch (or a failed fetch) shows the policy notice, logs a console error and leaves the policy rejected, so nothing from it renders. renderRepText(rf, policy: VerifiedPolicy) (web/src/render.ts) is the only path from policy text to DOM: the VerifiedPolicy brand means an unverified bundle does not type-check; approved ids resolve through the textEntries walk (lines, situation points, headline.*, concern.*, situation.*, caution.*, plan.*) of the same policy_version; an approved_template is filled only after checkTemplateParams / checkPointParams (a {country} shows the country's display); a rewrite needs its verified text from a shown rewrite message. Anything else throws RefusedText. Text is set with textContent only; tests/unit/web-dom-safety.test.ts greps web/src for HTML-parsing APIs. DOM tests run under happy-dom and are typechecked by web/tsconfig.tests.json (the root project has no DOM lib).
Replay clock and player (COPILOT-046). The browser owns the call clock (INV-COPILOT-009). web/src/clock.ts: VirtualClock (rAF, speed, seek(t), tick(t)), AudioClock over <audio src="/api/calls/:id/audio"> (Range; position from timeupdate, interpolated on frames at the playback rate for at most 1 s so buffering holds it, never moving back between seeks; the recording's ended is the end of the call, so a file a little shorter than duration_s still reaches the end and flushes the last utterances), and ReplayClock, which lets the recording drive up to AUDIO_MAX_SPEED = 4× (Chrome caps playbackRate) and hands over to the virtual clock above it (the recording paused and muted) or when the recording fails (play() refused or a load error; an AbortError from a pause interrupting play() is not a failure). UtteranceDriver sends utterance{i} for every next i (strictly last + 1, backchannels included, so out_of_order cannot happen) once the clock passes its t_end; a scrub sends seek{i} for the end of the longest prefix of t_end ≤ t (a prefix because a merged turn can end after the backchannel that follows it; before the first t_end it seeks to 0) and resumes from i + 1. A send that finds no open socket stops, and the next tick retries from the same i. If the session still reports error{out_of_order, detail:{last_i}} (an object woken from hibernation resumes at its last decision and expects the transcript-only utterances after it again, per the rehydrate client contract, but the socket stayed open so no hello says so; seen live on 3303259297, backchannels #48/#49 before a 51 s turn), the player resyncs the driver to last_i and re-sends from last_i + 1, once per cursor. web/src/player.ts renders the compact player in the live top bar (Replay pill, never Live; play, a scrubber without ticks, speed, the catching-up badge, the source, the clock) and the full bar in review (decision and risk ticks, speeds 0.5×–20×, source virtual while the virtual clock drives). A hello of a new session moves the clock to hello.last_i; the same session after a drop keeps the clock and resumes play, the driver re-sending what the session missed. Play is off before hello, while disconnected, after call_replaced and after credits_empty (alerts are per session: a new session's hello clears them). The badge counts queue.pending − 1, the utterances queued behind the running one.
Live view presenter (COPILOT-093). The engine decides what is true; web/src/presenter.ts decides when the rep sees it (DESIGN.md §Live view "Cadence rules"). present(prev, incoming | null, clock, utterances) is pure. The page calls it on every decision and every clock tick with the call clock the browser owns (INV-COPILOT-009), and seekPresented(decisions, clock, utterances) starts it over at the decision the new clock has reached (decisionAt, the prefix rule of passedIndex). The items of a plan (states, order, the current card) move only at a change moment (COPILOT-105, decision 1b): the end of the client's run, isClientRunEnd(us, u, upto) (a client turn whose next turn, skipping backchannels and acks, is not the client's or has not arrived before upto); with LIVE_CARD_AT_SENTENCE_END (or present's cardAtSentenceEnd option) every client part end. A switch decided on a client part applies at the first boundary at or after max(arrival, dwell end, its run's end): that time itself unless a client run is in progress there (then that run's end); when LIVE_MIN_DWELL (20 s from when the card opened) sets it, a running rep turn is waited out too. An inserted concern:/rate: item uses the same rule without the dwell. A plan decided on a rep turn waits for the next run end no rep turn is running at; after the last t_end anything pending is flushed; the newest plan replaces an older pending one. A plan that keeps the open card applies on arrival for a client-part decision, so its new point shows within a sentence; at most one new uncovered point appears per applied plan, and one added while a client run is in progress is in midrun (rendered data-midrun="1" by web/src/plan.ts) until that run ends. A new card opens whole. Point ticks, the purpose, the chips and the caution come from the latest decision at once. A covered point in the open card is held ("Covered just now") until LIVE_COVERED_LINGER (5 s) after the next client run ends, then folds. The caution is shown while clock − at.t < 15 with elapsed for the CSS fade. A chip whose updated_at is under 10 s old is in updated. listening is true before the first decision, when no item is current and after an unknown decision, which changes nothing else.
Point lifetime (COPILOT-107, decision 3, R1-R4). The presenter also decides how long a point on the open card stays. Presented keeps revealed (in reveal order, so later points stack under earlier ones), revealedAt, kept (each revealed point as last applied, with its situation and any tick), dropped and grace, and outputs lastSuggestion: {item, point, until} | null. The rules are pure functions of the clock and apply in this order: (1) a covered point folds by LIVE_COVERED_LINGER as before; (2) an uncovered point whose situation is in a decision's dropped_hard (108a; absent = none) leaves as that decision arrives; (3) at most LIVE_POINTS_MAX = 3 suggestions show, the card's slots (uncovered plus held) and the last-suggestion line together: the line goes first, then the oldest uncovered point, even inside its floor, and an evicted point is not revealed again on that card; a new card opens with at most 3 uncovered points; (4) on a switch the old card's newest uncovered point becomes lastSuggestion for LIVE_LAST_SUGGESTION = 12 s (a card that left none keeps the previous line until it expires); (5) a point a newer plan on the same card drops keeps its slot until LIVE_POINT_MIN = 12 s after it was revealed. Rep-turn grace (R4): an uncovered point that a plan decided on a rep turn introduces (absent from the item in the previous decision's plan) is not revealed until the next rep-turn decision arrives (a client decision does not release it, since only rep turns tick points) or LIVE_REP_POINT_GRACE_S = 3 s pass; with the tick it arrives grey. An item with due_by: 'client' (110f) opens without the dwell, like a concern. seekPresented starts all of it over. Over the 49e3e48f fixture the amount point (introduced on rep i 9) is never tinted: rep i 11's decision releases it covered at 101.2 s; the first-transfer-date point keeps its 110.9-134.1 s; the one last suggestion is the who-holds point after the 568.8 s switch.
Presenter simulator (COPILOT-105a). scripts/presenter-sim.mjs --session <session_id> [--log <file>] measures the presenter's cadence without a browser. It reads a replay session's decision_json rows from remote D1 (scripts/lib/d1-remote.mjs, SELECT only, shared with export-call-curation.mjs) and the utterances from GET /api/calls/:id, refusing a call whose transcript_rev has moved on. It then runs present() over them on a 0.1 s call clock (scripts/lib/presenter-sim.ts, pure, a loop counter and not a timer). At each step it feeds the decisions that have arrived by then, in arrival order, then the clock alone, as the page does. Arrivals come from 118a's log (arrived_ms × speed / 1000) or are projected as t_end + 0.3 s and labelled projected. It reports on the open card only. A point is on screen while it shows uncovered, and its interval ends covered, hard_drop, cap, switch, withdrawn (a newer plan on the same card dropped it) or end. "Points shown" counts what the presenter caps: slots (uncovered plus held) plus the last-suggestion line, the live card's data-points (107b). shortest_on_screen and left_within_floor check the 12 s floor over the exits no earlier rule explains (107b). A client run is consecutive client turns that no rep turn interrupts. Over the 49e3e48f fixture it reproduced FEEDBACK §3 before 107 (the amount point 100.3-101.3 s, then covered; the first-transfer-date point 110.9-134.1 s, then withdrawn); since 107 the amount point never shows uncovered and last_suggestion reads the presenter's line. The core imports the web presenter, so it and its test are typechecked by web/tsconfig.tests.json.
Displayed feedback and the measured table (COPILOT-118b). The simulator also reports what the rep saw change, distinct from 118a's transport figures. What changes on screen is a set of elements: a point or a tick on the open card, a chip, the caution. A decision introduces the elements its plan carries that the previous known decision's did not (its current card's points and ticks, its ticks on the card open when it arrives, its chips, its caution). Its first presenter change is the first appearance of one of them at or after its arrival while it still owns it: a later decision that stops carrying an element ends the claim, one that carries it too does not take it over. So a switch held by the dwell stays the deciding decision's, and fades, folds, the floor's releases and the last suggestion's expiry are nobody's. presented_after_part_end_p50_ms / _p95_ms are that time minus the part's t_end (nearest rank; a change on the arrival's own step is timed at the arrival), and no_change counts the decisions (unknown ones included) that showed nothing, outside the percentiles. It also reports the amount point's first render (AMOUNT_POINT: ask_about_the_transfer shown for amount_not_said; tinted or grey, with the decision then latest and whether it was a rep turn), max_points_shown, decisions_with_uncovered_point (arrivals that left one on the open card) and items_with_point. scripts/m1b-measure.mjs --call <id> --session <id> [--log <file>] prints the M1b measured table (scripts/lib/m1b-measure.ts): from the stitched fixtures, decision points and the longest and p90 gap for v1 and v2, and the word-weighted spoken-to-visible wait (a word is spoken at the middle of its even share of the line; visible at t_end under v1, at revealedWords's first moment under 102; 9.1 s on 3339895706 v1); from a presenter-sim.mjs child run, the presenter rows. tests/unit/design-constants.test.ts keeps DESIGN.md's §Cadence rules constants table equal to the presenter's LIVE_ exports.
The M1b acceptance check (COPILOT-118c). scripts/m1b-acceptance-check.mjs turns "M1b is done" into an exit code. It asserts health == HEAD and reads the following, read-only: the session, its decisions and each decision's checkpointed concern (session_state_json) from D1; the admin fixture facts (marks on 3339895706, labels on 3303259297, pd_ct_id on 3485591407); the call's utterances and the session's policy from the API. It runs simulate and reviewModel in process. The pure core, scripts/lib/m1b-acceptance-check.ts, checks conditions 0-9 of the PRD. "Before 240 s" and "at a decision" read the decision's utterance t_end. Condition 4 checks that revealedWords draws no word k before t + k × span / words (INV-COPILOT-018). The simulator now lists every render of the amount point (amount_point_renders), so condition 3 holds for every render, not just the first. A pass needs 118a's arrival log: its header must name this session, call, transcript_rev, policy_version and the deployed sha, with wall_clock true and exactly one line per decision point. --projected reports only (exit 2). Condition 7 (the seed) is required only with --expect-seed, and the source is read from sessions.precall_json's {precall, seed}.
Live feed and transcript (COPILOT-047). The player reports the call clock to the page (PlayerDeps.onTime(t, seeked), subscribeTime in web/src/app.ts), and GET /api/calls/:id is read once per page for the player and the panels. web/src/live-feed.ts folds the session and the clock through the presenter: every decision the dispatcher adds (a re-sent i is a new object) goes through present, every tick through present(prev, null), and a seek or a new session through seekPresented; every live panel draws from its one Presented. web/src/transcript.ts draws the transcript in two forms. Live (G): the lines the clock has reached, word by word (COPILOT-102: the pure revealedWords(u, t) spreads a line's words evenly over [t, t_end], null before t; a row still being spoken carries data-partial, rows stay in i order so a backchannel inside a long turn sits above it, and a seek back removes and truncates), "You"/"Client", asides grey, the line being spoken tinted, a covered tag on whole lines only (the item's title through renderRepText) on each line that is an item's done_at.i or a point's covered_at.i in the latest decision.plan, the presenter's caution under its turn (data-shown-at, --elapsed, one "Careful" tag) and "Listening" while presented.listening; it follows the newest line unless the rep scrolled up. Review (D): every line with #i, the decision-point diamond, REP/CLIENT, risk marks from the latest decision, "covered a point" (a point's covered_at, or a plan item's done_at), "Share of words" and the rep-monologue warning (over 70 rep words since the client's last turn) over the lines passed; lines after the clock are grey, the current line follows the latest decision, and rows are focusable options (tabindex=0, aria-selected) for the mark form's span selection. scrollToUtterance(i) / flashUtterance(i) centre a line within the pane (never scrolling the page) and flash it --surface-2 for 1.6 s. The CSS fade of a caution runs in wall time from the --elapsed it was drawn with (redrawn after a seek), while its removal follows the call clock (the presenter): at 1× the two agree; a faster replay removes the caution before its fade has run, and a pause does not stop the fade.
Plan list (COPILOT-051). web/src/plan.ts draws the presenter's plan in two live slots. The plan-head panel adds the tally in words ("4 done, 5 to come, 1 not needed"; "to come" counts next and later) and the purpose line: the purpose's approved label as a quiet button that opens an inline list of the pinned policy's plan.purposes under "Change the purpose of this call"; a choice sends set_purpose{purpose_id} (sendSetPurpose), shows the chosen label with "Set by you" at once, and keeps it until the re-sent decision carries the override. Before a purpose is known the line is empty. The plan panel draws the items in plan order with the not_needed ones after a "Not needed on this call" label at the foot. Each row has a CSS mark (tick, dot, ring, dash), the title and its state word ("Done 02:58", "Now", "Next", "To come", "Client raised 00:03, done 03:47", "Raised by the client 03:55", "Client asked 01:12, to come", or the not-needed reason). Every time is a link to its line (it shows the utterance's start, as the transcript does) that calls the transcript's flashUtterance(i). Done, next and later rows with a hint or points expand in place. The current item is the one open card: the title as an h3, the hint, and ONE tag over its points ("Worth saying", or "Say" on a rate: item). A rate item also shows the situation's headline in quotation marks and, when it has ask_first, ONE "Ask first" tag over those questions as a sub-checklist. A covered point in the presenter's held keeps its slot as a grey line ("Covered just now, 04:44"); the other covered points fold into "Also covered at 08:16: …" lines. Rows are redrawn only when their item changed (a JSON key of the item and its held points), so an expanded row stays open, focus stays put, and only a point the card had not drawn before gets the settle animation. renderPlanItem(item, opts) is exported for the review view (COPILOT-049). All policy text goes through renderRepText; a refused title drops the row and a refused point drops the point.
Client facts chips (COPILOT-094). web/src/chips.ts (styles web/styles/chips.css, scoped under .panel-chips) draws plan.facts from the presenter into the live chips slot: in the plan head, under the purpose line and above the list. The view keeps the engine's order (factChips: group, then importance, then recency) and never re-sorts. Only filled slots arrive, so there is never a placeholder. It shows at most CHIP_CAP = 8 chips, then a "+N more" button that shows the rest inline, and "Show fewer" folds them again; focus stays on that button. Each chip is a button that calls flashUtterance on the turn that last updated its value, or else the turn that set it. While the presenter lists its slot in updated (LIVE_CHIP_UPDATED = 10 s of call time after updated_at), the chip also reads "updated 04:12", with the utterance's start time in DM Mono. Chip text goes through renderRepText only, which checks an approved_template's params by type again; a refused label drops its chip. The row is redrawn only when the chips, their marks or the expand state change.
Call stage and Concerns (COPILOT-048). Review view only; the live view has no stage or concerns slot. Both read the decision history (DashState.decisions: hello.history plus each decision). web/src/stage.ts (styles web/styles/stage.css, .panel-stage): the latest decision's stage.current in words (rate_and_pricing → "rate and pricing"), "how sure 0.71" from stage.confidence, "since #i" (the first decision of the current stage's latest run), stage.history as chips with the current one marked, and the note "Reps do not see the stage: the live plan shows where the call is by position." web/src/concern.ts (styles web/styles/concern.css, .panel-concern): a decision only carries the concern as it stands, so concernEpisodes rebuilds every concern of the call. An episode is keyed by start_i; each distinct since is a raise. An open decision followed by a closed one ends it. It was cleared when that decision's clearing signals pass concern_clear (the engine's own test, CLEARING_SIGNALS exported from src/engine/concern.ts) or it closed before concern_max_age; otherwise it expired. Rows are newest first: the approved concern.<type>.headline through renderRepText ("Unclear concern" for unknown), an open / closed chip, "Raised at #34, again at #39. Topic: money. How sure: 0.66" and the closing rule in words for an open one, "#12 to #15. Closed: …" for a closed one. Mark as false alarm sends flag_false_positive{kind:'concern', id: type, i: the latest open decision}; the row reads "Marked as a false alarm." at once and from dismissed_since after that, until a re-raise. Every review #i is refLink (web/src/dom.ts, .ref in web/app.css), which calls scrollToUtterance.
What the rep saw, The plan and Risk flags (COPILOT-049). Review view only (seen, review-plan and risks slots), in web/src/review-plan.ts (styles web/styles/review-plan.css). What the rep saw is reconstructed, because browser arrival times are not recorded: RepSaw runs the live presenter (present, COPILOT-093) over the decisions the clock has reached, each arriving at its utterance's t_end, then ticks it to the clock. It is incremental while the clock plays forward and starts over after a seek back or a re-sent decision. The current item is drawn by the live renderPlanItem(item, { compact: true }) (20px title, 14px points, one "Worth saying" tag); then the caution if it was showing, "How sure Jev is 0.58 (a card shows from 0.35)" (card.confidence against next_move_min_conf), "Held back: move, until …" (card.suppressed[0]'s rule in words, heldBackReason) and "How the plan changed" (planEvents: purpose set, client item added, item opened, point covered, item done, caution shown, concern raised again, concern closed, and the move Jev leaned towards while listening), newest first with time and #i. The plan reads the latest decision's plan: tally meta, the purpose ("set by you" on an override), every item in plan order with a state word (done / now / next / to come / not needed) and a shape mark, "Raised at #i. Done at #i.", the not-needed reason, and the must-say value from decision.checklist for items whose template done_when has a must_say (said / unsure 0.52 / not yet). A said must-say has Mark as false alarm (flag_false_positive{kind:'must_say', id, i: the lock}); other values have nothing for the engine to dismiss, so no button. Risk flags lists the latest decision's latched risks, newest first: ▲ and the label in --risk, #i, the quote (the utterance text), "Jev's score 0.71." ("Set by a rule." for code rules), whether a decision ever carried it as plan.caution ("Shown to the rep once as a caution." / "Not shown to the rep."), and Mark as false alarm (flag_false_positive{kind:'risk', id, i}); a dismissed flag is greyed and struck through with "Marked as a false alarm."
Issue resolution (COPILOT-050). Review view only, customer-success calls only (resolution slot, panel hero-cs), in web/src/hero-cs.ts (styles web/styles/hero-cs.css). It reads the customer-success hero of the decision history. The engine's CS hero carries resolution_at (the turn each state was reached, from SessionState.resolution_at) and raw (the health before the EMA; health is the smoothed value). The steps "no issue / issue raised / owned by rep / resolved" mark the latest resolution as current and earlier steps as done, each reached step with its turn's start time and #i. "call health, 0 to 1" shows the latest raw, "smoothed" (health) and the last change, then a Chart.js line chart (bundled from npm by esbuild, no CDN): one point per check (a decision with answers; an unknown decision holds the last snapshot and is skipped), health solid --ink, smoothed dashed --accent, gridlines at 0 / 0.5 / 1, a dotted --said marker labelled "owned at #19" at each step reached (labels that would touch drop a line), and "each check" under the x-axis. Colours and fonts are read from the review tokens on the canvas, and re-read when the OS theme changes. The chart is made once per panel and redrawn as decisions arrive; HeroCsPanel takes a chart factory, so the unit tests check the series without a canvas. "What moved it" is the latest check with a non-empty moved: at most 3 rows by |Δ|, name | #i | ±Δ (up --said, down --risk); weight ids read as words (signalName; the code signals concern_open and talk_penalty read "Concern open" and "Rep share of the talking", since Δ is the effect on health). Before any check moved health it reads "Nothing has moved it yet." No text says EMA, probability or likelihood (INV-COPILOT-012).
Signals, Jev usage and the session alerts (COPILOT-052). Signals (signals slot, web/src/signals.ts, styles web/styles/signals.css) and Jev usage (usage slot, web/src/telemetry.ts, styles web/styles/usage.css) are review view only; the live shell has neither slot. Signals reads the latest decision's signals: nouls and facts are on (≥ signal_on), unsure (in uncertain_band, never off), off, or "no answer" for null (never 0), listed in that order and then the Scores, whose level reads high / medium / low (lamp lit at high). A held fact carries the lock "kept for the rest of the call". The legend states the policy's thresholds in words. Jev usage reads every decision's JevTelemetry: the latest model, with stub-1 ▲ expected jev-1.13.0 in --uncertain for the rest of the session once any answer came from a model other than hello.model_expected (so a JEV_MODE=stub session always shows it); "Response time, average / slowest 1 in 20" (mean and nearest-rank p95 over checks that sent a request); "Last check, tokens sent / received" and "Tokens so far" (requests only; a hit sends nothing); "Cost so far" (cum_cost_usd, $0.042/M input); "Questions dropped to fit the size limit" (the latest check's budget drop steps); "Answers reused n of m" (cache: 'hit' of all checks). "Show the last request and answer" is disabled with microcopy until the debug drawer (COPILOT-053). The alert frames render in the alerts row of both views (web/src/alerts.ts), with words built from the session rather than the frame's message: credits_empty (--risk, "Nothing after #i was judged"; the player's playBlocked disables play and speed in both views), model_drift (amber, Got it calls dismissAlert for the rest of the session), budget_skipped (amber, the latest skipped decision and the policy's token_budget.hard). The live view says the same without #i, model ids or numbers, on G's neutral alert surface.
Debug drawer (COPILOT-053). ?debug=1 on the review view (debugFromLocation in web/src/route.ts) is the only way in: the shell then adds the debug slot under the grid, and the page loads its session with load{debug:true}. The live view has no slot, whatever the URL says. load.debug is fixed when a session starts (a resume keeps the stored flag), so a debug page keeps its own tab nonce (nonceKey(call, true)) instead of resuming a plain session that would never echo. web/src/debug.ts (styles web/styles/debug.css) is full-width and sticky at the bottom of the window. The tab row is its handle: collapsed means no tab is selected. Jev usage's "Show the last request and answer" opens it on Last request; without ?debug=1 that link is disabled with "Open the page with ?debug=1 to see it." The tabs:
- Last request / Last response: the latest decision's
jev.last_request/jev.last_response, pretty-printed and coloured. An echo cut at 32 KB does not parse, so it shows as sent, with a note. - Weights: every threshold and weight of the verified policy. The form is built once per
policy_hash, so typing survives redraws. It sendsset_weightswith the changed keys only, then rendersrecomputed: "state drift: N of M decisions changed" and a per-decision diff of the hero (n of m said, or resolution · health) and plan tally, before → after, for decisions where either moved. - Rewrites:
rewrite{rewrite:null, debug_candidate}frames kept inDashState.droppedRewrites, struck through withrejectedand its verification score (data from COPILOT-077).
Each tab keeps the mockup's right column: the drift line and the latest dropped rewrite. recomputed and droppedRewrites reset on a new session_id.
Mark a moment (COPILOT-054). Review view only: the mark slot docked under the transcript in the spine (web/src/mark.ts, styles web/styles/mark.css). MarkPanel owns the span selection (SpanSelection: an anchor and a moving head, normalised to start_i ≤ end_i) and paints it through the review transcript (ReviewTranscript implements MarkTranscript: paintSelection, paintPins, order, row), which keeps both across its one rebuild when the call's lines arrive. Mouse: a click selects one line, shift-click or a drag extends from the anchor (the mousedown is prevented, so a drag selects lines, not text). Keyboard on a focused row: ↑/↓ move the focus, Shift+↑/↓ extend from the focused row (restarting there unless it is the moving end), m opens the form, Esc clears; Esc or Cancel in the form closes it and keeps the selection. The form sends mark_moment{start_i, end_i, verdict, tag, note?} over the session socket: verdict from Good example / Acceptable / Avoid (model / acceptable / avoid), tag kind or kind:words from a select of the markable moment kinds plus optional words (capped so the tag fits the protocol's 64 characters; the server's tagMoment files it). It waits for the reply the way the debug drawer does, by identity of DashState.marked / DashState.error: marked shows the --ink toast (showToast, web/src/dom.ts) "Marked #a to #b as ● marked, resets the form and returns focus to the transcript; error{note_rejected} gives the note a --risk border and "Remove names or numbers from the note." and focuses it; any other error is named inline, and a call_replaced while waiting says the mark was not saved. Spans marked earlier are pinned from GET /api/moments?call_id= once per page. The server redactor that decides note_rejected (redactFreeText, src/session/marks.ts) was built for transcripts: a sentence-initial capitalised word outside COMMON_WORDS (e.g. "Answered …") counts as unresolved, so such a note is refused although it holds no name.
Audit controls (COPILOT-055). Review view only. "Mark as false alarm" is one helper, web/src/flags.ts: flagSender(send) builds flag_false_positive{kind, id, i} (kind risk | must_say | concern), falseAlarmButton is the link button, and FALSE_ALARM is the "Marked as a false alarm." wording. The Risk flags, The plan and Concerns panels (web/src/review-plan.ts, web/src/concern.ts) take the one flag: FlagSender from web/src/app.ts. Each panel greys a row it flagged at once and keeps it greyed from the next decision (RiskFlag.dismissed, ConcernState.dismissed_since, the must-say lock gone). The speaker swap (web/src/speaker-toggle.ts, the review top bar's speaker slot) shows for source === 'upload' or role_map_conf < 0.80. It posts role_map.map with rep and client swapped to POST /api/calls/:id/speaker-map, which is a 501 stub (src/routes/speaker-map.ts) until COPILOT-070. A 409 call_has_derived_rows opens the shared invalidation dialog (web/src/dialog.ts, confirmInvalidation); its confirm re-posts with force:true. On success the page forgets the tab's nonce and reloads. The reload re-reads the rebuilt utterances, and mount loads a new session with resume:false, because hasSession (web/src/session.ts) is false for a tab with no nonce.
| Panel | Shows | Source |
|---|---|---|
| Player | <audio> element (uploads and calls with audio; seeking needs the Range-enabled audio route, §4) or a virtual clock; play/pause/seek; speed 0.5-20×; call picker; scenario badge; "catching up (N queued)" badge driven by the DO's pending counter (§4) at high speed |
client |
| Transcript | stitched utterances with time and speaker colour, backchannels dimmed, decision points marked, talk-share bar and rep-monologue warning (code) | GET /api/calls/:id + decision.talk |
| Hero (onboarding) | checklist: each applicable must-say item as said / uncertain / unsaid / n/a with the rep utterance it locked on; completeness x/y; risk flags in red with the offending utterance and value | decision.hero |
| Hero (CS) | resolution chip none → reported → owned → resolved with timestamps; health bar + EMA line; "what moved it" top 3 | decision.hero |
| Stage strip | history chips; current stage with confidence | decision.stage |
| Open concern | type or parent bucket, confidence, since, what would clear it; the topic's approved lines | decision.concern |
| Next best move | title, what, up to 3 approved lines with the highlighted one marked "say:", tailored line if a rewrite arrives, "rule: … hidden because …" note, "listening…" state; ack/dismiss buttons |
decision.card, rewrite |
| Signals | grid of lit / uncertain / off signals, lock icons on persisted facts, score levels | decision.signals |
| Weights & thresholds (M2) | editable → set_weights → recomputed (zero Jev calls) with the state-drift count; belongs with corpus calibration (§6.5, §9), not in Stevan's M1 ask; in M1 the messages exist only behind the debug drawer and are not an acceptance criterion |
|
| Jev telemetry | model id (with a drift warning if ≠ model_expected), latency mean/p95, tokens, budget degradation, cumulative cost, last request/response drawer, 402 alert |
decision.jev, decision.budget |
| Mark moment (M1) | select an utterance span → model / acceptable / avoid + tag + note → moments + marks |
WebSocket protocol (reference's hello / update / recomputed / phrasing / error extended, dissection §7c; every message {v:1, type, seq, ts, …}; Access JWT on the upgrade):
type ClientMsg =
| { type: 'load'; call_id: string; policy_version?: number }
| { type: 'utterance'; i: number } // the browser clock passed utterances[i].t_end
| { type: 'seek'; i: number } // DO resets and re-evaluates 0..i; cached rows cost no Jev call
| { type: 'set_weights'; weights: Policy['weights']; thresholds: Partial<Policy['thresholds']> } // experiment, not publish; M2 panel, M1 debug drawer only
| { type: 'ack_card'; card_id: string } | { type: 'dismiss_card'; card_id: string }
| { type: 'mark_moment'; start_i: number; end_i: number; verdict: 'model'|'acceptable'|'avoid'; tag: string; note?: string }
| { type: 'flag_false_positive'; kind: 'risk'|'must_say'|'concern'; i: number } // writes a label
| { type: 'set_speaker_map'; map: Record<string, 'rep'|'client'|'other'> } // uploads only
| { type: 'ping' };
type ServerMsg =
| { type: 'hello'; session_id: string; call: CallMeta; policy: { scenario; version; policy_hash; bank_hash; playbook_hash }; model_expected: string; history: Decision[]; last_i: number; transcript_rev: number; acked: string[] }
| { type: 'decision'; d: Decision } // one per decision point; d.i, hero, risks, stage, concern, checklist, card, signals, health, talk, budget, jev
| { type: 'rewrite'; i: number; move_id: string; rewrite_id: string; text: RepFacingText | null; rejected?: string; verification: Record<string, number> }
| { type: 'recomputed'; timeline: Decision[]; policy_hash: string; state_drift: number }
| { type: 'queue'; pending: number } // length of the DO's serial chain (§4); drives the "catching up" badge
| { type: 'alert'; code: 'credits_empty'|'model_drift'|'budget_skipped'; message: string }
| { type: 'pong' } | { type: 'error'; code: string; message: string }
| { type: 'reconnect'; after_i: number }; // COPILOT-058b: reopen the socket, load{resume:true}, re-send from hello.last_i + 1
// Rep-facing text is typed so the UI cannot render anything else.
type RepFacingText = { kind: 'approved'; text_id: string; policy_version: number }
| { kind: 'rewrite'; rewrite_id: string; verified: true };
The seek cache is keyed (call_id, i, bank_hash, state_hash) for judging answers and (call_id, i, playbook_hash, state_hash) for move answers; scrubbing back and forth costs nothing after the first pass as long as the rebuilt state matches, and re-asks (paid, logged) when it does not. A judging hit with a move miss sends the moves-only request (§3); the DO never re-pays for judging answers it already holds, and a full request is sent only when answers misses.
8.2 Admin console
M1 ships the seed: the mark-moment form on the replay page, moments emitted by the engine at each session (kinds and triggers below), and a read-only GET /api/moments?call_id= list. M2 ships the console on Pages behind Access, with the review queue ordered by information value (corpus §3c): uncertain-band moments first, then unlabelled top-composite calls, then bottom-composite, then disagreements between trainer_pick and the atomic composite.
| Moment kind | Emitted when | Admin sees |
|---|---|---|
must_say |
a said_* noul first crosses 0.70 on a rep turn |
the rep turn ± 2, which item, the value |
risk |
rep_made_guarantee / rep_made_rate_prediction ≥ 0.60, third-party funding, booking-without-binding |
the rep turn, the value |
concern |
concern opened → cleared/expired (episode cut, cap 8 decisions) | opening client turn, rep turns, resolution turn |
next_step |
next_step_agreed ≥ 0.70 |
the exchange |
uncertain |
any gate value in 0.30-0.70 | the turn; most valuable for criteria repair |
trainer_pick (M2) |
per-episode exemplar_candidate ≥ 0.70 from the batch scorer's episode request |
the episode |
coach_seed |
coach_scores_json.quotes[].turn_idx |
the old coach's pick, as a weak hint only |
Actions and effects (M2): mark model / acceptable / avoid (+ "use as example") → marks row; a model rep turn becomes a candidate playbook line (verbatim, tokens cleaned, editable, source_moment_id set, status='draft' until approved_by) and a candidate examples snippet for the relevant criterion; avoid → not_for text and a false example; all three → gold labels for L1/L2. Edit weights/thresholds → new draft policy version, corpus re-scored from answers, before/after diff (0 Jev calls). Edit question wording → new bank_hash, harness re-scores the labelled set only (~$0.30). Approve lines → compiled into the next playbook version (playbook_hash changes; judging answers untouched). Promotion: draft → eval run → diff view (flipped L1 cases, moved exemplars, ranking deltas) → publish (D1 pointer + KV mirror + audit_log row) → background backfill under the new hashes; old rows kept for rollback. Question wording changes are drafted by an LLM from Stevan's intent in the house style and gated by the harness (corpus §3b option B); nobody edits a live prompt directly.
9. Evaluation harness & promotion gate
scripts/eval.mjs runs against POST /api/evaluate on the deployed Worker (same model path as production) and, from M2, from an admin "Run eval" button; results go to R2 eval/{run_id}/ and a row in eval_runs. The golden set is snapshotted to R2 golden/{policy_version}.json with a copy committed to the repo so CI runs without D1.
L1 runner (COPILOT-057). node scripts/eval.mjs --scenario onboarding|cs|both --layers l1 [--policy-version N] [--run <id>] [--fresh] [--dry-run]; the logic is scripts/lib/eval-l1.ts. Each case of loadAllCases() is one set:'bank' request for the named version (default published). The state is built as a session builds it: the context turns through observe(), the case's call_facts written into the SessionState (facts as locks at i: -1; a fact buildJevState would drop fails the case load), buildJevState() at the latest turn. The case's fits::/situation:: checks go in the request's fits list, which /api/evaluate adds to the full request as a session does. X-Eval-Run: <run>-<scenario> attributes the jev_requests rows. Question checks use checkPasses; engine checks run step() locally on the answers (checklist_unchanged: the held must-say set is the same after the step; risk_flags: each latched; expect_no_risk: none latched). A case passes when every check does; pass_rate is passed cases / cases. Report eval/reports/<run_id>.l1.json: {run_id, scenario, policy_version, policy_hash, bank_hash, playbook_hash, model, passed, n, pass_rate, checks_passed, checks_total, cases[]} with the per-case checks, answers observed, latency and input tokens. Answers are cached in eval/.cache/l1/ (git-ignored) by (state hash without uid, bank_hash, playbook_hash, model_expected, asked fits). playbook_hash is in the key because the request also carries next_move, phrasing:: and the gate questions. --fresh bypasses the cache and sets uid = run_id. A case whose state the Worker's isPseudonymised guard would refuse is not sent: it fails as refused. --dry-run lists these cases. On 2026-09-27, 49 of 172 cases were refused this way: rates such as 1.2690 redact as numbers, and sentence-initial words ("Company", "Don't", "Delivery") were unresolved. Since COPILOT-074b, validateCases passes every case text through the production redactor in the guard's context (cases and case reports carry redacted; since COPILOT-074c rate figures such as 1.2690 are kept, so no current case text changes on load). A changed latest turn needs redaction_reviewed, and a redacted text that isPseudonymised still refuses fails the load. COMMON_WORDS gained those ordinary words, and unresolvedIn looks up a whole contraction before cutting at the apostrophe ("Don't" resolves, "Don's" does not), so no case is refused (docs/runbook/l1-gate-triage.md).
L5 runner (COPILOT-073). node scripts/eval.mjs --scenario onboarding|cs|both --layers l5 [--policy-version N] [--run <id>]; the logic is scripts/lib/eval-l5.ts over L1's caseRequest/scoreCase. Every case of loadAllCases() is sent 5× with uid = <run_id>:<k> (k = 0..4, always fresh; the local cache is neither read nor written) under the one X-Eval-Run: <run_id>; the guard's scannedState exempts a uid that is the run id or the run id plus :<1-3 digits>. Per case and checked question, std is the population standard deviation over the answered runs of the noul value, the Score's score/(levels−1), or the Choice's probability of the option most runs picked (ties: the earliest run's), with choice_agreement = the fraction of runs picking it. A flip is a check (question or engine) that passes in some answered runs and fails in others. Report eval/reports/<run_id>.l5.json: {max_std, max_std_at, flips[], unstable[] (per question id, its largest std over the cases, highest first: the rewrite order), cases[] (runs, questions, flips)}. Recorded as layers.l5 {max_std} with the report under report.l5 and its tokens in cost_usd; report only, never a gate (a failed request exits 1). The stub's STUB_JITTER (±noise on nouls and chosen-option probabilities) and its "on the fence" rule (client_objecting exactly 0.60) exercise it in tests/unit/eval-l5.test.ts. First live run (2026-09-27, policy v2): onboarding max_std=0.0432 flips=1 (docs_status_known ≥ 0.7 passed 1/5), cs max_std=0.0367 flips=0, 615 requests, $0.35.
L4/L6 runner (COPILOT-058). node scripts/eval.mjs --scenario onboarding|cs|both --layers l4,l6 [--policy-version N] [--run <id>] [--no-fresh] [--speed N]; the logic is scripts/lib/eval-l4.ts and eval-l6.ts over replay-client.ts. Each exemplar of the scenario is replayed over the CallSession WebSocket with load{policy_version, eval_uid: <run>} (fresh by default; --no-fresh drops eval_uid and the requests == decision_points check). The others run in parallel at --speed (default 20); the shortest, 3347356034 (CS-weak, 329 s), runs alone at 1× and its per-decision utterance sent → decision received times are the end-to-end latency. L4's onboarding shapes are agreement with the labelled ground truth eval/labelled/must-say.json (docs/runbook/must-say-labels.md; read, never written). Each exemplar's said keys, not_said and not_applicable must cover the policy's checklist ids with none listed twice, else exit 2 naming the missing or duplicated ids. A labelled id the evaluated checklist no longer has (retired by the policy, e.g. the derived checklist, COPILOT-110g) is reported as retired on the agreement, the recall detail and the report lines, and left out of recall, precision and the label completeness; it never exits 2 and the label file is not edited. An id counts as persisted when signals[id].persisted held, or its checklist entry was said, on any decision of the replay (a later false-alarm dismissal does not undo the lock). Recall: every labelled said id persisted (≥ 0.8 once ≥ 5 are labelled). Precision: no checklist id outside said ∪ not_applicable persisted. The final hero's completeness must equal done / max(1, applicable), reported beside the label completeness |said| / (|said| + |not_said|). next_step_agreed persisted (any decision) exactly when the label's value is true; the risk flags seen equal the labelled ones (none on either call). Onboarding-weak's completeness is below strong's while the labels imply it; otherwise the run prints below_strong: not implied by the labels. The report lists each labelled id with its label, peak score and lock i. CS-strong ends resolved; CS-weak never reaches owned or resolved; per replay: finished, 0 errors, one model, requests == decision_points when fresh, a card shown on 50-100% of decision points. L6 takes every decision that sent a request (cache miss/moves-only) from the replay telemetry until COPILOT-059's GET /api/eval-runs/:id/requests: every input_tokens ≤ 12,000, p95 ≤ 11,000 reported only, p95 provider latency ≤ 1.5 s, cost per replayed call (Σ input tokens × $0.042/M) ≤ L6_LIMITS.cost_per_call_usd = $0.06 (decision 16; $0.05 until COPILOT-106), every model == model_expected, e2e p95 ≤ 3 s (the M1 gate) when the 1× sample ran, printed against the 1.5 s target as well (COPILOT-058c meets the target). Report eval/reports/<run>.l4l6.json: {run_id, version, policies, l4{pass, checks, agreement}, l6{…, checks}, replays[]}, each replay with per-decision telemetry and a first-occurrence timeline (must_say, persisted, risk, concern, resolution, card by i) that the demo script cites. A failed L4 or L6 check exits 1.
Eval-run persistence (COPILOT-059). src/routes/eval.ts. POST /api/eval-runs (service token only; an email actor is 403) records one scenario's run {run_id, scenario, policy_version, policy_hash, bank_hash, model, layers{l1?, l4?, l5?, l6?}, cost_usd, report} once per run id (409 run_exists). A run vouches only for the exact candidate it evaluated: the hashes must equal the named policy_versions row's (422 hash_mismatch), and every session opened with eval_uid = run_id must have pinned that version (422 hash_mismatch, field: session_policy_version). L1's rate and n are recomputed from report.cases[] (a case passes when answered, not refused, and every check is ok); an L1 layer needs n ≥ 25 and at least n jev_requests rows with run_id = run_id, an L4/L6 layer at least one eval_uid session (else 422 insufficient_evidence). passed = l4.pass && l6.pass (M1a; COPILOT-074 adds L1). The report goes to R2 eval/<run_id>/report.json first, then one batch inserts the row and, only if that insert landed, audit_log{eval.run}. GET /api/eval-runs/:id/requests lists the jev_requests rows with run_id = :id plus those of sessions with eval_uid = :id. scripts/eval.mjs posts one run per scenario after its layer set with run id <run>-<label>, which is both L1's X-Eval-Run and the replays' eval_uid (scripts/lib/eval-record.ts). L4 counts the scenario's own exemplars; L6 is the invocation's joint budget run, so both rows carry it. A layer without request evidence (L1 cases from the cache or refused, a --no-fresh replay) is left out with its reason.
Measured cost and latency (COPILOT-060). GET /api/jev-requests/stats?since=<iso>[&until=<iso>] (src/routes/eval.ts, service token only) reads every jev_requests row created in the window (at most 50,000, else 422 range_too_large) and aggregates it in the Worker (src/db/jevStats.ts): per (kind, scenario) and per kind, the attempts, the answered ones, unknown_usage_requests (error/timeout, usage NULL), Σ input/output tokens and Σ request_chars over status = 'ok' rows only, mean and p95 latency (linear interpolation, as engine.py percentile), sessions and decision points (distinct (session_id, i) of full/moves-only); per scenario the replays' decision points and tokens; and the cold-start proxy (each session's first answered request against its own median). The scenario is the recorded eval run's (L1 rows), else the call's, else unknown. scripts/cost-report.mjs --since <iso> --balance-before <usd> --balance-after <usd> (scripts/lib/cost-report.ts) writes docs/reports/cost-latency.md: the credit delta against Σ input tokens × $0.042/M gives the effective $/Mtok, and a residual above the balance's cent resolution (or 10% of the input cost) means output is charged; the measured divisor is Σ request_chars / Σ input_tokens of answered requests. It prints the proposed token_budget.chars_per_token and the command that writes it into policy/src/rules-*.json; the divisor is policy data and changes only through a new policy version.
Demo script check (COPILOT-061). docs/demo-script.md walks 3339895706 (with its recording) and 3303259297 through the dashboard; each table row is | [mm:ss] | i <n> | … \event:id` … | … |under a heading namingcall . node scripts/demo-check.mjs eval/reports/ (scripts/lib/demo-check.ts) holds each tag against the report: a timeline tag must first occur at exactly that i; unsure:(the report keeps a peak, not ani) must never have locked, peak inside thresholds.uncertain_band, and imust be a decision point.[mm:ss]must equal the floor of utterancei's tintests/fixtures/stitched/(identical to the D1utterancesrows). It also refuses fewer than 10 onboarding / 6 CS rows, and any hostname but the workers.dev one or any file path. Theiis the D1 utterance index the review view shows, not the call-coachturn_idx(stitching merges turns: coach turn 69 is#54, member_ids [69, 71]`).
| Layer | Gold | Check | Gate |
|---|---|---|---|
| L1 labelled utterances | eval/labelled/{onboarding,cs}.json: 25-40 cases per scenario, plus eval/labelled/rate.json (rate pressure, both scenarios, each case naming its own; the runner reads loadAllCases(): the playbook's five client lines, none negatives, each do-not with a positive and an approved-line negative, the qualifying question and the concession check), and eval/labelled/like-for-like.json (COPILOT-096, both scenarios: a positive on a client example and a negative for every situation, a none-of-them case per move, fits::/situation:: checks generated from the playbook; docs/policy/like-for-like.md §4.6), hand-written from the four exemplars plus synthetic edge cases: negations ("we're not worried about the rate"), an injected instruction inside client speech, "is that guaranteed?" (must not tick rep_made_guarantee), and the client paraphrase "so you never hold my money?" (said_who_holds_funds < 0.40, checklist unchanged). Coach quotes are hints for where to look, never gold: two LLMs agreeing is not a promotion gate |
per-case threshold checks as data (client_objecting >= 0.60, onboarding_objection_type in {rate}, …), the reference's report format (dissection §9b) |
≥ 90% pass: reported in M1a, the M1b exit target, a hard gate from M2 (as L5); no case regressing across a threshold vs the published version |
| L4 call shapes | the four exemplars (+ 2 per scenario in M2); onboarding labels eval/labelled/must-say.json |
onboarding (both): recall of the labelled said ids = 1 (≥ 0.8 from 5), no persisted id outside said ∪ not_applicable, completeness == done/applicable, next_step_agreed as labelled, 0 risk flags; onboarding-weak below strong's completeness while the labels imply it; CS-strong reaches resolved; CS-weak never reaches owned; every replay: zero Jev errors, requests == decision points, one model id; a card shown on 50-100% of decision points |
hard |
| L5 stability | L1 × 5 with a fresh uid in state |
per-question std ≤ 0.03; list of threshold-crossing flips (each is a criterion to rewrite first) | report in M1, gate in M2 |
| L6 budget | every run | every usage.input_tokens ≤ 12,000 (hard), p95 ≤ 11,000 (report), p95 Jev latency ≤ 1.5 s, cost per call ≤ $0.06 (decision 16), model == model_expected, e2e p95 at 1× ≤ 3 s (target 1.5 s, reported) |
hard; a model change fails L6 and opens a drift run |
| L2 marked episodes (M2) | marks |
`mean(handling_quality | model) > acceptable > avoidby ≥ 0.3;exemplar_candidate ≥ 0.70on ≥ 80% ofmodel` |
| L3 ranking (M2) | admin-ranked ~30 calls + outcomes via pd_ct_id (coverage 82% Onboarding / 64% CS, selection bias reported) |
Spearman(hero, admin rank) ≥ 0.6; AUC(completeness → activated within 30 days) reported | soft |
| Calibration (M2) | ≥ 30 labels per noul | reliability plot and ECE per noul; thresholds chosen per question at a target precision (risk flags ≥ 0.9, checklist ticks balanced), replacing the uniform 0.60/0.70 | report → thresholds |
Promotion gate. M1: a policy version's status becomes evaluated only when scripts/eval.mjs writes a passing eval_runs row whose created_at is later than the policy's, and the loader refuses to mark published otherwise; the loader also refuses any bundle containing [VERIFY] or a line without approved_by. M1a had one audited bypass (publish-policy.mjs --bootstrap, note='bootstrap'), because the DO pins only a published row and the replay had to run before the harness gate existed; COPILOT-074 (M1b) deleted it. As built: POST /api/eval-runs passes a run only when l1.pass_rate ≥ 0.90 && l4.pass && l6.pass (L1 recomputed from the report's cases), and in the same batch moves the named draft to evaluated (eval_run_id = the run) when its policy_hash/bank_hash are the run's and its created_at is before the run's, writing audit_log{policy.evaluated}; the response carries evaluated. POST /api/policy/:scenario/publish takes only {version} and publishes only an evaluated row (a draft → 409 not_evaluated). Versions are immutable, so each criteria repair round is a new candidate: build-policy.mjs --next-version → publish-policy.mjs uploads it as draft (its publish step is refused until evaluated) → eval.mjs --layers l1,l4,l6 --policy-version N (L1 reads ?version=N, the replays load{policy_version: N}, so the candidate is evaluated end to end, never the published bundle) → publish-policy.mjs --version N. Failed candidates stay draft with their eval_runs history. The [VERIFY]/approved_by refusal has no bypass at any stage. M2: the same rule enforced on the publish route with the diff view in front of it. Model drift: the DO compares response.model with model_expected on every answer, sets sessions.model_drift, shows the warning, and the nightly harness run opens a drift eval_runs row with model as the changed variable. Answers for eval are cached by (state_hash, bank_hash, model), so re-running an unchanged bank is free.
10. Compliance & PII
Pseudonymise in code, at import/upload, before anything reaches D1, the DO, Jev, R2 eval/ or the browser (redact.ts, versioned redact_version): IBAN (mod-97), UK sort code + account number, card numbers (Luhn), phone numbers (UK/intl formats and spoken digit runs ≥ 6, "four two one one oh nine" normalised first), emails, postcodes, dates of birth, passport-like tokens → typed tokens [IBAN], [PHONE], [EMAIL], [POSTCODE], [DOB]; the rep's name from call_records.agent and a rep first-name list → [REP] (employee names are personal data; said_rm_contact works on the token); client names from pd_first_name/pd_last_name → [CLIENT], plus the capitalised-token-after-Hi|Hello|Thanks|Bye heuristic for spoken names not in metadata ("Hi, Karen" in the exemplar). Amounts and currencies are kept (they drive pair_known and the amount extractor; ASSUMPTION that amounts alone are not identifiers, to confirm). Exchange-rate figures (\d{1,3}\.\d{2,5}, e.g. 1.2690) are kept too: a rate is a market price, not personal data, and Jev needs it in rate conversations. A digit run of 6 or more without a decimal point is still replaced, also beside a rate (COPILOT-074c, docs/runbook/redaction.md). utterances.redactions_json records what was replaced; no mapping is stored in M1. M2 adds a Jev pii_present noul sweep over the pseudonymised text with ≥ 0.50 routed to a redaction queue (corpus §1d).
What never enters D1: the coach's summary, quotes[].reason, risks, action_items (LLM prose that names clients) stay in R2 raw/{call_id}/coach.json; only numeric scores and turn indices are copied. Raw audio, un-redacted transcripts and nova-3 JSON live only under R2 raw/ with a lifecycle rule.
Private review context for uploads (COPILOT-068). An upload's unresolved proper nouns ({token, fragment_i}) and the operator's classifications (extra_names to redact, allow to keep) are stored by src/ingest/reviewContext.ts as immutable objects at R2 raw/review/<call_id>/<review_ref>.json, each with a fresh UUID review_ref, so a later re-stitch reproduces the same redaction and a rejected concurrent ingest cannot replace the context a transcript was built with. calls.pending_review_ref points at the context awaiting classification (set by the guarded needs_review update); calls.review_ref points at the one the accepted transcript was built with and is set only by the winning replacement batch (COPILOT-069). GET /api/calls/:id/review (src/routes/review.ts) is the single, bounded exception to "tokens never reach the browser": email actors only (a service token gets 403 email_actor_required), Cache-Control: no-store on every response, one audit_log{call.review_read} row per read (the row records the ref and a count, never the tokens), and it returns the pending context, or the accepted one when nothing is pending, never the transcript.
Scenario classification for uploads (COPILOT-067). src/ingest/scenario.ts classifyScenario(utterances, judge, ctx?) sends one Jev request (kind:'scenario') with the fixed scenario Choice {onboarding, customer_success, other} (policy/src/classify.json) over recent_transcript = the first 12 stitched utterances as {speaker, text}. It throws before any Jev call on text that fails assertRedactedText. other, a confidence below 0.60 (SCENARIO_V1.minConfidence) or a failed request gives scenario: null; scenarioStatus() then gives needs_scenario, which COPILOT-069's ingest writes. A needs_scenario call never replays: the CallSession's load answers error{scenario_unsupported} (src/session/host.ts). POST /api/calls/:id/scenario {scenario, force?} (src/routes/scenario.ts, any Access actor, only onboarding|customer_success, else 422) is the admin override. On needs_scenario it is a guarded update to scenario_source='admin', status='ready' at the same revision. On a ready call with a different scenario it is a replacement under the COPILOT-015 policy: 409 call_has_derived_rows with counts unless force, then one batch claims the call with a fresh replace_token, bumps transcript_rev (open sessions stop) and deletes the derived rows (derivedDeletes, shared with calls-import); the utterances stay. The same scenario is 200 unchanged, and processing / needs_review are 409 call_not_reviewed. Every write adds audit_log{call.scenario} with {scenario, previous, from_status, transcript_rev, force}. The live probe is scripts/probe-scenario.mjs.
Upload ingest (COPILOT-069). POST /api/calls/:id/ingest {accept_unresolved?, extra_names?, allow?, force?} (src/routes/ingest.ts, any Access actor) turns an upload's immutable STT object (calls.stt_ref; 409 stt_missing without one) into utterances. fromNova3 or fromWhisperSegments by the object's format, then redact() on every fragment with the rep first names, the allowlist plus allow, and extra_names plus every capitalised transcript token of ≥ 4 letters within one edit of an extra name or of one of its parts (nearNames, allowlisted words included; redact() and REDACT_V1 are unchanged). Unresolved tokens without accept_unresolved stop the ingest before any Jev call: a new review context (COPILOT-068) and ONE batch whose guarded update sets status='needs_review', pending_review_ref and pii_review_json {reason:'unresolved_names', unresolved_count, tokens, n_extra_names, n_allow} (WHERE transcript_rev = ? AND stt_ref = ? AND (status IN ('processing','needs_review') OR force), so a ready call re-enters review only with force), 202. Otherwise roles: mapSpeakers when every fragment has a diarised speaker, else labelSegments; a non-empty fragment left without rep/client stops the same way with reason:'speaker_map', and role_map_json/role_map_conf are stored for the admin toggle (COPILOT-070). role_map_json is {source, keyed:'speaker', map, conf} or {source, keyed:'segment', map, guesses, confs, n_review, conf}, labels and numbers only. Then stitch() → classifyScenario → the accepted review context → ONE db.batch() in the COPILOT-015 shape: the claim (transcript_rev = ?, stt_ref = ?, and unless force status IN ('processing','needs_review') plus NO_DERIVED_ROWS) bumps transcript_rev, takes a fresh replace_token, points review_ref at this ingest's context and clears pending_review_ref, and sets the role map, scenario/scenario_source='jev'/status (needs_scenario for no scenario), counts, versions, engine and pii_review_json; token-guarded deletes (derived rows only with force), the utterance inserts and audit_log{call.ingest} follow. A loser changes nothing, so a rejected concurrent ingest never re-points review_ref; 409 call_already_ingested / call_has_derived_rows / concurrent_replacement. The route counts its own subrequests (D1 reads and batches, the R2 read and write, every Jev attempt, one bufferedSink flush of jev_requests) against INGEST_V1.maxSubrequests = 45 and passes labelSegments what is left after reserving 7 for the steps after it. GET /api/calls/:id exposes pii_review.reason. Live probe: scripts/probe-ingest.mjs --call <id> [--names …] [--expect-keyed …] recomputes stitch() from the R2 object and checks Σ member_ids.
Upload resolution panels (COPILOT-072). An upload that stops in needs_review or needs_scenario is resolved in the review view's upload panel (web/src/upload.ts), which opens one resolution panel under its steps: web/src/review.ts (names), web/src/speakers.ts (speaker roles) or web/src/scenario.ts (scenario); the shared segmented choice, failure line and buttons are web/src/resolve.ts, styles web/styles/review.css. The names review reads GET /api/calls/:id/review (email actors only, audit_log{call.review_read} per read) and is the bounded exception in INV-COPILOT-004: the tokens live only in that panel's DOM and memory (never web storage, a URL or the console), and hiding the upload panel or leaving the final state calls close(), which drops them and removes the panel's DOM; opening it again is a new audited read. Each distinct token gets "is a name → redact" or "harmless → allow" with its fragment indices; Apply and ingest re-posts POST …/ingest {extra_names, allow} (the earlier classifications from the review context plus these) and Accept remaining and ingest adds accept_unresolved: true; both are enabled only once every token is classified. reason: 'speaker_map' opens the speaker editor: speaker-keyed maps get one row per diarised speaker (rep / client / other, preset from role_map) and post the complete map; segment-keyed maps list only the unknown segments of GET /api/calls/:id/segment-roles (redacted text, guess, how sure) with rep / client, and Accept all guesses posts the rows set so far with accept_guesses: true. needs_scenario opens a two-option selector posting POST …/scenario {scenario}. After any successful post the panel re-reads the call, refreshes the picker's list and shows the next state (another resolution panel, or Open replay enabled at ready). The picker links a needs_review / needs_scenario upload to ?view=review&resolve=<call_id>, which opens the panel on that call (UploadPanel.resumeCall).
Data processing. typesafe/jev on Workers AI is a "Third-party" model; whether TypeSafe's DPA / no-training / ZDR commitments cover traffic proxied through Cloudflare is unresolved (guide §4.11), and prior processing by the OpenAI coach pipeline is not a lawful basis for a new processor (struck from production-shaped on the judges' reading). Rule: until Stevan confirms (§13 Q1), client-derived text goes to Jev only for the four exported, pseudonymised samples and the synthetic L1 cases; every deliverable in §11 is written to be acceptable on that corpus. Gateway body logging is off on jev-copilot; REST calls also send cf-aig-collect-log: false. Deepgram mip_opt_out=true. Cloudflare Access with Stevan's email in front of the Worker and the Pages docs project from the first deploy, and a service token for the scripts (§4): a workers.dev or pages.dev URL with call transcripts or call-level reports is never open.
Approved-text policy. Every string a rep can see is a RepFacingText: an approved playbook line by text_id + policy version (status='approved', approved_by, approved_at), or a rewrite whose rewrites row holds verified=1 and a value for each of the verifier nouls in policy/src/verify.json (eight as of 2026-09-25, incl. improves_rate_or_offers_discount and explains_spread). The policy loader hard-rejects any bundle containing [VERIFY] or a line without approved_by; the brief's v0 playbooks carry ~10 [VERIFY] lines, so M1 needs a 30-minute red-pen sitting with Stevan (as compliance stand-in) or those lines are dropped from v1. No badge, no "draft shown in yellow".
Audit log. audit_log rows, actor = Cloudflare Access identity, for: policy upload / eval / publish / retire, role-map override, private review reads (call.review_read), label and mark writes, weight experiments (set_weights with the hashes, M2), bootstrap publish (M1a only), rewrite shown / dropped, session start / end with policy_version and model. decisions.shown_json records what was on screen at each decision point from which hashes; answers.state_json records what Jev saw. Together they reconstruct any moment of any session.
Recording disclosure. Whether calls must be announced as recorded is a Stevan/compliance question (brief §2.3); the copilot does not change what Aircall records in M1.
11. Milestones
M1: replay PoC (target 10-12 working days of agent-driven development, in two halves)
Ships on the four exported samples; scales to more calls the day Q1 in §13 is answered.
Order and cut line, so the PRD can plan against it. Summing the deliverables at their low ends gives 10.5-14.5 days (engine + DO alone is 2.5-3.5 d with the explicit queue and per-step persistence), and risk 1 says the first L1 pass will be well under 90%, so the original 8-10 days were credible only with a cut. M1a (days 1-7) is the committed half: deliverables 1-6, 8a and 9 on the four samples, demo at the end of M1a on call-3339895706.mp3 with its Aircall transcript and audio, which needs no upload path. M1b (days 8-12): deliverables 7, 8b and 10 (stretch). M1b slips before M1a does. Rows are listed in build order.
| # | Phase | Deliverable | Acceptance criteria |
|---|---|---|---|
| 1 | M1a | Infra: Worker jev-copilot (from wrangler.jsonc of jev-lab) with AI, D1, R2, DO and static-assets bindings; D1 schema v1 (§5) as migrations; AI Gateway jev-copilot with body logging off and a spend limit; Cloudflare Access on the Worker hostname with two policies (Allow: Stevan's email; Service Auth: service token jev-scripts) and on the Pages project; Jev REST token as a wrangler secret; vitest with a stubbed env.AI |
wrangler deploy from clean; migrations apply; an unauthenticated request to /api/* and /ws/* is rejected; a request carrying the service-token headers is accepted and its audit_log.actor is the token's common_name; git grep for the token strings finds nothing; the gateway logging setting and spend limit are screenshotted into the runbook; day-1 probe records whether the binding honours {gateway:{id}} (else REST fallback is switched on) |
| 2 | M1a | stitch() v1 + redact() v1 as pure modules with fixtures from data/samples/ |
the four samples stitch to 40-80 decision points each with no cross-speaker merge on hand-check; a grep over utterances.text for phone/IBAN/email/postcode patterns, the agent name and pd_* names finds nothing; stitch_version and redact_version recorded on calls |
| 3 | M1a | Importer scripts/import-call-coach.mjs + POST /api/calls/import, authenticating with the service token; --audio <file> puts a local recording in R2 raw/audio/ for the M1a demo |
the four samples import with scenario_source='call-coach', coach_scores_json numeric-only, raw transcript + coach JSON in R2 raw/; call-3339895706.mp3 lands in R2 with audio_r2_key set; inserts use ≤ 9 rows per statement; re-import is idempotent |
| 4 | M1a | Policy v1 per scenario from brief §4-5 via scripts/build-policy.mjs (Score levels flattened to strings, bank_hash over the flattened bank, token_budget.chars_per_token=3.5, + none on stage Choices, [REP]/[CLIENT] in examples, must_say/requires/blocked_by on moves, rewrite_allowed=false on safeguarding/binding/fees moves), three hashes, loader, scripts/publish-policy.mjs --bootstrap (§9) |
loader rejects a bundle with [VERIFY] or an unapproved line (unit test); no built bank contains an object-valued Score level (unit test); v1 loads only after the red-pen sitting; v1 is made published for M1a by --bootstrap with an audit_log row, and the command refuses once a passing eval run exists; one representative request through the binding returns jev-1.13.0 and an answer for every question id, including the three Scores; estimated tokens ≤ 11k at 3.5 chars/token on the longest window in the samples |
| 5 | M1a | step() engine + CallSession DO: masks, persistence (rep-turn-only said_*), concern lifecycle, stage hysteresis, CS resolution machine, hero metrics, risk flags, allowedMoves, card hysteresis, budget drop order, retry/timeout policy, explicit serial chain with pending (§4), SessionState written to ctx.storage per step and rehydrated in the constructor, serializeAttachment on the socket, answers/move_answers/decisions/moments persistence, per-hash seek cache with the moves-only path (§3), reconnect history, 402 alert |
unit tests with stubbed env.AI cover masks, rep-turn-only locks (client paraphrase does not tick), concern open/clear/expire with since refresh, resolution transitions, recompute over stored answers with zero Jev calls and a correct drift count (engine-level), budget degradation, cache hit/miss on state change, judging-hit + move-miss issuing a moves-only request and no full request; 10 utterance messages sent in a burst produce decisions in order with each state_json containing the previous decision's persisted facts; a forced eviction (or 30 s pause) mid-replay resumes with the same persisted facts and no reset; replay of onboarding-strong locks ≥ 5 must-say items and persists next_step_agreed; onboarding-weak ends lower; reload mid-replay restores the timeline |
| 6 | M1a | Dashboard (§8.1) as Worker static assets, browser-owned clock, all M1 panels for both scenarios (weights editor is M2), Range-enabled GET /api/calls/:id/audio, mark-moment form, telemetry with drift and credit alerts |
the four samples replay end to end at 1×/5×/20× with pause/seek; call-3339895706 replays in sync with the <audio> element from the R2 copy; seek to 5:00 in Safari and Chrome starts playback within 1 s; "catching up" badge appears at 20× and clears; hero panels differ by scenario; every rendered rep-facing string is a RepFacingText (type-level test); a mark writes moments + marks + audit_log |
| 8a | M1a | Eval harness scripts/eval.mjs (service-token auth against the deployed Worker) with eval/labelled/*.json (25-40 hand-written cases per scenario incl. the negative and injection cases), L4 shapes, L6 budget; eval_runs rows; reports only |
L1 pass rate reported for both scenarios (no gate yet); L4 holds on the four samples; L6: every measured usage.input_tokens ≤ 12k; report JSON committed under eval/reports/ |
| 9 | M1a | Docs on Pages ct-copilot-docs behind Access: this document, a 5-minute demo script with utterance numbers, the eval report, and a measured cost/latency report from the first replays (tokens, latency, credit-balance delta reconciled against Σ usage, effective $/Mtok, whether output tokens are charged, Σ chars / Σ usage.input_tokens written back as token_budget.chars_per_token) |
published Pages URL that answers with the Access login page when unauthenticated; the demo script reproduces on the deployed Worker; §12's estimates replaced by measured numbers; no host, path, key or credential in any published file (grep in CI) |
| 7 | M1b | Upload path: PUT /api/calls/upload (raw body streamed to R2, container sniffed, §7) → nova-3 REST from the R2 object → speaker map (heuristic + Jev Choice + toggle) → stitch → redact → D1; audio-synced replay |
data/samples/call-3339895706.mp3 uploads, transcribes, maps speakers at ≥ 0.80 or flags for the toggle, and replays in sync with the <audio> element; a non-audio file and a mislabelled file are rejected with 415 before any Deepgram call; a WAV above 25 MB is rejected with the convert-to-mp3 message; neurons per minute, the m4a/wav proxy result and the body-size result for a 30-40 min file are recorded |
| 8b | M1b | Promotion gate: L5 stability runs; policy status='evaluated' only from a passing run; --bootstrap removed |
policy v1 reaches L1 ≥ 90% on both scenarios (M1b exit target; hard gate from M2); L5 per-question std reported; a deliberately broken draft (a true/false swap) fails and cannot be marked evaluated; --bootstrap no longer exists |
| 10 (stretch) | M1b | Rewrite path behind the flag, default off | when on: a candidate quoting a rate not on the call is dropped with invents_fact ≥ 0.50 recorded; a candidate containing "guaranteed" is dropped by the denylist without a Jev call |
Out of M1: KV, Queues, corpus-wide scoring, admin console beyond the mark form, the weights/thresholds editor and recomputed UI (M2, §8.1), per-call authorisation beyond the Access gate (§4), CT-DB pre_call join, Vectorize, live audio.
M2: curation console + corpus scoring (2-3 weeks)
score-calls Queue consumer running the same step() with a batch bank (no phrasing::, no rewrites) over all usable Onboarding (1,895) and CS (1,226) calls after the DPA answer, plus one episode request per concern (resolved, response_specific, response_compliant, handling_quality, exemplar_candidate, corpus §2 Layer 2); measured rate limit and consumer CPU recorded; dead-letter queue. Admin console on Pages: review queue by information value, moment view with model/acceptable/avoid, "use as example", "promote to playbook line" with source_moment_id and approved_by; weights editor with corpus recompute; bank editor via LLM-drafted wording gated by the harness; enforced publish gate with diff view; KV mirror; audit. Jev scenario Choice with disagreement flags; speaker-role Choice for unlabelled calls; pii_present sweep; L2/L3/calibration; per-question thresholds; outcome join via pd_ct_id to broker_accounts.verified_at and trade_bookings (read access needed, §13 Q8) and logistic-regression weights after Stevan's review; pre_call block in state from clients.*; rewrite path on for CS if compliance agrees; nightly harness on model drift; R2 retention rules.
M3: live audio
Transport decision first (Aircall media stream, browser softphone capture, per-leg audio; stt §5.3); nova-3 WebSocket from the DO with interim_results, endpointing=300, utterance_end_ms=1000, KeepAlive; a complete noul + silence timer for interim commits (prior-art §2.5); finals become the same utterance messages into the unchanged engine; latency budget end-of-utterance → card ≤ 1.5 s; rep-side overlay with alert-fatigue controls (one card per turn, cooldowns, no repeat within 4 minutes); observer mode for managers; rep feedback (ack_card, thumbs) feeding labels.
12. Cost & latency budget
Unit prices: Jev $0.042/M input tokens pass-through, output free, +5% on credit purchases (verified, guide §4.6, §4.10; today's probe: 400 input tokens, keySource: Unified); nova-3 ~473 neurons/min ≈ $0.31/h (verified); a Haiku-class rewrite generator ~$1/$5 per Mtok (ASSUMPTION); D1/R2/DO/Pages inside the Workers Paid plan at this scale est.. Call shape est.: a usable call averages ~7 min (229 h / 1,895 Onboarding), ~55 decision points, ~9.6k tokens per request at the measured 3.5 chars/token (§6.2; the earlier 8.4k assumed 4 chars/token and was ~15% low).
| Item | Per replayed call | Per 1,000 historical calls |
|---|---|---|
| Jev decisions (55 × 9.6k) | 528k tokens ≈ $0.022, $0.023 with fee | ≈ $23 |
moves-only re-asks after a playbook edit (55 × 2.1k), M2 |
— | ≈ $5 per re-scored 1,000 |
Batch bank (no phrasing::, 55 × 8.3k), M2 corpus scoring |
— | ≈ $19 |
| Episode requests (~4 × 1.5k) | — | ≈ $0.25 |
| Rewrite verification, if enabled (~8 × 1.5k Jev + ~1k generator) | ≈ $0.01 | ≈ $10, only if on |
| nova-3 (uploads only, 7 min) | ≈ $0.036 | ≈ $36 for 117 h, not needed for the corpus |
| PII noul sweep (M2, ~70 utterances × 400) | — | ≈ $1.2 |
| Total | ≈ $0.023 (transcript) / ≈ $0.06 (upload) | ≈ $20-24 Jev-only |
Full usable corpus (3,121 calls, 356 h) ≈ $60-75 in Jev credits. Harness run (~150 cases × 5 for L5) ≈ $0.30. Reconcile on day one: note the credit balance, replay the four samples, compare the balance delta with Σ usage.input_tokens × $0.042/M to learn the effective rate and whether output tokens are charged on Cloudflare (guide §4.10); M1 deliverable 9 publishes the result.
Latency per decision point: Jev 0.44 s over REST measured today (400-550 ms range), ~0.8 s via the binding including cold start, steady-state binding latency unmeasured (ASSUMPTION ~0.5 s); stitching is pre-computed; decision code < 5 ms; D1 write < 20 ms est.; WebSocket push < 50 ms est.. Target p95 utterance-to-card ≤ 1.5 s: measured 452 ms at 1× on 3347356034 once the D1 commit moved behind the send (COPILOT-058c, §4 "Decision latency"); L6 gates it at 3 s for M1 and reports it against the target. At 1× replay a decision every 2-6 s leaves the loop idle; at 20× the DO's explicit serial chain (§4) falls behind by up to ~40 s over a 7-minute call, shown as "catching up"; second passes hit the seek cache and are instant. Upload transcription: 8.6 s for 9:46, so est. ~30 s for a 30-minute file. Batch scoring of 1,000 calls at 8 concurrent requests and ~0.6 s each ≈ 1.2 h; if the Workers AI Text Generation default of 300 rpm applies to Jev (undocumented, guide §4.9) ≈ 3 h. Rewrites arrive 1-3 s after the card and are never awaited.
13. Risks, unknowns, and the questions only Stevan can answer
Risks
- Messy speech vs literal criteria. The v0 questions were written against tidy phrases; Aircall fragments are garbled. Expect the first L1 pass well under the reference's 100% and expect
uncertainto be common. Mitigation: stitching quality, criterionexamplesdrawn from real turns (which is whatmomentsandmarksexist for), L5 on real turns, nova-3 re-transcription where audio exists. - No version pinning on Cloudflare (
guide §8). A silentjev-1.14moves every threshold. Mitigation:model_expectedon the policy,modelon every answer, drift alert in the session, L6 fails and opens a drift run; the harness costs ~$0.30. - 32k context and undocumented overflow. Design point 9.6k at 3.5 chars/token, soft 11k, hard 12k with a drop order; L6 hard-fails any measured request over 12k. Adding
pre_call, value-extraction Choices or more moves in M2 must go through the same gate. - Data-processing status of a third-party model on Workers AI (Q1). Everything beyond the four samples waits on it; M1 is written to be complete on those four.
- Draft must-say list and
[VERIFY]lines. Half the onboarding hero depends on wording compliance has not approved; the loader will refuse to ship the draft as is, so the red-pen sitting is on the critical path (Q3). - Diarisation on mono narrowband audio degrades on overlap (94% agreement on one call is one data point); the Jev role Choice and the toggle are the mitigations; per-leg audio in M3 removes the problem.
- Unmeasured Workers AI rate limit, steady-state binding latency and Queue-consumer CPU for Jev: all measured by M1 deliverable 9 before M2 is sized.
- LLM-assigned categories and coach scores:
categoryis treated as a label to verify, and coach quotes as hints, never gold. - Recompute approximation: threshold edits that feed back into state are not exactly replayable from stored answers; the drift count keeps this honest and a paid re-ask is one click.
- Choice option order is part of the question (
guide §3.5): a playbook diff that reorders moves changesplaybook_hashand must pass the harness like any other edit.
Unknowns to test in the first two days: whether the binding honours {gateway:{id}} and per-call log suppression (else REST); the binding request shape for nova-3 (REST is verified); the proxy body-size ceiling for a 30-40 min file; whether audio/mp4 and audio/wav survive the nova-3 proxy (only audio/mpeg is verified); real usage.input_tokens for the full bank and the chars-per-token ratio (3.5 assumed from the reference bank); whether mip_opt_out changes nova-3 billing; the cold-start profile of the DO.
Questions only Stevan can answer
- DPA / processor: may pseudonymised, client-derived transcript text be sent to
typesafe/jevthrough Workers AI (and audio to Deepgram via Cloudflare) under CT's data-processing obligations, and who signs that off? Until answered, M1 runs on the four exported samples only. Also: retention period forraw/audio and transcripts in R2. - Corpus selection for M1+: once Q1 is cleared, which calls (a rule such as top/bottom deciles by
call_value_scoreper category, or a hand-picked list)? - Must-say wording and applicability: the compliance-approved way to explain safeguarding, who holds funds, how CT is paid, the FSCS question, the £5k minimum, binding bookings; which
[VERIFY]lines inbrief §5are true as written; which items apply per client type (personal vs corporate) and per stream (PFX/CFX/SAR/IL); which payment partners reps may name. - Rewrite path: acceptable at all in onboarding, CS only, or neither? Default stays off.
- Hosting: keep the Worker on
workers.devbehind Access with Stevan's email for M1, or acurrencytransfer.comsubdomain from the start (needed before the SPA moves to Pages in M2)? Who else gets Access (RMs, managers)? M1's tenancy model is full-corpus read for every Access identity, which is fine while that identity is Stevan alone; extending Access needs either therep_reffilter on the call routes (§4) or an explicit decision, recorded here, that all Access users read the whole corpus. - CS hero: resolution state alone, or resolution plus the numeric health bar (this design shows both, health secondary)?
- Recording disclosure: must calls be announced as recorded, and is that itself a must-say item?
- Outcome join (M2): read access to CT
broker_accounts/trade_bookings(replica, nightly extract, orct-sql-skill queries by hand), and the activation window N (days) for "activated within N days of the call". client_typeper call: available from Pipedrive/CT for the checklist's applicability rules, or inferred from the transcript in M1?- Token and key housekeeping: the
cfut_token pasted today is active and reaches Jev with credits (verified);~/.config/jev/cloudflare.envalready holds a token. Should the new one replace it, and should it be an Account token rather than a user token for the deployed Worker (guide §4.5)? The token is unrelated to host access. Host access for the call-coach server is documented only in the unpublished ops runbook; because access details for that host were pasted into planning contexts before this document was scrubbed, the keys it references should be audited and rotated, and the rotation date recorded in the runbook.
14. Sources
/home/stevan/dev/jev/docs/jev-guide.md(§1 model, §2 contract and derived types, §3 semantics and thresholds, §4 Cloudflare integration: gateway, billing, envelope, errors, rate limits, terms, §5 patterns, §6 cookbooks, §7 jaggedness and prompt rules, §8 gotchas)/home/stevan/dev/jev/docs/research/reference-copilot-dissection.md(§1 state, §2 questions, §3 maths and recompute limit, §4 facts and objection lifecycle, §5 playbook and gating, §6 personalize and verifier pass-through, §7 protocol and process model, §8 video sync, §9 eval, §10 transfer notes)/home/stevan/dev/jev/docs/research/stt-options.md(§1, §3.1, §3.3, §5, "Live verification": mono Aircall audio, nova-3 REST result, speaker agreement)/home/stevan/dev/jev/docs/research/prior-art.md(§2.1decide.jshysteresis, §2.3 commitment-risk noul, §2.9 curation prior art, §4.4 cookbooks, §5 primeline calibration, §6 UI vocabulary, §8 ranked ideas)/home/stevan/dev/jev/docs/research/ct-domain-brief.md(§1 CT and journey, §2.3 must-say, §2.4 objections, §4 question banks, §4.6 composite drafts, §5 playbooks with[VERIFY], §6 questions)/home/stevan/dev/jev/docs/research/corpus-and-curation.md(§0 constraints, §1 ingestion and PII, §2 rubric layers, §3 curation actions and queue ordering, §4 playbook selection, §5 storage limits and schema, §6 harness, §8 open questions)/home/stevan/dev/jev/docs/architecture/proposal-poc-fastest.md,proposal-production-shaped.md,proposal-data-first.mdand the three judges' verdicts (task input, 2026-09-24)/home/stevan/dev/jev/docs/vendor/typesafe/model-jaggedness__jev-1.13.md,concepts__how-to-build-with-system-one.md,cookbooks__consistency_noul_cookbook.md,cookbooks__sde_cascade.md,cookbooks__classification_using_confidence.md/home/stevan/dev/jev/reference/call-coach/README.md,config/rubrics.json(onboarding and customer_service dimensions and checks),scripts/download_call_audio.py/home/stevan/dev/jev/reference/jev-sales-copilot/copilot/{engine.py,constants.py,personalize.py,server.py}via the dissection/home/stevan/dev/jev/data/samples/onboarding-strong-3339895706.{txt,json},call-3339895706.mp3,call-3339895706.nova3.json,customer-service-{strong,weak}-*.txt,onboarding-weak-3485591407.txt/home/stevan/dev/jev/data/call-coach/{call_records,call_turns,call_features}.jsonl.gz(row shapes;pd_ct_idcoverage measured by data-first)/home/stevan/dev/jev/src/index.ts,/home/stevan/dev/jev/wrangler.jsonc,/home/stevan/dev/jev/scripts/jev.mjs- Verified facts supplied with the task (2026-09-24) and re-verified in this session: REST Jev call 0.44 s,
jev-1.13.0,result.resultnesting,keySource: Unified; token active; Aircall recording download;eval/cf-parity.json(380,577 input tokens over 48 requests, the basis of the 3.5 chars/token divisor)
15. Decision log
| # | Decision | Alternatives considered | Why |
|---|---|---|---|
| D1 | poc-fastest is the M1 spine; production-shaped's guarantees and data-first's engine are grafted on | build production-shaped's M1 (RBAC, KV, Queue, Workflow eval before replay); build data-first's M1 (batch scorer and a 60-moment marking sitting before replay) | Both alternatives invert or overload the milestone Stevan defined ("step by step", replay first); two of three judges ranked poc-fastest first on fit and deliverability. The grafts cost days, not weeks |
| D2 | Pure step() engine shared by the DO now and the Queue consumer in M2 |
port the reference CallSession as a stateful class inside the DO |
Makes corpus scoring configuration rather than a rewrite; unit-testable without a DO; the judges' most-cited structural idea (data-first §2) |
| D3 | Three hashes: bank_hash for judging answers, playbook_hash for move/phrasing answers, policy_hash for decisions; state_hash on every answer |
poc-fastest's single bank hash including the playbook; production-shaped's bank/policy split without separating move answers | A playbook line edit is the main curation action; folding it into the bank hash invalidated every stored answer. state_hash fixes the seek cache serving answers to a different state. M1 has no playbook editing, so the split has no M1 consumer; it is M2 infrastructure paid for early (§2) |
| D4 | Budget with drop order (phrasing beyond top-4 → all phrasing → window 8 → skip), soft 11k, hard 12k | hard reject at 12k (poc-fastest); assert at 16k (data-first) | A 120-word window can trip a hard reject; 16k leaves less headroom under an undocumented overflow. Dropping whole fixed-option questions is hash-safe; a per-turn shortlist inside next_move is not |
| D5 | said_* locks on rep turns only, with an L1 negative case |
window nouls lock on any turn (poc-fastest, production-shaped) | A client paraphrase would tick a compliance item; the hero metric would record a must-say as delivered when it was not |
| D6 | Six single-condition verifier nouls + code denylist (eight in policy/src/verify.json since the rate encoding, 2026-09-25), verification failure = drop; safeguarding/binding/fees moves never rewritable |
one compound makes_promise_or_prediction noul (poc-fastest, production-shaped); four nouls (data-first); reference pass-through on error |
guide §7: one condition per noul, decompose and aggregate with max; a regulated line must never reach a rep unverified; scheme implication and partner naming were missing from all three |
| D7 | Browser owns the clock; DO only evaluates; "catching up" badge | DO setTimeout chain with alarm watchdog (production-shaped); DO sleep loop (data-first) |
Timers conflict with WebSocket Hibernation and drift at 10-20×; the audio element gives true audio sync for uploads; live STT sends the same messages in M3 |
| D8 | Typed RepFacingText, loader hard-rejects [VERIFY] and unapproved lines, red-pen sitting on the M1 critical path |
yellow "unverified draft" badge (poc-fastest); status='draft' lines loaded and rendered (data-first) |
A badge is a convention, not a control; the compliance judge ranked this the decisive difference |
| D9 | Dashboard as Worker static assets in M1, docs on Pages; SPA moves to Pages under a CT hostname in M2 | Pages SPA + Worker API from day one (production-shaped, data-first) | One origin = one Access application and no cross-site cookie/WebSocket work; Stevan's decision names deliverables, which do go to Pages; the move is a hosting question (§13 Q5), not an architecture change |
| D10 | Curation seeded in M1 (engine-emitted moments + one mark form), full loop in M2 | curation entirely M2 (poc-fastest, production-shaped); full loop in M1 (data-first) | Honours Stevan's curation decision inside M1 at near-zero cost without depending on a marking sitting or on the DPA answer |
| D11 | Hand-written L1 labels + synthetic edge cases, 90% reported in M1a, M1b exit target, hard gate from M2; coach quotes are hints only | bootstrap labels from coach quotes[].reason, gate 85% (poc-fastest); labels only from Jev-tagged moments (data-first); 90% as an M1 hard gate (this document's first draft) |
Two LLMs agreeing is not a gate; Jev-tagged-only labels never contain false negatives; 90% matches the reference and the other proposals; as an M1 hard gate it is an open-ended criteria-tuning loop (risk 1), not a build task |
| D12 | [REP] token; coach prose stays in R2 raw/; numeric coach fields only in D1 |
keep the rep's real name in Jev traffic (production-shaped); coach_json in D1 (all three) |
Employee names are personal data; coach summaries name clients; a "pseudonymised" store must be pseudonymised |
| D13 | DPA gate explicit: four samples + synthetic cases until Q1 is answered; "already processed by OpenAI" is not a basis | proceed on ~20-100 calls on the assumption it is fine (all three, to varying degrees) | guide §4.11 says do not send private data until resolved; the plan must not stall on a gate it raises itself, so M1 is complete on the four samples |
| D14 | Recompute declared exact for weights, approximate for feedback thresholds, with a drift count | "recompute is free" without caveat (all three) | Stored answers were given against the old known_facts/checklist_done; honesty in the UI beats a silent approximation |
| D15 | D1 batch size = floor(100 / columns) via a helper | "≤ 12 rows" (all three) | 12 rows × 11 columns = 132 bound parameters > D1's 100 |
| D16 | 402 code 2021 = operator alert, no retry; SDK-mirroring retry policy otherwise | generic 4xx fail-fast (poc-fastest) | An empty credit balance mid-replay must not look like a bad request |
| D17 | Binding with {gateway:{id}} first, REST fallback decided on day 1 |
REST only (jev.mjs shape); binding only | The binding needs no token secret, but per-request log suppression on it is unverified (guide §4.3); REST is verified at 0.44 s |
| D18 | Stage Choices get none; objection type read only when client_objecting fires; parent-bucket fallback below 0.40 |
draft stage Choices without none; type shown whenever the Choice names one |
A Choice always crowns a winner (guide §3.1); the classification-with-confidence cookbook lifts unsure cases by reporting the parent |
| D19 | Hero metrics noul-first; Scores only in the secondary health bar; no closing probability, no "activation likelihood" bar | brief §4.6's activation-likelihood composite as a bar | Stevan's decision; nouls calibrate best (prior-art §5); a probability-shaped bar invites the wrong reading |
| D20 | Mono recordings, diarise + code/Jev/admin speaker map; historical corpus needs no STT | ask Stevan whether recordings are dual-channel (all three) | Already answered by the live verification in stt-options.md; Aircall's own participant_type is the better speaker source for history |
| D21 | Explicit serial chain in the DO + SessionState stored per step + socket attachment |
rely on "DO messages are processed serially" (all three proposals); keep state in memory between messages | Input gates only close during storage ops, so awaiting Jev lets utterance events interleave at 5-20×; hibernation evicts in-memory state after a few idle seconds. Both were correctness bugs in the text, not in any proposal's code |
| D22 | Access service token for scripts; Pages docs behind Access; no hosts, paths or credentials in published docs; Jev REST token only as a wrangler secret |
email-only Access policy (scripts blocked by the login page); public docs site; access details inline in this document (first draft) | Non-interactive clients need Service Auth; the docs name calls and rep tokens; a design document that lists a working privileged path into a PII store must not be publishable |
| D23 | M1a / M1b cut line with an audited bootstrap publish; eval harness reports in M1a, gate in M1b | single 8-10 day M1 with a 90% L1 hard gate and deliverables that summed past the budget; dashboard listed before the harness its policy load depended on | The DO pins only a published row; without a bypass the replay demo was blocked on the harness (a dependency inversion in the table); the bootstrap is refused once a passing run exists and removed in M1b |
| D24 | set_weights/recomputed panel moved to M2; messages kept behind the M1 debug drawer |
required M1 dashboard feature (first draft) | Not in Stevan's milestone-1 ask; its payoff is corpus calibration, which is M2 (§6.5, §9); on four calls the dials have little to show |
| D25 | Token estimator divisor is policy data (3.5 chars/token, measured), re-derived from the first replays | 4 chars/token constant (all three proposals and the first draft) | eval/cf-parity.json gives 3.52; a divisor of 4 under-reports by ~14% and would let the estimator pass requests that breach the 12k hard gate it exists to enforce |