#CT call copilot on Cloudflare + Jev: proposal "poc-fastest"

Status: architecture proposal, 2026-09-24. Angle: fastest credible Milestone 1 with the least machinery, without painting the project into a corner. Every non-obvious claim cites a local file; assumptions are marked ASSUMPTION. Numbers marked est. are my estimates, not measurements.

#1. Summary

1. Milestone 1 is a replay copilot: pick a historical Aircall call (or upload a recording), press play, and watch a dashboard update utterance by utterance with CT signals, stage, must-say checklist, open concern, risk flags and a next-best-move card.
2. The engine is a TypeScript port of the reference copilot's CallSession (reference/jev-sales-copilot/copilot/engine.py) running inside one Durable Object per session; one speculative fan-out request to env.AI.run('typesafe/jev') per decision point; all arithmetic, gating, memory and hero metrics in code.
3. Historical calls are ingested as-is from the call-coach exports (~/dev/jev/data/call-coach/*.jsonl.gz): role internal|external gives rep/client for free, so M1 needs no STT for the corpus. Nova-3 is used only for new uploads.
4. Two question banks (Onboarding, Customer Service) and two playbooks ship as versioned JSON in the repo, seeded from the v0 drafts in docs/research/ct-domain-brief.md; their content hash is stored with every raw Jev answer so re-weighting is free and re-scoring is attributable.
5. Hero metrics follow Stevan's decision: onboarding = must-say completeness + risk flags; customer success = resolution state + call health. No closing probability anywhere.
6. Rep-facing text is pre-approved playbook lines. The tailored-rewrite path is specified (generator + three Jev verifier Nouls incl. a CT promise/prediction check), built behind a flag, off the critical path, and defaults to off in M1.
7. Storage is R2 (call JSON, audio) + a small D1 (calls, decisions with raw answers, bank versions, audit). KV, Queues, Vectorize and a separate Pages app are deferred; the dashboard is a single static page served by the Worker.
8. Token budget per request is ~8k est., asserted in code at 12k, against the 32k Cloudflare limit (docs/jev-guide.md §2.5, §8).
9. Cost: ~$0.02 per replayed call in Jev credits, ~$21 per 1,000 historical calls; nova-3 adds ~$0.04 per uploaded 7-minute call.
10. M1 is 8 deliverables, sized for 1-2 weeks of agent-driven work; M2 adds the curation console and corpus scoring; M3 adds live audio.

#2. Design stance

The angle is poc-fastest: get a convincing, real-data replay in front of Stevan in days, not weeks, and leave hooks rather than machinery for everything else. Concretely it optimises for:

• Reuse over rebuild. The call-coach corpus already has role-labelled, timestamped turns for 11,311 calls, an LLM category, and a first-pass stitch (call_turns.data.stitched_chain_len, members; see the export sample in data/call-coach/call_turns.jsonl.gz). M1 does not re-transcribe it. The reference engine's mechanics (window state, speaker masks, fact locks, concern lifecycle, confidence gates, recompute-without-Jev) are ported line for line rather than redesigned (docs/research/reference-copilot-dissection.md §10a).
• One process shape for replay and live. The browser owns the clock (an <audio> element for uploads, a virtual timer for transcript-only calls) and sends utterance messages when t_end passes; the Durable Object only evaluates. Live STT later delivers the same utterance messages, so nothing in the engine changes for M3 (docs/research/stt-options.md §1, §5.2).
• No corner-painting. Three things are non-negotiable even in M1 because retrofitting them is expensive: (a) every raw Jev answer is stored with the bank hash and model id; (b) banks, playbooks and weights are versioned data, not code constants; (c) transcripts are pseudonymised before they reach Jev or D1.

What it trades away: no admin UI, no corpus-wide scoring, no Vectorize, no CT-DB pre-call brief, no live audio. Each of those has a named slot in §11.

#3. System overview

flowchart LR
  subgraph devbox
    EXP[(call-coach exports\n*.jsonl.gz)] --> IMP[scripts/import-call-coach.mjs\nselect, stitch, pseudonymise]
  end
  IMP -->|POST /calls| W
  UP[Browser: upload mp3/m4a/wav] -->|POST /calls/upload| W

  subgraph Cloudflare account 694e…
    W[Worker jev-copilot\nHTTP routes + static dashboard]
    DO[(Durable Object\nCallSession per session)]
    R2[(R2: calls/{id}.json\nraw/audio/{id})]
    D1[(D1: calls, decisions,\nbank_versions, shown_cards)]
    AI[Workers AI binding]
    GW[AI Gateway 'jev-copilot'\nlogging off]
    JEV[typesafe/jev]
    STT[@cf/deepgram/nova-3]
    GEN[generative model\nflag-gated rewrite]
  end

  W -->|write canonical call JSON| R2
  W -->|index row| D1
  W -->|audio| AI --> STT
  DB[Browser dashboard] <-->|WebSocket /calls/:id/session| DO
  DB -->|GET audio / call JSON| W
  DO -->|per decision point| AI --> GW --> JEV
  DO -.->|off critical path| GEN
  DO -->|raw answers + shown card| D1
  DO -->|read call JSON once| R2

Data flow per replay: dashboard loads calls/{id}.json and (if present) the audio; as playback passes each utterance's t_end it sends {type:"utterance", i}; the DO stitches, decides whether this is a decision point, builds the state, calls Jev, updates facts/concern/stage/hero metrics, persists the decision row, and pushes an update snapshot.

#4. Components on Cloudflare

Worker jev-copilot. One Worker (Hono or plain fetch router, TypeScript) owning: POST /calls (accept canonical call JSON from the import script), POST /calls/upload (audio → nova-3 → canonical JSON), GET /calls, GET /calls/:id, GET /calls/:id/audio (streams from R2), GET /calls/:id/session (WebSocket upgrade forwarded to the DO), POST /evaluate (the existing jev-lab shape from src/index.ts, kept for the eval script), and static assets for the dashboard (assets binding; ASSUMPTION that Workers static assets are preferred over a separate Pages project for a one-page UI; the docs deliverables still go to Pages per Stevan's decision). Limits that matter: request body 100 MB (docs/research/stt-options.md §3.3), which bounds upload size; wrangler dev bills real AI usage (docs/jev-guide.md §4.2), so unit tests stub env.AI.

Durable Object CallSession. The reference's in-memory CallSession maps 1:1 onto a DO (docs/research/corpus-and-curation.md §5 table): single writer, strongly consistent, holds the WebSocket and the rolling window, facts, concern state, EMA'd health, stage history and the last N raw answers. One DO id per (call_id, session_nonce) so two tabs replaying the same call do not share state. Messages within a DO are processed serially, which is what we want (a 20x replay burst queues Jev calls in order). SQLite-backed storage is used only as a crash buffer; the durable record is D1. Soft limit 1,000 req/s per object is irrelevant at ~1 decision/s.

D1. Cross-call queryable truth: calls index, decisions (raw answers, one row per decision point), bank_versions, playbook_versions, weight_sets, shown_cards (audit), and moments (the M1-optional "mark this moment" button, which seeds M2 curation). Limits: 2 MB row (a decision row is ~3-4 KB), 100 bound parameters per statement (batch inserts ≤ 12 rows), 10 GB per database (docs/research/corpus-and-curation.md §5). Utterances live inside the R2 call JSON in M1; they move into a D1 utterances table in M2 when cross-call SQL is needed.

R2. Private bucket jev-copilot: calls/{call_id}.json (canonical, pseudonymised), raw/audio/{call_id}.{ext} (uploads and any Aircall recordings fetched for the sync demo), raw/stt/{call_id}.json (nova-3 output, contains un-pseudonymised text, restricted), eval/{run_id}.json. Lifecycle rule on raw/ (retention to be set by Stevan, §13). Write-once blobs, no queries.

KV. Deferred. In M1 banks and playbooks are bundled JSON identified by SHA-256; the DO reads the bundled copy. In M2, when admins publish versions from a UI, D1 becomes the source of truth and KV mirrors the published version under a version-addressed key (KV's 60 s eventual consistency is fine for version-addressed reads, not for a mutable pointer; docs/research/corpus-and-curation.md §5).

Queues. Deferred to M2 for corpus scoring (per-call messages, concurrency ≤ 8, idempotent writes keyed on (call_id, i, bank_hash)). M1 scores calls only when someone replays them.

Pages. Hosts the deliverables (this proposal, the demo guide, eval reports) per Stevan's decision. The dashboard itself is served by the Worker to keep one deploy and one origin for the WebSocket.

Workers AI models. typesafe/jev for every judgment (verified: binding ~800 ms incl. cold start, REST ~400-550 ms, answers nested at result.result over REST; no version pinning, so response.model is logged per decision). @cf/deepgram/nova-3 for uploads (verified: results.utterances with speaker/start/end/confidence, 1.9 s for 81 s of audio, ~473 neurons/min). @cf/openai/whisper-large-v3-turbo is not used: no speakers (docs/research/stt-options.md §3.1). Generative rewrite model: ASSUMPTION @cf/meta/llama-3.3-70b-instruct-fp8-fast as the CF-native default, swappable for Anthropic Haiku through AI Gateway BYOK later; behind a flag.

AI Gateway. A dedicated gateway jev-copilot with request/response logging disabled, selected via the binding's third argument { gateway: { id } } (docs/jev-guide.md §4.3). Reason: the default gateway stores request and response bodies regardless of ZDR (§4.11). Jev is billed from prepaid credits (402 code 2021 when empty); a per-gateway spend limit is set before any batch run.

#5. Data model

Canonical call JSON (R2 calls/{id}.json), compatible with the reference loader shape and the corpus memo's format (docs/research/corpus-and-curation.md §1):

interface CanonicalCall {
  call_id: string;                       // Aircall id, e.g. "3339895706"
  scenario: 'onboarding' | 'customer_service' | 'unknown';
  scenario_source: 'call-coach' | 'jev' | 'admin';
  rep_ref: string;                       // hash of agent name; display label "Rep"
  direction: 'inbound' | 'outbound';
  duration_s: number;
  recorded_at: string;                   // ISO
  source: { system: 'call-coach' | 'upload'; engine: 'aircall' | '@cf/deepgram/nova-3'; audio_key: string | null };
  speaker_map?: { [diarised: string]: 'rep' | 'client' | 'other' };   // uploads only
  utterances: Utterance[];               // stitched, pseudonymised
  coach?: { call_value_score: number; dims: Record<string, number>; quotes: { turn_idx: number; reason: string }[] }; // from call_records.data, for eval only
}
interface Utterance {
  i: number; t: number; t_end: number;
  speaker: 'rep' | 'client' | 'other';
  text: string; words: number;
  backchannel: boolean;                  // shown in transcript, never a decision point
  members: number[];                     // source fragment indices (call_turns.turn_idx or nova-3 utterance ids)
  redactions: string[];                  // e.g. ["PHONE","IBAN"]
}

D1 schema (M1):

CREATE TABLE calls(call_id TEXT PRIMARY KEY, scenario TEXT, scenario_source TEXT, rep_ref TEXT,
  direction TEXT, duration_s REAL, utterances INTEGER, decision_points INTEGER, r2_key TEXT, audio_key TEXT,
  imported_at TEXT);
CREATE TABLE bank_versions(bank_hash TEXT PRIMARY KEY, scenario TEXT, questions_json TEXT, created_at TEXT, note TEXT);
CREATE TABLE playbook_versions(playbook_hash TEXT PRIMARY KEY, scenario TEXT, playbook_json TEXT,
  approved_by TEXT, approved_at TEXT);
CREATE TABLE weight_sets(weights_hash TEXT PRIMARY KEY, scenario TEXT, weights_json TEXT, thresholds_json TEXT, created_at TEXT);
CREATE TABLE decisions(call_id TEXT, i INTEGER, bank_hash TEXT, model TEXT, answers_json TEXT,
  state_json TEXT, input_tokens INTEGER, output_tokens INTEGER, latency_ms INTEGER, decided_at TEXT,
  PRIMARY KEY(call_id, i, bank_hash));
CREATE TABLE shown_cards(session_id TEXT, call_id TEXT, i INTEGER, move_id TEXT, line_id TEXT,
  source TEXT CHECK(source IN ('playbook','tailored')), verification_json TEXT, shown_at TEXT);
CREATE TABLE moments(moment_id TEXT PRIMARY KEY, call_id TEXT, start_i INTEGER, end_i INTEGER,
  label TEXT CHECK(label IN ('model','acceptable','avoid')), tag TEXT, note TEXT, by TEXT, created_at TEXT);

Versioning rules:

• A bank is {scenario, questions}; bank_hash = sha256(canonical JSON). The DO includes the hash in every decisions row and refuses to mix hashes inside one session. Changing one word of one criterion is a new hash (the consistency cookbooks' _rubric_fingerprint discipline, docs/jev-guide.md §6).
• A playbook is {scenario, moves[]} with per-line {id, text, approved_by, approved_at, source_moment?}. The next_move and phrasing::* Choices are generated from it, so the playbook hash is part of the bank hash.
• Weights and thresholds are a separate weight_sets row. Recompute = replay decisions.answers_json through the engine with a different weight set; zero Jev calls. Unlike the reference (docs/research/reference-copilot-dissection.md §3e), the engine re-derives features from raw answers, so thresholds (persist 0.70, open concern 0.60, and so on) are retunable offline too.
• state_json is stored per decision (the exact pseudonymised state sent) so any answer can be reproduced and so the eval harness can replay a decision against a new bank without re-stitching.

#6. Per-utterance decision loop

#6.1 Utterance stitching from Aircall fragments

The exemplar data/samples/onboarding-strong-3339895706.txt shows the problem: 159 fragments in 587 s, with overlapping speech split into shards like REP: to / REP: to that, / REP: It's it's not a pay even ... cut. and interleaved backchannels (CLI: Mhmm.). Rules, applied in the import script for historical calls and in the Worker for nova-3 output:

1. Merge consecutive same-speaker fragments when the gap is ≤ 1.5 s, or the earlier fragment has no terminal punctuation. Keep members for traceability.
2. Backchannel = a fragment of ≤ 3 words from a lexicon (yeah, yes, no, mhmm, okay, right, sure, gotcha, absolutely, exactly, mm) sitting between two fragments of the other speaker. Mark backchannel: true and merge across it (the other speaker's run continues). Backchannels are rendered in the transcript but never trigger Jev.
3. Decision point = a stitched utterance with ≥ 4 words, or ending in ?, or the last utterance of a speaker run. Everything else updates the transcript only.
4. Words ≥ 120 → split at sentence boundaries into sub-utterances sharing t (keeps state literal and short).

ASSUMPTION (to check on 20 calls): this yields 45-75 decision points for a 6-10 minute call, roughly half the raw fragment count.

export function stitch(frags: Fragment[]): Utterance[] { /* rules 1-4; pure, unit-tested on the 4 samples */ }
export const isDecisionPoint = (u: Utterance) =>
  !u.backchannel && (u.words >= 4 || u.text.trimEnd().endsWith('?') || u.endsRun);

#6.2 State sent to Jev

Small, filtered, JSON object; only fields questions reference by backticked path (docs/jev-guide.md §7 prompt rules; jaggedness "Large state full of irrelevant detail").

interface JevState {
  call_facts: {
    scenario: 'onboarding' | 'customer_service';
    client_type: 'personal' | 'corporate' | 'unknown';     // M1: 'unknown' unless obvious; M2: from CT DB
    stage_history: string[];                                // last 6 distinct stages
    known_facts: string[];                                  // persisted *_known facts
    checklist_done: string[];                               // persisted said_* items
    open_concern: { open: false } | { open: true; type: string };
    resolution: 'none' | 'issue_open' | 'owned' | 'resolved';   // CS only
  };
  recent_transcript: { t: string; speaker: 'rep' | 'client'; text: string }[];  // last 12 decision-point utterances
  latest_utterance: { t: string; speaker: 'rep' | 'client'; text: string };
}

Numbers that only code needs (talk ratio, minute, word counts) stay out of the state. Token budget per request, est. at 4 chars/token, with the fixed ~300-token overhead the guide infers from the API examples (docs/jev-guide.md §4.10):

Hard assert in the DO: estimated tokens ≤ 12,000 or the request is rejected and logged; the eval harness (§9) fails the build if any measured usage.input_tokens exceeds 12,000. This keeps 2.5x headroom under Cloudflare's 32k figure and avoids the undocumented overflow behaviour (docs/jev-guide.md §2.5).

#6.3 Question bank structure per scenario

One bank per scenario, assembled at build time from four JSON files: shared.json, onboarding.json or customer-service.json, the scenario playbook, and verify.json (rewrite verifiers, sent only in the verification request). Wording starts from the v0 drafts in docs/research/ct-domain-brief.md §4.3-4.5 and §5, which already follow the jev-1.13 rules (literal single-condition Nouls, {true,false} example criteria, {what, not_for} Choices with none, 3-level descriptive Scores). The bank is data; nothing in the engine knows question names except through the signals config that maps ids to roles:

type Role = 'client_turn' | 'rep_turn' | 'either' | 'window_fact' | 'must_say' | 'risk' | 'stage' | 'concern_type' | 'next_move' | 'phrasing';
interface SignalSpec { id: string; role: Role; weight?: number; persist_at?: number; light_at?: number; }

Two additions to the drafts: every phrasing:: option carries the line id (p1) with null description plus the text, so choice is a verbatim copy and Jev cannot invent a line (docs/jev-guide.md §5.5); and must_say items are tagged on moves so code can steer the next move toward the oldest unsaid item (§6.5).

#6.4 Hero metrics in code

// Onboarding hero: must-say completeness + risk flags. No probability of anything.
function onboardingHero(s: Session): Hero {
  const applicable = MUST_SAY.filter(m => m.applies(s));            // e.g. said_docs_needed only if docs_status_known or stage kyc_and_documents seen
  const done = applicable.filter(m => s.checklist.has(m.id));
  const uncertain = applicable.filter(m => !s.checklist.has(m.id) && s.lastNoul(m.id) >= 0.30);
  const risks = RISKS.filter(r => r.fires(s));                       // rep_made_guarantee>=0.6 this turn, funding_from_third_party>=0.7 persisted,
                                                                     // booking discussed && !said_booking_is_binding, jurisdiction_concern>=0.7
  return { kind: 'onboarding', completeness: done.length / Math.max(1, applicable.length), done, uncertain, risks };
}
// CS hero: resolution state machine + call health (0..1 composite).
function csResolution(prev: Res, a: Answers, speaker: Speaker): Res {
  if (speaker === 'client' && a.issue_reported.noul >= 0.70 && prev === 'none') return 'issue_open';
  if (speaker === 'rep' && a.issue_resolved_or_owned.noul >= 0.70 && prev === 'issue_open') return 'owned';
  if (speaker === 'client' && a.client_accepts.noul >= 0.60 && prev === 'owned') return 'resolved';
  return prev;
}
const health = (f: Features) => clamp(0.5 + 0.20*centered(f.engagement) + 0.15*centered(f.trust)
  - 0.20*f.client_confused - 0.25*f.client_disengaging - 0.10*talkPenalty(f.talk_ratio_rep) - 0.15*(f.concern_open ? 1 : 0), 0, 1);

Both scenarios show call health as a secondary bar with an EMA (α = 0.40, the reference value) and a "what moved it" list; only the hero differs. centered(score) = score/(levels-1) - 0.5 so the middle level is neutral (docs/jev-guide.md §3.4).

#6.5 Gates, thresholds, anti-flicker

All thresholds live in weight_sets.thresholds_json; defaults are the reference's, which are starting points, not tuned values:

Retry policy mirrors the SDK defaults (408/429/5xx, 2 retries, 500 ms doubling, jitter); fail fast on 4xx (docs/jev-guide.md §4.8).

#6.6 Tailored-rewrite path (flag-gated, off critical path)

Specified in full, built in M1 behind REWRITE_ENABLED=false, enabled once Stevan and compliance accept §10's policy:

1. Trigger (reference rules): card un-gated, move not in a skip list, ≥ 2 decisions since last rewrite, and (move changed, or a new fact locked, or 8 decisions on the same move).
2. Generator (ctx.waitUntil, 3 s timeout, ≤ 40 words): prompt = move title/what + its approved lines + last 6 utterances + known facts; instruction "use only details in the transcript; never state numbers, rates, dates, partner names or promises that were not said". Code cleans and length-checks the line.
3. Code denylist before Jev: regex on guarantee|always cheaper|will (go|come) (up|down|back)|FSCS|risk[- ]free|definitely → drop.
4. Jev verification (a second, small request; state = {candidate_line, move:{title,what}, recent_transcript, known_facts}), three Nouls, all true = bad: invents_fact (drop ≥ 0.50), off_move (drop ≥ 0.50), and CT-specific makes_promise_or_prediction: "Does candidate_line promise or guarantee a rate, saving, arrival time or outcome, predict which way an exchange rate will move, or state that funds are protected by a specific scheme?" (drop ≥ 0.30, deliberately strict). Verification unavailable → drop (the opposite of the reference's pass-through, dissection §6d), because the line is a compliance surface.
5. Stale check (move changed meanwhile → drop), then a phrasing WebSocket message; the card renders it under the approved lines, visibly labelled "tailored (verified)". Every shown or dropped candidate goes to shown_cards with its verification scores.

#7. Ingestion & STT

Historical Aircall calls (no STT). scripts/import-call-coach.mjs reads the three exports (psql COPY escaping: unescape backslashes before JSON.parse), selects calls by category ∈ {Onboarding, Customer Service}, duration ≥ 3 min and ≥ 20 turns, and for each: maps role internal→rep, external→client, drops external_phone and other metadata, stitches (§6.1), pseudonymises (§10), attaches the coach record's call_value_score, six dimension scores and quotes[].turn_idx (eval labels only), and writes canonical JSON. M1 imports the 4 exemplars plus ~20 calls Stevan picks (or the top/bottom deciles by call_value_score per category). scenario comes from category with scenario_source='call-coach'; the Jev scenario classifier (docs/research/corpus-and-curation.md §1e) is M2.

New audio uploads (nova-3). Browser → POST /calls/upload (multipart, mp3/m4a/wav, ≤ 100 MB) → R2 raw/audio/ → env.AI.run('@cf/deepgram/nova-3', …) with diarize, utterances, punctuate, smart_format, language=en-GB, numerals, keyterm (CurrencyTransfer, GBP, EUR, SWIFT, IBAN, mid-market, forward, drawdown, safeguarding). The REST shape is verified (binary body + query params); ASSUMPTION the binding takes {audio:{body, contentType}, ...params} and needs the same one-time check as the whisper test did. Output results.utterances[] → fragments {start, end, speaker, transcript} → stitch → canonical JSON with source.engine='@cf/deepgram/nova-3'. Observed cost: 473 neurons/min ($0.31/h). Long-file ceiling on the Cloudflare proxy is unverified (docs/research/stt-options.md §5.1 test 2); the upload route returns a clear error rather than chunking in M1.

Mapping diarised speakers to rep/client. Three layers, all in the Worker: (1) heuristics: on an outbound call the first speaker after ring is the rep; the speaker who says "Currency Transfer" or a known agent first name in the first 60 s is the rep; (2) one Jev request with a Choice per speaker cluster over its first 6 turns (rep / client / other, act at confidence ≥ 0.80); (3) a swap toggle in the dashboard that rewrites speaker_map and re-saves the call. If the recording is dual-channel, multichannel=true replaces diarisation entirely; whether Aircall recordings are stereo is an open question (§13).

Audio for historical calls. Recordings are downloadable via reference/call-coach/scripts/download_call_audio.py (Aircall API, credentials on the call-coach server). M1 fetches 2-3 recordings for the exemplars so the audio-synced demo runs on real calls; the Aircall transcript is kept as the text (no nova-3 cost) and the audio only drives the clock. ASSUMPTION: recording URLs are still valid for 2025-12 calls.

#8. Dashboard and Admin console

Dashboard (one static page, TypeScript, no framework; Chart.js as in the reference):

WebSocket protocol (reference's, extended; dissection §7c):

// client -> DO
{ type: 'load', call_id, bank_hash?, weights_hash? }
{ type: 'utterance', i }                      // browser clock passed utterances[i].t_end
{ type: 'seek', i }                           // rewind: DO resets and re-evaluates 0..i (real Jev calls; cached by (call_id,i,bank_hash) in D1)
{ type: 'set_weights', weights, thresholds }  // recompute from stored answers
{ type: 'mark_moment', start_i, end_i, label, tag, note }
{ type: 'set_speaker_map', map }              // uploads only
// DO -> client
{ type: 'hello', model, bank_hash, playbook, weights, thresholds }
{ type: 'update', i, speaker, text, decision: boolean, hero, stage, concern, coaching, signals, talk, health, jev, sensitivity }
{ type: 'recomputed', ...same as update for the last decision }
{ type: 'phrasing', i, move_id, text|null, rejected, reason, verification }
{ type: 'error', message }

Seek is cheap: before calling Jev the DO checks decisions for (call_id, i, bank_hash) and reuses stored answers, so scrubbing back and forth costs nothing after the first pass.

Admin console (M2; the data model is ready in M1). Panels: ranked calls per scenario (composite from stored decisions, coach score, rep, engine); a moment browser listing Jev-tagged moments (concern episodes, must-say locks, risk flags, uncertain-band decisions sorted lowest-confidence first); for each, the transcript excerpt and buttons model / acceptable / avoid plus a tag from a fixed taxonomy (scenario × topic). Effects: model moments become candidate examples inside the relevant criterion and candidate playbook lines (cleaned of tokens, edited, then approved_by set); avoid moments become not_for text and false examples; all three become gold labels for the harness. A weights editor (same recompute loop as the dashboard, applied corpus-wide from decisions), a bank editor (edit instructions/criteria/examples → new draft hash), and a promotion flow: draft → harness run on the golden set → diff of flipped cases → publish (D1 published=1, KV mirror) → background backfill of the corpus under the new hash via Queues. Old hashes are kept for rollback.

#9. Evaluation harness & promotion gate

M1 ships the harness in the reference's shape (tests/test_integration_jev.py, tests/test_e2e_replay.py per dissection §9), pointed at the Worker:

• Labelled utterances (eval/labelled/{onboarding,cs}.json): 30 cases per scenario built from the four exemplars and from the call-coach quotes[].turn_idx + reason pairs (weak labels; an LLM-written reason like "Client concerned about tax residency implications of address proof" maps to client_asked_about_documents ≥ 0.6, onboarding_objection_type ∈ {process_friction}). Each case = context turns + latest utterance + threshold checks (T_ON 0.60, T_FACT 0.70, T_OFF 0.40). Gate: ≥ 85 % pass in M1 (the reference reached 100 % on cleanly written text; Aircall speech is messier), ≥ 90 % from M2.
• Call-shape assertions on the exemplars: onboarding-strong ends with completeness ≥ 0.5 and next_step_agreed persisted and no risk flag; onboarding-weak ends with completeness < strong's; CS-weak never reaches owned; CS-strong reaches resolved; every replay has zero Jev errors, requests == decision points, one model id.
• Stability: 5 re-runs of the labelled set with a fresh uid in state; per-question std ≤ 0.03; report decisions that cross a threshold (those questions get rewritten first).
• Calibration (M2, once ≥ 100 admin labels exist): reliability plot per question type; Nouls are expected to calibrate best (primeline ECE 0.012 vs 0.254 for Score, prior-art §5), which is why hero metrics avoid Scores.
• Budget: input_tokens ≤ 12,000 on every request (hard), p95 latency ≤ 1.5 s through the binding (report), cost per call ≤ $0.05 (hard).
• Promotion gate (M2): a bank/playbook/weights change is published only after L1 pass rate, call shapes and budget pass, and the admin has reviewed the diff of flipped cases. Model drift is handled the same way: the harness runs nightly and on any change of response.model (Cloudflare does not pin versions).

scripts/eval.mjs runs everything with the existing REST harness pattern in scripts/jev.mjs, writes eval/reports/<date>.json, and posts a summary to the Pages site.

#10. Compliance & PII

• Pseudonymise before Jev, D1 or the browser. In code, at import/upload: strip external_phone and all Aircall metadata; regex [PHONE] (UK/intl formats and spoken digit runs ≥ 6), [EMAIL], [IBAN] (mod-97 check), [SORT_CODE]/[ACCOUNT], [CARD] (Luhn), [POSTCODE]; replace the agent's name with "the rep" and the Pipedrive pd_first_name/pd_last_name with [CLIENT]; a small name list of known reps. Spoken names not present in metadata are the known gap ("Hi, Karen" in the exemplar); M1 mitigates by keeping a Stevan-approved allowlist of calls and by adding a Jev pii_present Noul sweep per utterance in M2 (docs/research/corpus-and-curation.md §1d). Raw nova-3 JSON and audio stay in R2 raw/, never leave the Worker, and get a retention rule.
• Data processing. Jev on Workers AI is a third-party model; whether TypeSafe's DPA/no-training/ZDR terms cover proxied traffic is unresolved (docs/jev-guide.md §4.11). Until Stevan confirms, only pseudonymised transcripts go to Jev, on a small approved subset. AI Gateway logging is disabled on the dedicated gateway (the binding's per-call log switch is not documented; the gateway-level setting is). Deepgram: pass mip_opt_out=true (docs/research/stt-options.md §5.1). Cloudflare Access in front of the Worker from day one (a workers.dev URL with call transcripts must not be open).
• Approved-text policy. Every string that can be shown to a rep is a playbook line with approved_by and approved_at, or a tailored line that passed the three verifiers and is labelled as such. [VERIFY]-marked lines in the v0 playbooks (docs/research/ct-domain-brief.md §5) are shown with a yellow "unverified draft" badge in M1 so the demo is honest; they must be cleared before any rep sees the tool.
• Audit log. shown_cards records what was on screen at which utterance, from which playbook hash, with verification scores; decisions records the exact state and answers. Together they reconstruct any moment of any session.

#11. Milestones

#M1: replay PoC (target 1-2 weeks of agent-driven development)

Optional in M1 if time allows: the "mark moment" button and moments table (one form, one insert); the rewrite path behind its flag.

#M2: curation console + corpus scoring

Queues-driven scoring of all usable Onboarding and CS calls (3,121 calls) under the published bank hash; D1 utterances table; scenario classifier and speaker-role Choice for calls without labels; admin console (moment browser, model/acceptable/avoid, examples and playbook-line promotion, weights and bank editors, promotion gate with harness run and diff); KV mirror of published versions; pii_present sweep; calibration report; tailored rewrite enabled after compliance sign-off; CT-DB pre_call brief in state; outcome join (activated / traded within N days) for weight learning.

#M3: live audio

Transport decision first (Aircall media stream, browser softphone capture, or per-leg audio); nova-3 WebSocket from the DO with interim_results, endpointing=300, utterance_end_ms=1000, KeepAlive; finals become utterance messages into the unchanged engine; latency budget end-of-utterance → card ≤ 1.5 s; rep-side overlay UI; alert-fatigue controls (one card per turn, cooldowns).

#12. Cost & latency budget

Unit prices: Jev $0.042/M input tokens pass-through + 5 % on credit purchases, output free (verified); nova-3 ~473 neurons/min ≈ $0.31/h (verified); D1/R2/DO at these volumes are within free or negligible tiers est..

Sanity check against the corpus: usable Onboarding + CS = 3,121 calls, 356 h; scoring all of it once ≈ $70 in Jev credits.

Latency per decision point: Jev 400-550 ms warm over REST, ~800 ms via the binding with cold start (verified); stitching and persistence < 20 ms est.; WebSocket push < 50 ms est.. At 1x replay (a decision every ~6 s) the loop idles; at 20x the DO's serial queue falls behind by up to ~40 s over a 7-minute call and the UI shows a "catching up" badge; the seek cache makes second passes instant. Upload transcription: 1.9 s for 81 s of audio ⇒ est. ~15-20 s for a 10-minute file (linear extrapolation, unverified). Batch scoring of 1,000 calls at 8 concurrent requests and ~0.6 s each ≈ 1.3 h; the effective Workers AI rate limit for typesafe/jev is undocumented (docs/research/corpus-and-curation.md §0) and must be measured in M2.

#13. Risks, unknowns, and the questions only Stevan can answer

Risks

1. Messy speech vs clean criteria. The v0 questions were written against tidy example phrases; Aircall fragments are garbled ("It's it's not a pay even it's three like a to pay four. cut."). Expect the first labelled-set pass rate to be well under the reference's 100 %; the fix is stitching quality plus criterion examples drawn from real turns, which is why moments and examples exist. Mitigation in M1: the 85 % gate and a stability run.
2. 32k context and no version pinning on Cloudflare. The bank sits at ~9k est.; the assert at 12k protects the limit, but overflow behaviour is undocumented. A silent model bump changes thresholds; the harness runs on every model change.
3. Data-processing status of a third-party model on Workers AI is unresolved; M1 uses pseudonymised text on an approved subset only.
4. Diarisation on uploads. Nova-3 attribution degrades on overlap and narrowband audio; the manual swap and the Jev role Choice are mitigations, dual-channel recordings would remove the problem.
5. Draft must-say list and [VERIFY] lines. Half the onboarding hero depends on wording that compliance has not approved; the demo shows drafts with badges.
6. Coach labels are LLM-generated, not human; using quotes as eval labels bootstraps quickly but must be replaced by admin labels in M2.
7. Unmeasured rate limits and cold starts on the binding; replay at high speed may queue.

Unknowns to test in the first two days: binding request shape for nova-3; long-file ceiling on the proxy; actual usage.input_tokens for the full bank; whether the binding lets us select the dedicated gateway and whether that disables logging; whether Aircall recording URLs from 2025-12 still resolve.

Questions only Stevan can answer

1. Which 20 calls (or which selection rule) for M1, and may their pseudonymised transcripts go to Jev on Workers AI now?
2. Are Aircall recordings mono or dual-channel, and are the Aircall API credentials on the call-coach server usable from devbox to fetch 2-3 recordings?
3. The must-say list per scenario in approved wording (safeguarding, who holds funds, how CT is paid, FSCS wording, minimum, binding booking), and which [VERIFY] playbook lines are acceptable as-is (brief §6 Q5-9).
4. Is client_type (personal vs corporate) available per call from Pipedrive/CT for the checklist's applicability rules, or should M1 infer it from the transcript?
5. Should the "activation likelihood"-style composite from the brief (§4.6) appear at all as a secondary bar, or is anything probability-shaped off the screen?
6. Hosting and access: workers.dev behind Cloudflare Access with Stevan's email, or a currencytransfer.com subdomain from the start?
7. Retention for raw/ audio and transcripts in R2, and whether Deepgram mip_opt_out is required.
8. Rewrite path: leave the flag off for the whole of M1, or enable it for the CS scenario only?

#14. Sources

• /home/stevan/dev/jev/docs/jev-guide.md (contract §2, semantics §3, Cloudflare integration §4, patterns §5, jaggedness §7, gotchas §8)
• /home/stevan/dev/jev/docs/research/reference-copilot-dissection.md (state §1, questions §2, maths §3, facts/objections §4, playbook/gating §5, personalize §6, protocol §7, UI/video sync §8, eval §9, transfer notes §10)
• /home/stevan/dev/jev/docs/research/stt-options.md (§1, §3.1, §3.3, §5)
• /home/stevan/dev/jev/docs/research/prior-art.md (§2.1 decide.js anti-flicker, §2.3 commitment-risk Noul, §4.4 cookbooks, §5 primeline calibration, §6 UI takeaways, §8 ranked ideas)
• /home/stevan/dev/jev/docs/research/ct-domain-brief.md (journey §1.2, must-say §2.3, objections §2.4/§3.3, question banks §4, hero metrics §4.6, playbooks §5, questions §6)
• /home/stevan/dev/jev/docs/research/corpus-and-curation.md (constraints §0, ingestion §1, rubric §2, curation §3, playbook §4, storage §5, harness §6)
• /home/stevan/dev/jev/docs/vendor/typesafe/model-jaggedness__jev-1.13.md
• /home/stevan/dev/jev/reference/call-coach/README.md, /home/stevan/dev/jev/reference/call-coach/config/rubrics.json (onboarding and customer_service dimensions), /home/stevan/dev/jev/reference/call-coach/scripts/download_call_audio.py
• /home/stevan/dev/jev/data/samples/onboarding-strong-3339895706.{txt,json}, /home/stevan/dev/jev/data/samples/customer-service-weak-3347356034.txt
• /home/stevan/dev/jev/data/call-coach/{call_records,call_turns,call_features}.jsonl.gz (field shapes inspected)
• /home/stevan/dev/jev/src/index.ts, /home/stevan/dev/jev/scripts/jev.mjs
• Verified facts supplied with the task (2026-09-24): Jev REST/binding latency and response nesting, unified billing, nova-3 and whisper measurements, corpus counts.