Jev call copilot CurrencyTransfer research and architecture built 2026-09-28

Measured cost and latency

Measured from every Jev request logged between 2026-09-27T07:13:10.000Z and 2026-09-27T07:20:51.019Z (770 attempts: 770 answered, 0 without usage), Worker version 2f03062938a0. Generated 2026-09-27T07:20:51.356Z. These numbers replace the estimates in the architecture document §12.

Requests by kind

kind requests answered no usage Σ input tokens Σ output tokens mean latency ms p95 latency ms
eval 172 172 0 2,358,483 456,557 360 472
full 598 598 0 4,668,010 718,195 341 450
total 770 770 0 7,026,493 1,174,752

Latency is the provider round trip of answered requests; p95 is linearly interpolated.

By kind and scenario

kind scenario requests answered Σ input tokens mean latency ms p95 latency ms
eval customer_success 81 81 1,075,863 344 425
eval onboarding 91 91 1,282,620 373 521
full customer_success 306 306 2,207,139 330 441
full onboarding 292 292 2,460,871 353 478

Replays per scenario

scenario replay sessions decision points Σ input tokens tokens per decision point cost per replayed call
customer_success 4 306 2,207,139 7,213 $0.0232
onboarding 4 292 2,460,871 8,428 $0.0258

Cost per replayed call is Σ input tokens ÷ replay sessions × $0.042/M (output not priced); a session that stopped early counts as one call.

Credit balance against usage

credit delta: not measured (dashboard-only); effective $/Mtok assumed at list price $0.042/M input, output free

The AI Gateway "Credits Available" balance has no API. Reading it just before and just after a replay and re-running with --balance-before <usd> --balance-after <usd> reconciles the window.

AI Gateway analytics cross-check

aiGatewayRequestsAdaptiveGroups for gateway jev-copilot over the same window, independent of the request log above.

model requests tokens in tokens out gateway cost cost per Mtok in
typesafe/jev 765 6,990,232 1,169,400 $0.2936 $0.0420

The two are counted independently: the gateway sees every caller routed through it (local scripts as well as the Worker) and records its own view of attempts that failed, so its figures differ from the request log by a few percent.

Characters per token

Σ request characters ÷ Σ input tokens over the 770 answered requests: 25,261,132 ÷ 7,026,493.

measured chars_per_token: 3.6

unknown_usage_requests: 0 (0 characters sent without a usage figure; not in the divisor).

Cold start

A session's first answered request against its own median, as the cold-start proxy.

scenario sessions first request mean ms session median mean ms first − median ms
customer_success 4 822 304 517
onboarding 4 940 313 627

M1b: STITCH_V2 (104e)

Eval run m1b-stitch2-20260927 (L4 + L6, policy v4 for both scenarios, fresh answers, Worker a3bbf38), the first on the v2 stitch: the four sample calls were re-imported with sentence-sized parts on 2026-09-27. Compared with the M1a final run final-20260927 on the v1 stitch, at the same policy and model (jev-1.13.0). Cost is Σ input tokens × $0.042/M, as above.

call scenario decision points v1 → v2 cost per call v1 → v2 change
3339895706 onboarding 96 → 128 $0.0345 → $0.0460 +33 %
3485591407 onboarding 50 → 70 $0.0172 → $0.0237 +38 %
3303259297 customer_success 131 → 164 $0.0396 → $0.0494 +25 %
3347356034 customer_success 22 → 43 $0.0068 → $0.0131 +93 %
figure v1 (final-20260927) v2 (m1b-stitch2-20260927)
requests 299 405
max input tokens 10,511 10,411
p95 input tokens 9,395 9,332
p95 provider latency ms 498 440
e2e p95 ms (3347356034 at 1×) 457 (paced, gaps capped at 6 s) 5,319 (wall clock, COPILOT-118a)

Cost grows with the number of decision points, not with the prompt: the per-request input size is unchanged. Every call stays under the L6 limit of $0.05 per call, but 3303259297 is at $0.0494, 1.2 % under it.

The L6 e2e gate (≤ 3,000 ms) failed on this run. This is the first eval on wall-clock pacing: each utterance is sent at its own t_end, so real gaps are left in. The 43 figures have a median of 349 ms. Three consecutive decisions (i 26, 27, 28) took 10,894, 5,401 and 5,319 ms, and the p95 of 43 samples is the third-worst figure. The replay has no reconnect counter for this path, so the cause is inferred rather than measured. The likely cause is the session waking after hibernating in a gap, as COPILOT-118a found; the old 6 s cap on gaps hid that. It is not a stitch effect.

M1b: cadence re-tune (106)

Policy v12 for both scenarios (stitch_version 2), Worker a82fe8e, model jev-1.13.0, fresh answers. The counters that were tuned for one decision per turn now match the sentence-level cadence of the v2 stitch: concern_max_age 8 → 12, card_cooldown_utts 3 → 4, token_budget.window 12 → 16, purpose_max_decisions 8 → 12 and COVERED_DWELL 4 → 6. Kept: confirm_updates 2, stage_confirm 2, ema_alpha 0.4, soft 11,000 and hard 12,000 (decision 15).

The L6 cost gate is now $0.06 per call (L6_LIMITS.cost_per_call_usd, decision 16). The $0.05 limit quoted in the sections above was the gate when they were written.

Gate run m1b-tune-20260928b (L1 + L4 + L6), exit 0: L1 onboarding 97/97, customer success 80/82, L4 PASS, L6 PASS. The first run, m1b-tune-20260928, failed only L4 zero_errors: 2 unanswered decisions on each strong call, from "2018: Invalid User Credentials" upstream errors between 04:23:56 and 04:24:02, the same transient seen in COPILOT-122's gate.

call scenario decision points cost per call (104e, v4) cost per call (v12)
3339895706 onboarding 128 $0.0460 $0.0452
3485591407 onboarding 70 $0.0237 $0.0240
3303259297 customer_success 164 $0.0494 $0.0495
3347356034 customer_success 43 $0.0131 $0.0127
figure m1b-tune-20260928b
requests 405
max input tokens 10,266
p95 input tokens 9,037
p95 provider latency ms 410
e2e p95 ms (3347356034 at 1×) 442

The wider window costs little: no request in the run needed a budget drop. Of the 810 decisions in both runs, window_to_8 fired on 0 (the baseline was 10.5 %), and every request went out at window 16. The worst case the unit tests build (the longest 16-utterance v1 window that ends on a rep turn, with a rate concern open) is 11,876 tokens for onboarding and 10,675 for customer success, both after window_to_8. Real v2 turns are shorter than those v1 turns.

M1b: pre-judge of the item about to open (108b)

Policy v14 for both scenarios, Worker bed9a1e, model jev-1.13.0, fresh answers. On client turns the request also asks fits:: (up to 3) and the situation::<move> Choice for the item that would become current next, so a new item can open with points instead of "Listening" (decision 3 R6). The pre-judge is a list of its own, and the code-rule step prejudge drops it first when a request is over the soft budget, before fact_values. v14 differs from v13 only in token_budget.drop_order (prejudge added), so the bank and playbook hashes are unchanged.

Gate run m1b-prejudge-20260928b (L1 + L4 + L6, --fresh), exit 0: L1 onboarding 96/97, customer success 79/82, L4 PASS, L6 PASS. The first run, m1b-prejudge-20260928 (exit 0), answered every L1 case from the local cache, so it recorded nothing as evaluated. The four L1 misses in run b have all failed in earlier runs on the same bank: lfl_cs_payment_mentioned_no_amount in 23 of 23 runs, lfl_cs_upcoming_payments_unknown in 9 of 23, rate_ob_neg_rep_first_one_line (0.41 against < 0.4) in 7 of 23, and client_confused_sender_details (0.67 against ≥ 0.7) in 7 of 23.

call scenario decision points cost per call (v12, 106) cost per call (v14) delta
3339895706 onboarding 128 $0.0452 $0.0457 +$0.0005
3485591407 onboarding 70 $0.0240 $0.0246 +$0.0006
3303259297 customer_success 164 $0.0495 $0.0496 +$0.0001
3347356034 customer_success 43 $0.0127 $0.0127 $0.0000
figure m1b-tune-20260928b (v12) m1b-prejudge-20260928b (v14)
requests 405 405
max input tokens 10,266 10,340
p95 input tokens 9,037 9,256
p95 provider latency ms 410 424
e2e p95 ms (3347356034 at 1×) 442 799

The pre-judge costs less than expected ($0.0005-0.0006 per onboarding call against the $0.003-0.005 estimated). It is asked only while a next item exists and has candidate situations. On the typed templates, a fact item has no row until it surfaces, and a must-say waits until it is due, so often there is none. Of the 405 decisions in run b, 4 dropped prejudge for budget and none dropped anything else.

Live replay of 3339895706 on v14 (session m1b-108b-20260928), with scripts/ws-probe.mjs --count-points, against 108a's v13 session m1b-108a-20260928:

figure v13 v14
decisions_with_point 35 33
items_with_point 5 6
withdrawn_within_12s 2 4
items opening with a situation on their first decision 0 of 11 1 of 11

In the v14 session, explain_who_holds_the_money opened at i 146 already showing is_my_money_safe. Of the other 10 opens, 4 were inserted concern items, which the plan cannot foresee, and 1 was the call's first item. For the remaining 5 (i 10, 17, 28, 95 and 141), this probe does not show why they opened unseeded: the target might not have been pre-judged on a client turn, or its answers might not have passed the §3.3 rule. Each replay gets fresh answers, so a single call cannot tell a 2-decision difference from variance.

M1b: the whole milestone (118)

The cost report over the M1b window, from the first M1b iteration to the final verification (COPILOT-118): node scripts/cost-report.mjs --since 2026-09-27T14:43:47Z --out /tmp/m1b-cost.md, exit 0, Worker 5767218623d1 (the deployed HEAD for the final pass). Every replay, eval and probe of the milestone is in it, not only the final run. The final run's own cost per call is in eval/reports/m1b-final-20260928.l4l6.json (L6 PASS, $0.0495 per call at most against the $0.06 limit).

Measured from every Jev request logged between 2026-09-27T14:43:47.000Z and 2026-09-28T08:12:57.686Z (15,776 attempts: 15,195 answered, 581 without usage), Worker version 5767218623d1. Generated 2026-09-28T08:12:58.296Z. These numbers replace the estimates in the architecture document §12.

Requests by kind

kind requests answered no usage Σ input tokens Σ output tokens mean latency ms p95 latency ms
eval 3,546 3,546 0 49,025,409 9,472,487 349 448
full 12,168 11,587 581 90,756,150 13,870,067 336 440
moves-only 62 62 0 167,042 15,795 469 1135
total 15,776 15,195 581 139,948,601 23,358,349

Latency is the provider round trip of answered requests; p95 is linearly interpolated.

By kind and scenario

kind scenario requests answered Σ input tokens mean latency ms p95 latency ms
eval customer_success 1,636 1,636 21,890,644 337 426
eval onboarding 1,910 1,910 27,134,765 359 482
full customer_success 5,351 5,077 36,250,856 335 428
full onboarding 6,817 6,510 54,505,294 337 448
moves-only onboarding 62 62 167,042 469 1135

Replays per scenario

scenario replay sessions decision points Σ input tokens tokens per decision point cost per replayed call
customer_success 49 5,132 36,250,856 7,064 $0.0311
onboarding 97 6,634 54,672,336 8,241 $0.0237

Cost per replayed call is Σ input tokens ÷ replay sessions × $0.042/M (output not priced); a session that stopped early counts as one call.

Credit balance against usage

credit delta: not measured (dashboard-only); effective $/Mtok assumed at list price $0.042/M input, output free

The AI Gateway "Credits Available" balance has no API. Reading it just before and just after a replay and re-running with --balance-before <usd> --balance-after <usd> reconciles the window.

AI Gateway analytics cross-check

aiGatewayRequestsAdaptiveGroups for gateway jev-copilot over the same window, independent of the request log above.

model requests tokens in tokens out gateway cost cost per Mtok in
typesafe/jev 17,308 141,521,506 23,471,983 $5.9439 $0.0420

The two are counted independently: the gateway sees every caller routed through it (local scripts as well as the Worker) and records its own view of attempts that failed, so its figures differ from the request log by a few percent.

Characters per token

Σ request characters ÷ Σ input tokens over the 15,195 answered requests: 501,361,724 ÷ 139,948,601.

measured chars_per_token: 3.58

unknown_usage_requests: 581 (16,120,325 characters sent without a usage figure; not in the divisor).

Cold start

A session's first answered request against its own median, as the cold-start proxy.

scenario sessions first request mean ms session median mean ms first − median ms
customer_success 49 806 295 511
onboarding 97 875 372 504
2,417 words · Internal working documents. Do not share outside CurrencyTransfer.