Speech-to-text options on Cloudflare for the CurrencyTransfer call copilot
Research date: 2026-09-24. Scope: STT for (A) a proof of concept that uploads a recorded call (mp3/m4a/wav, 5-40 min, two speakers) and replays it utterance-by-utterance so the dashboard behaves as if live, and (B) live call audio later. Every non-obvious claim carries a source. Things I could not verify are marked ASSUMPTION or UNVERIFIED.
Local reference: /home/stevan/dev/jev/reference/jev-sales-copilot (the open-source copilot). The Jev guide is at /home/stevan/dev/jev/docs/jev-guide.md; the local vendor docs at /home/stevan/dev/jev/docs/vendor/cloudflare/ contain nothing about STT, so almost everything below comes from live Cloudflare/Deepgram/MDN pages and from the model definition JSON in the cloudflare-docs repo.
1. What the reference copilot actually does for speech (and why it is not enough)
- Replay/Video mode: it never transcribes audio at runtime. Each call is a JSON file of utterances
{t, t_end, speaker, text}; the UI fires an utterance when playback time passest_end("exactly as a live STT would finalize it"). Seestatic/index.htmllines ~470-500 anddata/calls/cloudtalk.json(48 utterances,"speakers labeled by hand", transcribed offline with faster-whispersmall.en).README.mdline 66-70 describes the hand-labelling. - Live mode: browser Web Speech API (
webkitSpeechRecognition,continuous=true,interimResults=true,lang='en-US'), with a manual rep/prospect toggle (skey) because there is no diarization. Comment instatic/index.html~line 447: "live mic (Web Speech API, Chrome only; no diarization -> manual speaker toggle)".
So the reference sidesteps STT entirely. For us, speaker attribution (rep vs client) is the single hard requirement: the speaker-masked Jev questions ("did the client just raise a rate objection?") are impossible without it. That rules out every option that returns plain text with timestamps and nothing else.
2. Requirements checklist
| # | Requirement | Why |
|---|---|---|
| R1 | Speaker diarization (rep vs client), or per-speaker channels | speaker-masked questions, "who said it" in the timeline |
| R2 | Utterance segmentation with start/end timestamps | replay pacing; feeding Jev one utterance at a time |
| R3 | Long-file handling (5-40 min, 5-60 MB) without hand-rolled chunking | PoC upload path and bulk ingestion of the historical corpus |
| R4 | UK English accuracy incl. FX vocabulary (GBP, EUR, SWIFT, IBAN, spot, forward, settlement) | domain terms drive the objection classifiers |
| R5 | Cost per hour of audio | corpus of "many" historical calls; ongoing live use |
| R6 | Live latency (< ~1 s to a final utterance) | mode B |
| R7 | Runs from a Worker with no extra vendor account, or at least via AI Gateway | "on Cloudflare" constraint |
3. Catalogue of options
3.1 @cf/openai/whisper-large-v3-turbo (Cloudflare-hosted)
Source: model JSON src/content/workers-ai-models/whisper-large-v3-turbo.json in github.com/cloudflare/cloudflare-docs (fetched raw 2026-09-24) and https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/
- Input:
audioisanyOf(a) base64 string, (b){body, contentType}object (schema). REST test today used (a) with JSON{audio: base64}. ASSUMPTION: form (b) is what the Workers binding uses to stream aReadableStreamwithout base64; untested. - Parameters:
task,language,vad_filter(default false),initial_prompt,prefix,beam_size(5),condition_on_previous_text(true; "Setting to false may help prevent hallucination loops"),no_speech_threshold(0.6),compression_ratio_threshold(2.4),log_prob_threshold(-1),hallucination_silence_threshold(seconds). All from the schema. - Output:
text,word_count,vtt,transcription_info{language, language_probability, duration, duration_after_vad},segments[]{start, end, text, temperature, avg_logprob, compression_ratio, no_speech_prob, words[]{word,start,end}}. No speaker field anywhere. - Verified by Stevan's test (2026-09-24,
scratchpad/video/whisper.out): 80.97 s clip -> 18 segments of ~4-7 s each, every segment carrying word-levelwords[]with start/end,vttpresent,language_probability0.9995, response in ~5.5 s (UNVERIFIED:whisper.outholds only the JSON body and no timing, so this figure comes from the parent session, not the file), billed 62.9 neurons (= 46.6 neurons/min, matching the price table exactly). - Price: $0.000513/min = $0.031/hour; 46.63 neurons/min (https://developers.cloudflare.com/workers-ai/platform/pricing/). At 10,000 free neurons/day that is ~214 free minutes/day.
- Batch:
async_queue: truein the model JSON -> usable with the Batch API (queueRequest: true, payload < 10 MB, https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/). - Size limit: undocumented. The Limits page only states "Automatic Speech Recognition: 720 requests per minute" (https://developers.cloudflare.com/workers-ai/platform/limits/). Cloudflare's own tutorial chunks the file into 1 MB byte slices before base64 (https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-workers-ai-whisper-with-chunking/,
const chunkSize = 1024 * 1024). Community reports on the older@cf/openai/whispersay <1 MB works and ~2-4 MB fails withInferenceUpstreamError(https://community.cloudflare.com/t/inferenceupstreamerror-for-large-audio-files-2-mb/624759 — page returned 403 to me; content taken from search snippet, UNVERIFIED). A docs issue confirms formats/sizes are simply not documented (https://github.com/cloudflare/cloudflare-docs/issues/17916). - Practical consequence for R3: a 40-minute call must be split into ~30-60 s pieces client- or Worker-side. Byte-slicing (the tutorial's approach) is only tolerable for MP3 (decoders resync on frame headers); it corrupts m4a/AAC (MP4 container). Real chunking needs decode -> PCM -> WAV chunks with overlap and timestamp offsetting. That is exactly the "pipeline work" the Kompozy review flags as "left to you" (https://kompozy.io/reviews/cloudflare-whisper).
- Accuracy/UK English:
languagetakes ISO codes; there is no en-GB variant (Whisper's English model is dialect-agnostic). Known failure mode is hallucination on silence/hold music and repetition loops (Deepgram's vendor-authored test of large-v3 reports median WER 42.9 on phone calls, https://deepgram.com/learn/whisper-v3-results — treat as marketing; the Hugging Face card gives no WER figure, only that turbo is faster "at the expense of a minor quality degradation", https://huggingface.co/openai/whisper-large-v3-turbo). Stevan's 81 s test on phone-quality YouTube audio came back clean. Mitigations exposed in the schema:vad_filter=true,condition_on_previous_text=false,hallucination_silence_threshold,initial_promptseeded with FX vocabulary. - Verdict: cheapest by 10x, clean word timestamps, but fails R1 and R3. Useful only as a text-only bulk path.
3.2 @cf/openai/whisper and @cf/openai/whisper-tiny-en
Source: model JSON for both. Input is raw binary (format: binary) or {audio: [uint8...]}; output text, word_count, words[]{word,start,end}, vtt. No segments object, no speakers, none of the anti-hallucination knobs. whisper costs $0.000453/min (41.14 neurons); whisper-tiny-en is beta with no price listed. Same size problem. Strictly dominated by v3-turbo for our purposes.
3.3 @cf/deepgram/nova-3 (Cloudflare-hosted partner model) — HTTP and WebSocket
Source: model JSON nova-3.json; https://developers.cloudflare.com/workers-ai/models/nova-3/ (raw markdown fetched); Deepgram docs as cited.
- Properties:
async_queue: true,partner: true,realtime: true, terms https://deepgram.com/terms. - Input (HTTP):
audio: {body, contentType}(both required in the schema). UNVERIFIED request shape: a search snippet suggested Cloudflare's usage example is REST withContent-Type: audio/mpegand--data-binary @file.mp3, options as query parameters (?detect_language=true), but the nova-3 model page as re-fetched on 2026-09-24 (HTML andindex.md) contains only parameter descriptions and pricing, no usage block, and the schema'saudio: {body, contentType}object is a different shape from a raw binary body. Whether options go in the query string or a JSON body, and whichContent-Typeis required, is unconfirmed (test plan step 0). If raw binary is accepted there is no base64, so a whole 40-min mp3 goes in one request subject to the Workers request-body limit (100 MB on Free/Pro, https://developers.cloudflare.com/workers/platform/limits/). Deepgram's own pre-recorded limit is 2 GB (https://developers.deepgram.com/docs/pre-recorded-audio). UNVERIFIED on the Cloudflare proxy — see test plan. - Parameters that matter to us (all in the schema):
diarize("Each word ... assigned a speaker number starting at 0"),utterances("Segments speech into meaningful semantic units"),utt_split,paragraphs,punctuate,smart_format,language(BCP-47; Deepgram listsen-GBfor nova-3, https://developers.deepgram.com/docs/models-languages-overview — note Deepgram normalises to American spelling even with en-GB, https://developers.deepgram.com/docs/language),keyterm(boost "GBP", "SWIFT", "IBAN", "spot rate", "forward contract", "CurrencyTransfer"),multichannel+channels,numerals,mode(general | medical | finance),detect_entities,sentiment,topics,custom_intent,mip_opt_out(opt out of Deepgram's model-improvement program; relevant for client-call data). Streaming-only:interim_results,endpointing,vad_events,utterance_end_ms. - Output: Cloudflare's published schema only lists
results.channels[].alternatives[]{transcript, confidence, words[]{word,start,end,confidence}},results.summary,results.sentiments. It does not listwords[].speakerorresults.utterances[]. Deepgram's real response includesspeakerper word andutterances[]{start,end,confidence,channel,transcript,words,speaker,id}whendiarize=true&utterances=true(https://developers.deepgram.com/docs/utterances, https://developers.deepgram.com/docs/diarization). ASSUMPTION: Cloudflare proxies the Deepgram JSON unmodified and the schema is just incomplete (it also omitsmetadata). This is the first thing to test. - Price on Cloudflare: HTTP $0.0052/min = $0.312/hour (472.73 neurons/min); WebSocket $0.0092/min = $0.552/hour (836.36 neurons/min). Diarization has no separate line item. Free tier covers ~21 HTTP minutes/day. Direct Deepgram list price (https://deepgram.com/pricing, fetched 2026-09-24): pre-recorded monolingual $0.0043/min ($0.258/h), multilingual $0.0052/min; streaming monolingual "Current price $0.0048/min, Regular price $0.0077/min" ($0.288/h promo, $0.462/h regular), multilingual current $0.0058/min, regular $0.0092/min; diarization "Included" for pre-recorded. Cloudflare's $0.0052 (HTTP) and $0.0092 (WebSocket) are exactly Deepgram's own multilingual pre-recorded and regular multilingual streaming prices, so "markup" only applies relative to the monolingual tier: ~20% batch, and ~19% streaming at regular price (~90% against the current promo), in exchange for no vendor account, neuron billing and the Workers binding.
- Diarization quality caveats (independent-ish): works "straightforward for a typical caller-and-agent call", degrades on overlap, similar voices, narrowband telephony; measure on your own audio (https://www.evalgent.com/blog/deepgram-stt-latency-diarization-stability). If CT's recordings are dual-channel (rep on L, client on R — common for dialers),
multichannel=truegives perfect attribution with no diarization at all (https://developers.deepgram.com/docs/multichannel-vs-diarization). Open question for Stevan: are the call-coach recordings mono or stereo? - Streaming mechanics (for mode B): WebSocket via AI Gateway
wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/workers-ai?model=@cf/deepgram/nova-3&encoding=linear16&sample_rate=16000&interim_results=true, headercf-aig-authorization, send raw binary PCM frames, receive JSON withchannel.alternatives[0].transcriptandis_final(https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/). Deepgram semantics: defaultendpointing=10ms; conversational recommendationendpointing=300,interim_results=true,utterance_end_ms=1000(https://developers.deepgram.com/docs/endpointing); connection closes after 10 s without audio unless{"type":"KeepAlive"}text frames every 3-5 s (https://developers.deepgram.com/docs/keep-alive); streaming diarization returnsspeakerbut notspeaker_confidence(https://developers.deepgram.com/docs/diarization). Cloudflare's own realtime example downmixes the SFU's 48 kHz stereo PCM to 16 kHz mono with a Speex WASM resampler before sending to Nova (https://github.com/cloudflare/realtime-examples/blob/main/ai-tts-stt/STTAdapter.md). - Verdict: the only Cloudflare-hosted model that meets R1-R4 and R6, at ~$0.31/h batch. Recommended for both PoC and live, pending the two verification tests below.
3.4 @cf/deepgram/flux (WebSocket only)
Source: flux.json; https://developers.deepgram.com/docs/flux/feature-overview. Input only linear16 PCM; parameters are eot_threshold, eager_eot_threshold, eot_timeout_ms, keyterm; events Update / StartOfTurn / EagerEndOfTurn / TurnResumed / EndOfTurn with transcript, words[]{word, confidence} (no per-word timestamps in the Workers AI schema), audio_window_start/end. No diarize parameter; Deepgram's Flux feature table lists no diarization or multichannel. $0.0077/min (700 neurons) = $0.462/h. Built for voice agents that must know when to speak; its turn-end events are attractive for "utterance finalised" but it cannot tell rep from client. Not suitable unless we already have per-speaker streams (then two Flux sockets would work, at 2x cost, with no advantage over nova-3).
3.5 Third-party ASR models in the unified catalogue (developers.cloudflare.com/ai/models/)
These are called with the same env.AI.run('<provider>/<model>', …) / POST /accounts/{id}/ai/run shape but are billed through AI Gateway credits or BYOK, not neurons. Evidence: today's typesafe/jev call returned HTTP 402 "Insufficient balance; add money to your gateway or use BYOK" (scratchpad jev-run-A.out). Pricing on every catalogue page is "View pricing in the Cloudflare dashboard"; I list provider list prices as a proxy.
| Model | Diarization | Timestamps | Input | Provider list price | Notes |
|---|---|---|---|---|---|
assemblyai/universal-3.5-pro (https://developers.cloudflare.com/ai/models/assemblyai/universal-3.5-pro/) |
speaker_labels, speakers_expected; returns utterances[] |
word-level ms | audio_url (public URL or data URI), audio_start_from/audio_end_at |
$0.21/h + $0.02/h diarization add-on (https://www.assemblyai.com/pricing) | Strong batch contender; no streaming via CF that I could find. universal-3-pro also listed, "Zero data retention". |
xai/grok-stt (https://developers.cloudflare.com/ai/models/xai/grok-stt/) |
diarize: true per word |
word-level | HTTPS URL or base64 data URI; 25 MB direct-upload limit, none for URL fetch | $0.10/h REST, $0.20/h streaming (https://docs.x.ai/docs/models) | Cheapest diarizing option; keyterm array (100 terms), multichannel 2-8 ch; 25 languages; ZDR. Streaming through Cloudflare is documented: a websocket boolean input "establishes a bidirectional WebSocket connection for real-time audio transcription", mutually exclusive with file/url (untested). Newer, less track record. |
openai/gpt-4o-transcribe (https://developers.cloudflare.com/ai/models/openai/gpt-4o-transcribe/) |
no | not exposed | data URI or HTTPS URL; formats flac/mp3/mp4/mpeg/mpga/m4a/ogg/wav/webm | $0.006/min = $0.36/h (https://developers.openai.com/api/docs/pricing) | OpenAI's gpt-4o-transcribe-diarize ($0.36/h) is not in the Cloudflare catalogue as of today. |
| ElevenLabs Scribe | v2: up to 32 speakers, word timestamps, 3 GB / 10 h files, realtime ~150 ms (https://elevenlabs.io/docs/capabilities/speech-to-text) | yes | — | $0.22/h batch, $0.39/h realtime (https://elevenlabs.io/pricing/api) | Not in the Cloudflare ASR catalogue; AI Gateway's ElevenLabs provider only documents text-to-speech (https://developers.cloudflare.com/ai-gateway/usage/providers/elevenlabs/). Would need direct API through the gateway's generic passthrough. |
| Deepgram direct via AI Gateway | as nova-3 | yes | replace https://api.deepgram.com/ with https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/deepgram/; WebSocket supported (https://developers.cloudflare.com/ai-gateway/usage/providers/deepgram/) |
$0.258/h batch; streaming $0.288/h at the current promo, $0.462/h regular | Same model as 3.3, own Deepgram account, ~17% cheaper on batch and regular-price streaming (~48% on streaming while the promo lasts), keeps logging/caching in the gateway. |
3.6 Browser Web Speech API (what the reference uses live)
Source: https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API and https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition/start
- Default engine is server-based (Chrome sends audio to Google's service); Chrome 139 (Aug 2025) added
processLocally = trueon-device mode gated by a one-time language-pack install (https://developer.chrome.com/blog/new-in-chrome-139; the language-pack requirement is on MDN's Using guide). - Chrome now also accepts
recognition.start(audioTrack)with any liveMediaStreamTrack, so a remote WebRTC track oraudioElement.captureStream()can be transcribed, not just the mic (MDN page above; Chrome intent-to-ship https://chromestatus.com/feature/5178378197139456). MDN marks this "Limited availability". - No diarization, no word timestamps, no utterance start/end (results only carry
transcript,confidence,isFinal);continuoussessions end on silence and must be restarted (the reference'srec.onendre-start loop). Contextual biasing exists viaSpeechRecognitionPhrase. - Cost $0, but audio from FX client calls leaves the browser for Google's servers with no DPA under our control, and results are not reproducible. Reject for production; acceptable only as a zero-cost demo toggle (which is all the reference uses it for).
3.7 Cloudflare Realtime family (live transport, not STT itself)
- Realtime SFU + WebSocket media adapter: forks a WebRTC track's audio to your WebSocket endpoint as 48 kHz, 16-bit, stereo interleaved PCM; created via
POST https://rtc.live.cloudflare.com/v1/apps/{appId}/adapters/websocket/newwithlocation: "remote",outputCodec: "pcm"; keep messages < 32 KB; $0.05/GB egress after 1,000 GB/month free (https://developers.cloudflare.com/realtime/sfu/features/media-transport-adapters/websocket-adapter/, https://developers.cloudflare.com/realtime/sfu/platform/pricing/). Reference implementation: SFU -> Durable Object -> nova-3 WebSocket (https://developers.cloudflare.com/realtime/sfu/examples/ai-audio/). Only relevant if the call itself runs over WebRTC into Cloudflare. - RealtimeKit (hosted meetings): built-in live transcription streams each participant's audio separately to nova-3 (speaker attribution for free), events via
meeting.ai.on("transcript", …), post-meeting transcripts with whisper-large-v3-turbo;en-GBfor live, post-meeting (Whisper) takesen; billed at the same neuron rates (836.36/min live, 46.63/min post) (https://developers.cloudflare.com/realtime/realtimekit/ai/transcription/). Only relevant if onboarding calls were hosted in a CT-built RealtimeKit room instead of phone/Zoom/Teams. @cloudflare/voice(Agents SDK, Beta):withVoiceInputgives STT-only pipelines;WorkersAIFluxSTTis the default model andWorkersAINova3STTis the one the docs recommend forwithVoiceInput, so Nova-3 must be selected explicitly. Server-side adapters: WebSocket, Twilio, Plivo; Telnyx only via a browser WebRTC bridge. No speaker attribution documented (https://developers.cloudflare.com/agents/communication-channels/voice/). Useful scaffolding if the live path is a Twilio-style media stream from the dialer.- The August 2025 "Realtime Agents" runtime announced free open beta (https://blog.cloudflare.com/cloudflare-realtime-voice-ai/); its docs URL (
/realtime/agents/) now redirects (301) to the Realtime overview, so treat@cloudflare/voiceas its successor. ASSUMPTION.
4. Comparison table
Costs are per hour of audio. "CF-native" = billed in neurons through the Workers AI binding with no extra vendor account.
| Option | Diarization (R1) | Utterances + timestamps (R2) | Long files (R3) | UK English / vocab (R4) | Cost/h (R5) | Live (R6) | CF-native (R7) |
|---|---|---|---|---|---|---|---|
@cf/openai/whisper-large-v3-turbo |
No | segments ~5 s + words | No (~1 MB/base64 chunks, undocumented) | language=en, initial_prompt; hallucination knobs |
$0.031 | No (batch only) | Yes |
@cf/openai/whisper / -tiny-en |
No | words only | No | none | $0.027 / n/a | No | Yes |
@cf/deepgram/nova-3 HTTP |
Yes (diarize) + multichannel |
utterances[] with speaker, words |
Yes (binary body, ≤100 MB Worker limit; 2 GB at Deepgram) | en-GB, keyterm, mode=finance, numerals |
$0.312 | — | Yes |
@cf/deepgram/nova-3 WebSocket |
Yes (speaker, no confidence) |
is_final, speech_final, utterance_end_ms |
streaming | same | $0.552 | Yes (latency unmeasured) | Yes (binding or AI Gateway) |
@cf/deepgram/flux WS |
No | turn events, no word times | streaming | keyterm |
$0.462 | Yes | Yes |
assemblyai/universal-3.5-pro |
Yes | utterances, ms words | URL/data URI | language_code |
~$0.23 (list) | no CF streaming | Gateway credits/BYOK |
xai/grok-stt |
Yes | words | ≤25 MB direct, URL unlimited | keyterm ×100 |
~$0.10 (list) | Yes (documented websocket: true param; untested) |
Gateway credits/BYOK |
openai/gpt-4o-transcribe |
No | no | URL/data URI | prompt |
$0.36 (list) | No | Gateway credits/BYOK |
| Deepgram direct via AI Gateway | Yes | yes | 2 GB | same as nova-3 | $0.258 / $0.288 promo ($0.462 regular) | Yes | BYOK |
| ElevenLabs Scribe v2 | Yes (32 spk) | yes | 3 GB | English | $0.22 / $0.39 | Yes | Not in catalogue |
| Browser Web Speech API | No | no timestamps | n/a | lang=en-GB, phrases |
$0 | Yes (Chrome) | n/a; audio goes to Google |
5. Recommendation
5.1 PoC (mode A: upload + timed replay)
Use @cf/deepgram/nova-3 over HTTP, one request per file, with:
diarize=true&utterances=true&punctuate=true&smart_format=true&language=en-GB&numerals=true&mode=finance&keyterm=<FX terms>&mip_opt_out=true
(mip_opt_out may change Deepgram's pricing per their note; check https://dpgr.am/deepgram-mip.)
Rationale: it is the only Cloudflare-hosted model that returns speaker-labelled utterances with timestamps, it accepts the whole file as a binary body so there is no chunking pipeline to build for the PoC, and $0.31/h means a 30-minute call costs ~15 cents (about 14,000 neurons, i.e. over the free daily 10k for the second call of the day). Map Deepgram utterances directly onto the reference's replay format: {t: u.start, t_end: u.end, speaker: map[u.speaker], text: u.transcript} and reuse its fireAt = t_end scheduler. Regroup consecutive same-speaker utterances into turns (Deepgram's utt_split defaults to ~0.8 s, so a rep monologue arrives as several utterances).
Speaker 0/1 -> rep/client mapping: no STT will tell you which is which. Cheap heuristics: the rep speaks first on outbound onboarding calls; the rep says "CurrencyTransfer"; or ask Jev one Choice question on the first two turns ("which speaker is the company representative?") and let the admin override. Mark as a design decision, not an STT problem.
Do these tests before writing any product code (each is one request against POST /accounts/{id}/ai/run/@cf/deepgram/nova-3 or the Workers binding):
0. Establish the request shape, which is UNVERIFIED (§3.3): try (a) a raw binary body with Content-Type: audio/mpeg via --data-binary, options as query parameters, and (b) JSON {audio: {body, contentType}, diarize: true, utterances: true} via the binding; record which works and use it for tests 1-3.
- Send
scratchpad/video/audio.mp3(81 s, two speakers) withdiarize=true&utterances=trueand confirm the JSON containsresults.utterances[]andwords[].speaker. If Cloudflare strips them (its published schema omits them), fall back toxai/grok-sttorassemblyai/universal-3.5-provia gateway credits. - Send a real 30-40 min CT recording (20-60 MB) to find the practical body-size ceiling on the Cloudflare proxy; if it fails, use the Batch API (
queueRequest: true, but that caps payloads at 10 MB) or chunk at silence boundaries. - Inspect whether CT recordings are stereo with one party per channel; if so run
multichannel=trueinstead ofdiarize=trueand the attribution problem disappears.
I could not run these myself: the earlier whisper test's CLOUDFLARE_API_TOKEN was in the parent session's environment, and ~/.config/jev/cloudflare.env (the location scripts/jev.mjs reads) does not exist on this box.
Keep @cf/openai/whisper-large-v3-turbo as a secondary, text-only path for bulk-indexing the historical corpus where you only need searchable text (10x cheaper), and for cross-checking nova-3's wording on FX terms. Set vad_filter=true, condition_on_previous_text=false, hallucination_silence_threshold=2, and initial_prompt with the FX glossary. Do not use it for anything that needs to know who spoke.
5.2 Live (mode B)
Use @cf/deepgram/nova-3 over WebSocket from a Durable Object (one DO per live call holding the socket, the KeepAlive timer and the running transcript), with encoding=linear16&sample_rate=16000&interim_results=true&endpointing=300&utterance_end_ms=1000&diarize=true&language=en-GB&keyterm=…. Fire a Jev evaluation on each is_final/speech_final result exactly where the reference fires on t_end. Budget $0.55/h on Cloudflare, or direct-Deepgram-through-AI-Gateway with BYOK at $0.29/h at the current promo ($0.46/h at regular price). If nova-3 streaming through the binding falls short, xai/grok-stt documents a websocket: true streaming mode (untested) and is the cheapest diarizing fallback.
Prefer separate audio streams per party over diarization whenever the transport allows it: rep mic on one socket (or channel), remote party on the other, so attribution is structural rather than statistical. That is what RealtimeKit does internally and what the WebSocket media adapter enables for WebRTC. Which transport you get depends entirely on where the calls live, which is the main open question.
Flux is not needed: its end-of-turn machinery solves the "when should the agent speak" problem we do not have, and it lacks diarization.
5.3 What to decide next
- Where do live calls happen (dialer / softphone / Zoom / Teams / mobile)? That picks the transport: dialer media stream (Twilio-style) ->
@cloudflare/voiceTwilio/Plivo adapter (selectWorkersAINova3STTexplicitly;WorkersAIFluxSTTis the default and has no diarization); browser softphone ->getUserMedia+getDisplayMedia({audio:true})or WebRTC remote track -> DO -> nova-3; CT-hosted room -> RealtimeKit with built-in per-participant transcription. - Recording format on the call-coach server (mono vs dual-channel, mp3 vs m4a, typical length) — decides
multichannelvsdiarizeand whether whisper chunking is ever worth building. - Data handling: Deepgram's model-improvement program (
mip_opt_out), theZero data retentionflag on third-party catalogue models, and whether client call audio may transit Google (Web Speech) at all. - Budget sanity: 1,000 historical hours through nova-3 HTTP ≈ $312 on Cloudflare vs ≈ $258 direct; through whisper-turbo ≈ $31 but without speakers.
6. Sources
Cloudflare
- Model definitions (raw JSON): https://raw.githubusercontent.com/cloudflare/cloudflare-docs/production/src/content/workers-ai-models/{whisper-large-v3-turbo,whisper,whisper-tiny-en,nova-3,flux}.json
- https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/ , /whisper/ , /nova-3/ , /flux/
- https://developers.cloudflare.com/workers-ai/platform/pricing/ (neuron table), https://developers.cloudflare.com/workers-ai/platform/limits/ (720 rpm ASR)
- https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-workers-ai-whisper-with-chunking/
- https://developers.cloudflare.com/workers-ai/features/batch-api/ and /workers-binding/
- https://developers.cloudflare.com/ai-gateway/usage/websockets-api/realtime-api/ ; /ai-gateway/usage/providers/deepgram/ ; /ai-gateway/usage/providers/elevenlabs/
- https://developers.cloudflare.com/ai/models/ (unified catalogue, saved copy in scratchpad), /ai/models/assemblyai/universal-3.5-pro/ , /ai/models/assemblyai/universal-3-pro/ , /ai/models/xai/grok-stt/ , /ai/models/openai/gpt-4o-transcribe/
- https://developers.cloudflare.com/realtime/sfu/features/media-transport-adapters/websocket-adapter/ , /realtime/sfu/examples/ai-audio/ , /realtime/sfu/platform/pricing/ , /realtime/realtimekit/ai/transcription/
- https://developers.cloudflare.com/agents/communication-channels/voice/ ; https://blog.cloudflare.com/cloudflare-realtime-voice-ai/ ; https://developers.cloudflare.com/changelog/post/2025-08-27-partner-models/
- https://developers.cloudflare.com/workers/platform/limits/ (request body 100 MB)
- https://github.com/cloudflare/realtime-examples/blob/main/ai-tts-stt/STTAdapter.md
- Local:
scratchpad/video/whisper.out(Stevan's 2026-09-24 test),scratchpad/jev-run-A.out(402 gateway-balance error),/home/stevan/dev/jev/scripts/jev.mjs
Deepgram
- https://deepgram.com/pricing ; https://developers.deepgram.com/docs/diarization ; /docs/utterances ; /docs/multichannel-vs-diarization ; /docs/endpointing ; /docs/keep-alive ; /docs/pre-recorded-audio ; /docs/models-languages-overview ; /docs/language ; /docs/flux/feature-overview
- Vendor-authored comparison (bias noted): https://deepgram.com/learn/whisper-v3-results
- Independent-ish: https://www.evalgent.com/blog/deepgram-stt-latency-diarization-stability
Others
- https://www.assemblyai.com/pricing ; https://elevenlabs.io/docs/capabilities/speech-to-text ; https://elevenlabs.io/pricing/api ; https://developers.openai.com/api/docs/pricing ; https://docs.x.ai/docs/models
- https://huggingface.co/openai/whisper-large-v3-turbo
- https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API/Using_the_Web_Speech_API ; https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition/start ; https://developer.chrome.com/blog/new-in-chrome-139 ; https://chromestatus.com/feature/5178378197139456
- Community (unverified, 403/429 on fetch): https://community.cloudflare.com/t/inferenceupstreamerror-for-large-audio-files-2-mb/624759 ; https://www.answeroverflow.com/m/1358937301530574978 ; https://github.com/cloudflare/cloudflare-docs/issues/17916 ; https://kompozy.io/reviews/cloudflare-whisper
Live verification (2026-09-24, added after the research pass)
Run by Claude with the CT account token (~/.config/jev/cloudflare.env):
- Aircall recordings are fetchable with
GET https://api.aircall.io/v1/calls/{id}(Basic auth with theAIRCALL_API_ID/AIRCALL_API_KEYfrom/etc/aircall-secrets.envon the call-coach server; the~/call-coach/.envcopy is stale and returns 403) →recordingsigned URL. Format: mono MP3, 22.05 kHz, 32 kb/s; 9:46 call = 2.3 MB (data/samples/call-3339895706.mp3). No dual-channel audio, somultichannel=trueis not an option; Aircall's own transcript carriesparticipant_type internal|externalper utterance and is the better speaker source for historical calls. @cf/deepgram/nova-3via REST (POST /accounts/{id}/ai/run/@cf/deepgram/nova-3?diarize=true&utterances=true&punctuate=true&smart_format=true&language=en-GB&numerals=true,Content-Type: audio/mpeg, binary body) on that call: HTTP 200 in 8.6 s, 111 utterances, 2 speakers, 4,616 neurons (≈ $0.05 for 9:46 → ≈ $0.31/h as documented). Speaker fields survive the Cloudflare proxy. Agreement with Aircall's labels by utterance midpoint: 103 of 110 (94%); the misses are merged cross-speaker utterances at fast turn-taking (e.g. "Hello, Karen speaking. Hi Karen, it's Tom..." as one speaker). On the 81 s YouTube clip (scratchpad/video/audio.mp3) it returned a single speaker.- Conclusion for M1: replay historical Aircall calls from their own transcripts (no STT); use nova-3 only for fresh uploads, and map
speaker 0/1→ rep/client in code (rep says "CurrencyTransfer" / speaks second after the greeting) with an admin override.