Implementation deep dive

Transcribing Voice Messages and Extracting Contact Details Safely

Choosing a speech model, pinning the language, and the verification rule that stops an AI inventing an email address that was never spoken.

Module: Speech and extraction pipeline
Started: 2026
Last updated: July 19, 2026

Stack

Groq
Whisper large v3
gpt-oss-120b
Workers AI
Cloudflare Workers

This is the part of the voice-note contact widget that turns a few seconds of audio into text a person can edit, and into contact fields that are safe to trust.

Most of the interesting decisions here are about restraint: which model not to use, which value not to accept, and what to do when the AI is confidently wrong.

What looked simple

What it looked like

Send the audio to a speech API.
Put the text in the message box.
Ask a model for the email address.

What it actually took 12 steps

Send the audio to a speech API.
Choose between an accurate model and a cheap one — on the right criteria.
Pin the language, because detection guesses wrong.
Fall back to a second provider on another platform when the first fails.
Return empty rather than failing when both are down.
Ask a model for the contact details.
Constrain it to a strict output schema.
Fall back to loose JSON mode if that schema is rejected.
Normalize spoken forms: "at" becomes @, "dot" becomes a period.
Verify every extracted value literally appears in the transcript.
Discard and log anything that does not.
Cap the length of whatever survives.

Two of these steps are the feature. The rest are the difference between a demo and something you would let fill in a form field unattended.

Choosing the speech model

Transcription runs on Groq, using Whisper large v3 — the full model, not whisper-large-v3-turbo. The cheap variant was rejected on purpose.

Turbo is roughly two to three times cheaper and quicker, and noticeably weaker in three specific areas: accents, proper nouns, and addresses spelled out loud. That list happens to describe the entire content of these recordings — business owners saying their own name, their company name, and their website one letter at a time.

The visitor waits once, for a few seconds, and the accuracy of those particular words determines whether the note is useful. So the full model is used, and the reasoning is written in a comment directly above the setting, so whoever reads it next knows it was a decision rather than a default — and knows exactly which line to change if latency ever matters more than accuracy.

The language is pinned, not detected

Automatic language detection sounds strictly better. In practice, given the first second of audio — a breath, a room, someone starting a sentence — it guesses, and a wrong guess produces a transcript in an entirely unrelated language. Not a degraded transcript: a useless one.

The language is therefore set explicitly, and sampling randomness is turned off. Transcription here is not a creative task, and there is no version of it that benefits from variety.

Two providers, one interface

Groq goes first, because it is fast. If that call throws — an outage, a rate limit, an expired key — the same audio goes to Workers AI, running @cf/openai/whisper-large-v3-turbo through a binding in the Cloudflare runtime itself. Different vendor, different infrastructure, no shared failure mode. Here the turbo model is the right call: a degraded transcript beats no transcript.

If both fail, the function returns an empty string rather than raising. The recording is already stored, the message will still send, and the playback link will still work. The transcript is an enhancement, not a dependency.

Transcription with failover

Groq — Whisper large v3

Language pinned to English, temperature zero, file extension matched to what the browser actually recorded.
On failure, Workers AI — whisper-large-v3-turbo

A runtime binding rather than an HTTP call, on a different platform, so one vendor outage does not take both paths down.
On total failure, an empty transcript

Logged, never thrown. The message and the audio link are unaffected.

Extracting the email and website

If someone says their address out loud, the form should already contain it by the time they look at the screen. This is where the genuinely dangerous part lives.

Speech does not spell

Dictated addresses arrive as words. “joe at oldplumbing dot com” has to become [email protected], which means normalizing the spoken connectors and removing the spaces speech-to-text puts between them.

The rule that matters

A language model asked for an email address, given a transcript that mentions a website but no address, will supply one. It will be well-formed, plausible, and built from context — info@ or contact@ at the domain that was mentioned.

This is the worst possible failure mode for this feature, because it is invisible. A fabricated address looks exactly like a real one sitting in a form field. The visitor glances at the form, sees their own website spelled correctly beside it, and has every reason to assume the whole thing worked.

So the instruction not to do this is not treated as sufficient. Every value the model returns is checked: normalized the same way the transcript is normalized, then required to literally occur inside it. Anything that does not is discarded and the drop is logged.

The check is a few lines of ordinary code, it runs on every result, and no phrasing in the prompt can route around it.

Extraction with verification

The transcript goes to gpt-oss-120b on Groq

Strict JSON schema, temperature zero, reasoning effort low — this is copying, not deliberation, and the visitor is waiting on it.
If the strict schema is rejected, retry once in JSON mode

A parameter a model does not support should degrade the contract, not silently disable the feature.
Normalize both the values and the transcript

Spoken connectors collapse, spaces and punctuation are stripped, case is flattened.
Require every value to appear in the transcript

A value that is not there was not said. It is dropped and logged.
Cap what survives

Length-limited before it reaches a form field.

Constrain the output, but survive the constraint being unavailable

Extraction requests a strict json_schema — the strongest contract Groq offers, since the response shape is enforced rather than merely asked for. This was part of why openai/gpt-oss-120b replaced the model before it.

Not every model on Groq accepts that parameter. If the request comes back as a 4xx, it is retried once in plain json_object mode, which every model there supports. Without that fallback, swapping in a model without strict-schema support would have quietly turned extraction off — no error anywhere, no visible symptom except that the fields stopped filling in.

What is live today

The whole transcription and extraction pipeline runs in production. Extraction for languages beyond English is the one item not built.

Component status

7 live · 1 planned

Component	Status	Note
Speech transcription	Live	Groq, Whisper large v3, English pinned, temperature zero
Provider failover	Live	Workers AI binding, whisper-large-v3-turbo
Empty-transcript degradation	Live	Never blocks the message or the audio link
Contact extraction	Live	Groq, gpt-oss-120b, strict JSON schema
Schema-rejection fallback	Live	Retries once in json_object mode on a 4xx
Transcript verification guard	Live	Unverifiable values discarded and logged
Spoken-form normalization	Live	—
Extraction for languages beyond English	Planned — not built	Not built

What we got wrong first

Challenge and lesson

The small model was not wrong in an obvious way — it was wrong in a plausible way

Initial assumption: Extraction is a mechanical task, so llama-3.1-8b-instant is sufficient — it is fast and cheap, and the prompt can simply tell it not to guess.
Hidden complexity: It guessed anyway, and its guesses were good. A correctly formatted address at a domain the speaker really had named is not detectable by inspection — not by the developer reading test output, and certainly not by a visitor glancing at a prefilled form. The failure would have arrived as replies bouncing weeks later, with no way to reconstruct which addresses had been invented.
Architectural response: openai/gpt-oss-120b replaced it, which reduced the frequency and brought strict schema support with it. The frequency was never the real fix. The real fix was accepting that this class of error cannot be prompted away, and adding a mechanical check against the source text that runs on every single result.

Lesson

Grade an AI failure by how detectable it is, not how often it happens. A rare, invisible, plausible error is more dangerous than a frequent obvious one. Where the output can be checked against its input, write the check — and treat the prompt as a preference, never as a guarantee.

Reusing this elsewhere

Here	Elsewhere
Spoken email and website	Any field a person says faster than they type
Verify against the transcript	Verify any extraction against its source document
Groq with a Workers AI fallback	Any AI dependency worth not being single-sourced
Empty transcript on total failure	Any enhancement that must not block the core action

Ways to scope this

The transcription step alone. Audio in, text out, with failover and honest degradation. Useful on its own for intake, support, or field notes.

Transcription with verified extraction. The above plus pulling structured fields out of speech, with the source-verification guard so nothing invented reaches a record.

A full intake path. The above wired into wherever the record belongs — CRM, help desk, shared inbox — with the original audio retained for as long as your record-keeping requires.

Common questions

Because the transcript is not a byproduct — it is the draft the visitor edits before sending. Speech-to-text gets names and company names wrong, and the only person who can fix that is the person who just spoke. Doing it in the background would save a few seconds and lose the correction step entirely.

Not as the primary. whisper-large-v3-turbo is roughly two to three times cheaper and faster than the full whisper-large-v3, but measurably weaker on accents, proper nouns, and addresses spelled out loud — which is most of what these recordings contain. A visitor waits once, so accuracy was chosen over latency. Turbo is still used as the failover, where a degraded transcript beats none.

By not relying on the instruction. Any extracted value is discarded unless it literally appears in the transcript, after normalizing spoken forms like "at" and "dot". The check runs in code on every result, so no phrasing in the prompt can bypass it. A dropped value is logged.

Workers AI is tried automatically — the same speech model family, but running as a binding inside the Cloudflare runtime rather than as an outbound HTTP call, so it fails independently. If both fail, the transcript comes back empty, and the recording, the message, and the playback link all still work. No AI step in this pipeline is allowed to cost a lead.

This deployment pins English deliberately, because automatic detection misfires on the first second of audio. Supporting more languages is a matter of choosing the language explicitly per visitor rather than leaving it to detection.

Who built this

Alex Taranov mapped the workflow, designed the data model and architecture, and oversaw implementation. AI-assisted tooling supported the coding; the product decisions, review, testing, and final responsibility stayed with Taranov Digital.

About Alex

Need a workflow like this in your business?

You do not have to buy a platform. This work can start with one focused module connected to the CRM, order system, or scheduling tool you already use. Tell me the workflow that is currently living in someone's messages.

Discuss a workflow