What is automatic speech recognition?
Automatic speech recognition, commonly abbreviated to ASR, converts spoken audio into text. On a telephone call it operates continuously on a compressed narrowband stream while the caller is still speaking. Everything downstream, from intent detection to the final transcript an auditor reads, inherits whatever the recogniser got wrong.
Also called: ASR, Speech to text, Speech recognition
Why every later error in a voice system begins here
An error made by the recogniser cannot be repaired by anything downstream of it. The language model receives text, has no access to the audio, and no way to distinguish a caller who said something unusual from a recogniser that misheard something ordinary. Telephone audio arrives narrowband through a codec designed for intelligibility rather than fidelity, which is a harder problem than the studio audio most accuracy figures are quoted on. The recogniser is therefore the component to specify first and the one to monitor permanently.
- Recognition errors propagate silently, since downstream components receive only text.
- Telephone audio is narrowband and compressed, which is harder than studio recording.
- Accuracy on numbers and names matters far more than accuracy on ordinary words.
- The transcript the recogniser produces becomes the evidence of what was said.
How automatic speech recognition works
- Audio frames arrive from the telephony layer and are normalised for level.
- An acoustic model converts those frames into a distribution over sound units.
- A language model weights candidate word sequences against expected phrasing.
- Partial transcripts are emitted while the caller speaks, so later stages can begin.
- Domain vocabulary is supplied so that account references and product names resolve correctly.
- The final transcript is retained with word timings for the call record.
How automatic speech recognition is evaluated
Recognition is judged on the words that carry the decision rather than on an average across all speech. Measure the following on audio captured from the live line:
- word error rate on telephone audio rather than on studio recordings
- entity error rate on account references and proper names
- error rate on the first utterance, before any context is available
- latency from end of speech to final transcript
- accuracy under packet loss and background noise
Limitations, risks and common failure modes
- Accuracy quoted on clean audio has little bearing on a mobile call in traffic.
- A recogniser tuned for one language degrades when callers switch mid sentence.
- Numbers spoken in groups are transcribed as separate digits or merged incorrectly.
- Vendor model updates change accuracy without any notice to the deployment.