Speech recognition in 2026: what actually works on your own computer
If you want to know how good a speech recognizer is, a percentage on a product page will not help you. It depends on what you record. A dictated paragraph at a desk and a meeting with four people in a reverberant room are two entirely different tasks, and a model can be excellent at one and unusable at the other.
So we measured instead of guessing. 63 audio files, a little over eight hours of material, each file with an attested reference transcript, all run on three machines. That makes 168 evaluated transcripts. This article shows the results, including the uncomfortable ones.
A reliable transcript still needs to be checked against the audio, no matter who wrote it. That is why memozero includes an editor for going through the recording word by word.
Where speech recognition stands in 2026
Since Whisper appeared in 2022, what counts as usable speech recognition has shifted twice. First the recognition became good enough to be useful. Then it became small enough to run on an ordinary laptop. That is what memozero does: the recording does not leave your computer, and the computation happens locally.
That changes the interesting question. No longer “does this work at all”, but “what do I give up by skipping a data center”? Public leaderboards help little here. They almost always measure on read-aloud texts from audiobook collections: one voice, a clean microphone, fully formed sentences. Everyday audio sounds different: the tape recorder sits on the table, someone takes a sip, a car honks, two people talk at once and use terms the model has never seen.
So we built our own test set, around the failure cases forensic users actually report.
The test set: what was measured
The 63 files are not a random sample. Each one stands for a test case, and each has a reference transcript, the attested wording of what was said.
| Group | Material | Length per file | Source | Kind |
|---|---|---|---|---|
| Recording conditions | 8 variants of the same clean recording | 2:00 min, 284 words each | LibriVox audiobook, 16 kHz mono, noise from DEMAND | real voice, artificially degraded |
| File formats | 5 containers of the same recording | 2:00 min | WAV 48 kHz stereo, MP3 VBR, M4A/AAC 96k, OGG Vorbis, Opus 32k | real voice, artificially converted |
| Spontaneous speech | 10 freely spoken answers to a question | 4 to 12 s | Common Voice Spontaneous Speech, 23 speakers | real recording |
| Deep dialect | 10 Alsatian clips | 5 to 19 s | Common Voice Alsatian | real recording |
| Emotion and whispering | anger, sadness, joy, disgust, fear, whispering | 2 to 4 s | EmoDB Berlin, Thorsten-Voice | real voices, acted in a studio |
| Multiple speakers | 2 Bundestag debates, 2 meetings | 3:00 to 29:47 min | Bundestag transcripts, AMI Corpus | real recording |
| Technical terms, numbers | 4 scripts with difficult content | 28 to 46 s | scripts with StPO, BKA, DNA, MRI, dates | AI voice (Cartesia) |
| Technical edge cases | short, very long, damaged, silent | 3 s to 4:16 h | truncated MP3, wrong extension, 0-byte file, silence | real voice, artificially damaged |
| Interpreting | simultaneous and consecutive, multilingual | 8:37 to 46:03 min | public press conferences | real recording |
What is real and what is not
Because this affects the results, here is the split of the 109 measurements:
- 66 measurements: real, unaltered recordings. Spontaneous speech, dialect, emotion, Bundestag, meetings. These are people in real situations, recorded with whatever they had to hand.
- 33 measurements: real voice, artificially altered. Recording conditions, file formats, technical edge cases. The speaker is real; we computed the degradation.
- 10 measurements: AI voice. The four scripts on technical terms, numbers, spelling, and language switching. The point was the content, not the voice: we needed sentences that put StPO, BKA, DNA, MRI and a dozen dates in a fixed, checkable order. No person reads that twice in the same way.
On the ten AI-voice measurements the pronunciation is cleaner than a human’s. The error rates in those two groups are therefore more likely too good than too bad. What they show is still solid, because the errors there are not about hearing but about how abbreviations and numbers are written. More on that below.
The emotion group has a different caveat: EmoDB consists of ten actors speaking ten fixed sentences in different affects, recorded in a studio. Real emotion sounds different from acted emotion, and a studio sounds different from a living room.
The most important design choice sits in the first two rows of the table. All 13 files there come from a single source file, a clean German audiobook excerpt with one speaker. We degraded them on purpose with ffmpeg: mixed in noise, lowered the level, cut the band to telephone quality, packed them into other formats. The spoken content is therefore identical in all 13 cases, and every difference in the result is due to audio quality alone.
How we measure
The metric is the word error rate, WER. It counts how many words were wrong, omitted, or invented, divided by the number of words in the reference. A WER of 0.05 means 5 in 100 words differ. Zero would be word-for-word identical.
We also compute the character error rate (CER) the same way, at letter level. Comparing the two tells you what kind of error you have. If WER is high and CER is low, the content is right and only the spelling differs. If both are high, real information is missing.
What these numbers do not capture: punctuation, who said what, and whether an error changes the meaning. A second review stage covers that; more at the end.
Each file ran exactly once. We tested three models, all based on OpenAI Whisper large-v3-turbo. That base has given the best recognition in our app testing from the start, which is why we measure against it, and not against an older or larger Whisper variant.
| Model | Choice in the app | Basis | Used for |
|---|---|---|---|
mlx-large-v3-turbo-german | Turbo (German), from 16 GB RAM | German fine-tune by primeline, MLX build at full precision (f16) | all German files |
mlx-large-v3-turbo-german-q4 | Turbo (German), under 16 GB RAM | the same fine-tune, quantized to 4 bit | all German files on machines with little RAM |
mlx-large-v3-turbo | Turbo (multilingual) | Whisper large-v3-turbo without fine-tuning | the 7 English and mixed-language files |
large-v3-turbo-german | Turbo (German) on Windows | the same German fine-tune, built for faster-whisper | all German files on the Windows machine |
Which of the two German MLX models runs is decided by the app from available RAM. Below we call them q4 and f16 for short.
We measured on three machines:
| Machine | Hardware | Backend | Model |
|---|---|---|---|
| MacBook Air M1 (2020), 8 GB | Apple M1, 8 GB unified memory, macOS 26.5 | MLX (GPU) | mlx-large-v3-turbo-german-q4 |
| MacBook Air M1 (2020), 16 GB | Apple M1, 16 GB unified memory, macOS 26.5.2 | MLX (GPU) | mlx-large-v3-turbo-german |
| Windows notebook (MSI) | Intel Core i7-11800H, 8 cores with 16 threads, 32 GB DDR4, NVIDIA GeForce RTX 3080 Laptop with 8 GB VRAM, Windows 11 Pro | faster-whisper on CUDA | large-v3-turbo-german |
The two Macs are the same machine: the same 2020 MacBook Air, the same M1 chip, the same macOS version. They differ only in RAM, 8 versus 16 GB. That is why the app loads two different models there, and we get a comparison of compressed versus full model without other hardware getting in the way.
All three runs are finished. 168 transcripts with a reference text are in the evaluation. The remaining exports are the interpreting files, which by design have no reference text.
On Windows the same fine-tune runs in a different implementation. What that means for the comparison is in the Mac versus Windows section.
Result: the average says little, the breakdown says a lot
Across all 168 measurements the word error rate is 0.26. On its own that number is worthless, because it throws very different tasks into one pot.
168 transcripts from three machines: two MacBook Air M1 with 8 and 16 GB (memozero 0.0.26 and 0.0.27) and a Windows notebook with CUDA (1.0.0-dev.93). As of 10 August 2026.
The gap between top and bottom is the actual finding. Between the easiest and the hardest task the factor is 29.
The surprise is at the top. Spontaneous High German sits at 0.026. Short everyday sentences, spoken freely, recorded with whatever people had to hand. About two thirds of these recordings came back word-for-word. Free speech is not a problem for the model; on the contrary, it is the easiest case in the whole set.
At the bottom sits deep dialect at 0.82. That number needs context; there is a section on it further down.
The same recording, degraded thirteen times
This is where it gets interesting, because every bar shows the same spoken text: 2 minutes, 284 words, one speaker. Only the recording quality changes each time.
- Original, no degradation
Mean across all three machines. All variants come from the same source file (clean German audiobook, 16 kHz mono), produced with ffmpeg. Noise from the DEMAND dataset. As of 10 August 2026.
Twelve of thirteen conditions sit between 0.067 and 0.090. That is practically the scatter you also see for the same test case on two different machines. In plain terms:
- File format barely matters. Opus at 32 kbit/s is 0.090 against 0.067 for uncompressed WAV. The difference is about six words in two minutes.
- A level that is too low is not a problem. Minus 30 dB, a recording you can barely hear, yields 0.082. Gain is applied internally.
- Clipping is not a problem either. Plus 20 dB with hard clipping: 0.067, exactly the unchanged original. We did not expect that.
- Telephone quality costs little. Even with highs and lows cut off, it stays at 0.086.
- It only breaks when the noise is as loud as the voice. Café noise at voice level quadruples the error rate to 0.249. The same noise 10 dB quieter costs almost nothing.
The practical rule: worry less about format, bitrate or gain, and more about the voice being louder than the room. That mostly means a quiet room and a microphone pointed at the person speaking. If more than one person is at the table, one microphone per person helps.
Counted: how much do the formats really differ?
An error rate is an average. Single outliers disappear in it. So we also lined up the five format transcripts against each other and counted how many words differ from uncompressed WAV.
- MacBook Air M1, 8 GB
- MacBook Air M1, 16 GB
- Windows with CUDA
The same two-minute recording in five containers, about 285 words. Counted are words that differ from the WAV transcript on the same machine. As of 10 August 2026.
At about 285 words per recording that is 0.4 to 3 percent. On the 8 GB Mac, M4A, OGG and Opus even produced a word-for-word transcript, even though Opus squeezes the file down to 32 kbit/s.
One exception is in the chart: Opus on the Windows machine with 24 differing words. That is why Opus sticks out slightly as the only format in the chart above. So far it is a single observation on one machine, and we do not know whether Windows decoding is the cause. We are watching it.
Speaker recognition: it is not the count that matters, it is talking over each other
We measure speaker diarization separately, because word error rate only says whether the words are right, not who they were assigned to. Five files in the set have more than one speaker.
Bundestag debates (official transcript as reference) and AMI Corpus. Mean across all three machines. As of 10 August 2026.
The result goes against intuition. The file with the fewest speakers has the highest error rate; the 30-minute conference with four people scores better than the five-minute excerpt from it.
What actually counts is conversational discipline. In the Bundestag one person speaks, the others shout briefly. Result: 0.21 and 0.233, no worse than a noisy single-speaker recording. In the AMI meetings people talk over each other, interrupt, break off sentences. There we land at 0.41 to 0.48.
On speaker count itself we cannot yet make a solid claim. For the four files with a fixed count the app returns exactly that number, as expected. With automatic detection it found three speakers in both Bundestag files, which fits the situation. On the multilingual interpreting recordings, public press conferences with few people on the podium, it consistently reports eight, which is far too many. Four data points are not enough for a conclusion. Next we will put files with a known speaker count into the set and compare what automatic detection finds with who actually spoke.
Why a high error rate does not always mean much
Two test cases sit almost on top of each other numerically and are still completely different. Comparing word and character error rate makes that visible.
Both come from the AI-voice group, so what was heard was technically clean. The technical-terms test contains abbreviations such as StPO, BKA, DNA, MRI and StGB. Result: WER 0.39, but CER 0.086. At letter level almost everything is right. The model hears the abbreviation correctly and only writes it differently, with dots or spaced out. For meaning that is harmless; for word error rate every deviation still counts as an error.
The numbers-and-dates test sits at WER 0.41 and CER 0.56. Here the character error rate is higher than the word error rate: “fourteenth of March” becomes “14 March”. Right in substance, completely different in spelling.
Neither case is a comprehension error; they are formatting questions. They also show the limit of the metric: a word error rate of 0.4 looks catastrophic, and it is not. That is why we also read every striking case by hand.
RAM: the same app, two different models
As described above, memozero picks the language model from available RAM. Under 16 GB it runs mlx-large-v3-turbo-german-q4, the 4-bit quantized build; above that mlx-large-v3-turbo-german at full precision. Both come from the same German fine-tune; they differ only in how precisely the weights are stored. This is not a marketing tier. It is a technical necessity: the full build needs more memory than smaller machines have.
That split arose by accident in the test. Two testers, two identical 2020 MacBook Air M1 machines, only different amounts of RAM; that gave us a clean head-to-head over 32 identical files. Because chip and OS are the same, the only remaining difference is really the model.
- q4, under 16 GB RAM
- f16, from 16 GB RAM
Only files that ran on both machines with identical audio. As of 10 August 2026.
Across all 32 pairs the compressed model sits at 0.157, the full model at 0.144. On 19 files the small model was worse, on 6 better, on 7 exactly equal. The difference is measurable, but small: about one extra percentage point of errors.
More interesting is where it shows up. On file formats and recording conditions both models are practically identical. On emotion (0.098 versus 0.062) and on the technical edge cases (0.139 versus 0.090) the larger model pulls away. The small one therefore loses exactly where the audio is already difficult. On a clean recording there should be no difference between the small and the large model.
The app shows which model ran on every transcription. If you do not know that, you cannot place your results.
Mac versus Windows: the stack matters more than compute
33 files ran on all three machines with identical audio. That is the only truly fair comparison the dataset allows.
Only files that ran on all three machines with identical audio. As of 10 August 2026.
The three sit close together, but the order is a surprise. The Windows notebook with an RTX 3080 and 32 GB of RAM lands behind a 2020 MacBook Air M1 with 8 GB. On 17 of the 33 files it was worse than both Macs, on only 7 better.
The overall figure hides that the three behave differently by task:
- MacBook Air M1, 8 GB (MLX, 4 bit)
- MacBook Air M1, 16 GB (MLX, f16)
- Windows with CUDA (faster-whisper)
Only files that ran on all three machines with identical audio. As of 10 August 2026.
The Windows machine loses where the audio is difficult: noise and reverb (0.120 versus 0.088) and multiple speakers (0.331 versus 0.292). On cleanly spoken material the picture flips; there Windows leads (technical terms 0.243, numbers 0.238).
This is not a hardware result. It is a software result. On Windows a different implementation of the same German fine-tune runs, faster-whisper instead of MLX. More compute does not make up for that.
What the GPU does bring is speed. The 4-hour recording finished on the Windows notebook in 20 minutes; on the 8 GB Mac it took 53 minutes, two and a half times as long. On the two-minute files the averages are 42 seconds for Windows against 54 and 70 seconds for the two Macs.
For users that means: choosing Mac or Windows changes accuracy little, and speed a lot. Anyone who processes many long recordings benefits from a graphics card. Anyone who watches every tenth of a point on the error rate should not decide on that basis. The ratio can shift again with the next model and the next chip. The figures here hold for the versions we tested.
What remains when everything is estimated at once
The charts above look at one dimension at a time. A linear regression (OLS) with word error rate as the outcome estimates the task types and the three software stacks against each other. The reference is spontaneous High German on the full Mac model. The bars show how many extra error points remain when the other factors are held constant. A coefficient is treated as significant if its 95% confidence interval does not contain zero; that corresponds to a threshold of α = 0.05. There is no correction for multiple testing.
Interpreting is missing from this model because there is no reference text. Interruptions and the number of technical terms are not recorded as their own columns; they sit inside the task group. The groups are mutually exclusive: a file is spontaneous speech or dialect, never both, so there is no interaction between those two factors.
- Spontaneous speech (reference)
OLS on 168 transcripts with a word error rate. Reference: spontaneous speech on mlx-large-v3-turbo-german (f16). R² = 0.89. Significance threshold α = 0.05, no correction for multiple testing. Horizontal marks are 95% confidence intervals; for emotion and file format the mark starts at zero because the interval extends below it. As of 12 August 2026.
Dialect remains by far the strongest factor: plus 0.79, and the confidence interval sits well above zero. Multiple speakers, technical terms and numbers stay clearly visible. Emotion and file format no longer differ reliably from spontaneous speech in this estimate; their intervals contain zero.
- Mac, full model (reference)
The same OLS regression. Reference: mlx-large-v3-turbo-german (f16). The confidence intervals for q4 and Windows contain zero. α = 0.05. As of 12 August 2026.
Once the task is accounted for, almost nothing remains of the machine difference. The compressed model sits at plus 0.001, Windows at plus 0.021; both intervals contain zero. That matches the direct comparison of the 33 shared files: the stack changes accuracy little once the same kind of recording is compared.
What still does not work
Deep dialect. The Alsatian clips sit at a mean word error rate of 0.82 across all three machines. The model falls back to the closest High German word, and on this kind of material that goes wrong.
That number needs two caveats, or it misleads.
First, the material is extreme. Alsatian is an Alemannic dialect and in practice a language of its own, not tinted High German. We listened to the recordings ourselves: as a native German speaker without practice, you only catch roughly what it is about in stretches. What fails here is the outer edge of the spectrum, not the everyday case.
Second, the reference texts are written in dialect, with their own spelling. The model is therefore also penalized for writing High German even when it caught the meaning. You see that in the two metrics: word error rate 0.82, character error rate only 0.37. At letter level the result is much closer to what was said than the first number suggests.
It remains honest, though: for deep dialect the output is not usable as a transcript. On regionally tinted High German, the case that is far more common in practice, these numbers say nothing, because we simply have not measured it yet. We are expanding the set at that point. The Alsatian clips stay in as a stress test, so we can see progress when there is any.
Hallucinations on English material. A 60-second English recording with about 130 spoken words produced 321 words with a repetition loop on one machine, and only 74 on the other. Both results are wrong, and in the unpleasant way: the text reads fluently but is partly invented. The case is documented and in progress.
Very long files. The 4-hour recording stopped at 46 percent on one of the three test machines and never finished. We logged the abort as a failure.
On the other two it completed, but with a clearly different result: 0.053 over 38,649 words on the Mac versus 0.160 over 37,101 words on the Windows machine. The Windows transcript is missing about 1,400 words. It is still notable that a four-hour recording is among the best values in the series: long recordings are easier for recognition than short ones, because the model has more context.
Overlapping speech. When two people talk at the same time, text is lost. That holds for every model we know, local or in the cloud, and it is not a question of compute.
Numbers and dates need proofreading. The content is usually right; the spelling rarely is.
What comes next
Next we close the gaps in the set: regionally tinted High German is missing entirely so far, and for automatic speaker count we will compare what the app finds with the known number of speakers.
In parallel we are building the second review stage: a language model compares transcript and reference and scores whether an error changes the meaning. A wrong year in a quotation weighs more than a missing conjunction, and that is exactly what word error rate does not see.
The method itself stays: every new model version runs through the same 63 files before we ship it. Without that comparison we would not know whether an update is actually better or merely newer.
We will keep expanding the test set. A fuller analysis is meant to follow this simple regression; we want to publish that separately.
Regardless of the numbers: anyone who needs a reliable transcript listens back. The editor is built for that.
If you know a recording situation that is missing from our set, write to us. We will record it.