In October I needed an exact record of a conversation I had recorded on my phone: 44 minutes outdoors, five people and one side of a phone call on loudspeaker, with wind, walking and people talking over each other. The first transcript, Whisper large-v3-turbo over the whole file, repeated one sentence 430 times. With the loop fixed it still had garbled words, and it had no speakers at all.
I built it on my experience from Wasabi, where I worked on multilingual speech-to-text and built diarization pipelines. Everything here runs on a Mac with local models.
The only judge who was there
No model knows what was actually said, and I was there. So my time went into an answer key: four test passages, four and a half minutes of audio, corrected by ear. They covered two people speaking clearly, an argument with overlap, the phone call and the exchange that mattered most to me, 530 words in 50 lines. Then I measured every model, filter and trick against my corrections instead of against each other.
What the measurements said
Four systems, each on the raw audio, on a light ffmpeg filter, and on audio cleaned by DeepFilterNet at full strength and with its attenuation capped at 18 dB. Word error rate on my passages:
| Raw | Light filter | DeepFilterNet | DeepFilterNet, capped | |
|---|---|---|---|---|
| Whisper large-v3 | 14.9% | 11.1% | 27.9% | 19.6% |
| Whisper large-v3-turbo | 17.4% | 19.2% | 31.5% | 19.8% |
| NVIDIA Parakeet-TDT-0.6B-v3 | 20.6% | 22.6% | 33.0% | 31.5% |
| Sheptooha, Polish w2v-BERT 2.0 | 42.6% | 38.5% | 48.3% | 47.0% |
Three things surprised me.
Noise removal made every model worse. DeepFilterNet's output is much easier for a person to listen to, and it added 10 to 17 points of errors compared with the same model without it, and still 2 to 11 with the cap. It stayed in the tool as a toggle for my ears only.
The Polish-only model came last. Sheptooha beats Whisper by a wide margin on its author's benchmark. On my recording it left out far-off speech: 142 of the 530 words were missing.
Combining the models didn't help either. Word-by-word voting across four systems, tuned on three passages and tested on the fourth, scored 13.0% against 11.1% for large-v3 alone. Adding the previous chunk's text to Whisper's prompt, which is supposed to give it context, scored 14.7%.
What worked was plain. I cut the speech into chunks of at most 28 seconds, split at the quietest moment, so Whisper never decodes long silence; that's where it invents "thanks for watching". Each chunk gets a prompt with the names and terms likely to be said, and the audio gets the light filter: a highpass, mild denoising and dynamic normalization.
The losing models as a second opinion
The models that lost were still useful. When large-v3 writes a word that fewer than two of the four other systems heard within 1.2 seconds, the word goes in italics. That marks 10% of the words, 39% of the marked words are wrong, and it catches 61% of the errors. Whisper's own confidence, the usual way to do this, caught 32%.
No rule was good enough to replace words automatically. Even the strictest combination marked words that were right more than half the time, so the transcript only shows doubt. Nothing is filled in from context: every word comes from a model or from me.
Timing was a separate problem. Against Parakeet, 80% of Whisper's word times were within 0.6 seconds and 10% were more than a second off. Parakeet and Sheptooha, two independent timers, agreed within 0.14 seconds on 80% of words, so line times come from Parakeet.
Who said what
Unsupervised speaker clustering, told there were five people, put the two people closest to the phone into one cluster. What worked was my corrections again: every line I checked is a voice sample with a name on it. Each two-second window of the recording is compared with each person's average voice, and the closest one wins. On checked lines held out from the samples, CAM++ named the right person on 98% of 49 lines. With all 141 lines I checked in the end, CAM++, ERes2Net and ERes2NetV2 were between 96.5% and 97.2%, so the tool tests all three and picks.
Two rules mattered more than the model. Someone who spoke once or twice appears only where I placed them; otherwise a rare voice that sounds like a common one takes over a minute of someone else's speech. And the caller appears only during the call.
The page
The first version of the review page had a checkbox on every line and the other models' versions under each word, and it was unusable. The second works like a text editor: one passage at a time, one player, one line per person. Enter splits a line where the next person starts, Backspace joins two, Ctrl and a number sets who is talking, and the line being played lights up.
Most of the later fixes were about time, because the highlight didn't follow the audio and passages cut off their first or last words. Line times now come from Parakeet, a line's end is capped at the next line's start, the highlight sits on the line that started last, and playback runs from a second before the passage to three seconds after it.
The four test passages took me 30 to 40 minutes, including learning the page. I then checked eleven more passages, about eight minutes of audio, in roughly an hour. The final transcript has 163 lines checked by ear and marked ✓, 359 of the 2,055 remaining machine words in italics, and 21 lines with a weak speaker match.
What went wrong
Before publishing, I ran it end to end on a recording made with the Mac's speech synthesizer, where the right text is known. The first word after a checked passage was missing. Writing the transcript, the tool removed machine words in a time window around each checked passage, and because Whisper's times drift, the window also took words nobody had checked. In my real transcript that had happened at 3 of 30 passage edges, one to three words each time. It now removes exactly the words the page showed me.
The answer key also leans toward the winner: my corrections started from the large-v3 text, so its mistakes were the ones I saw first. And all of this is one recording in Polish, which is why the tool measures again on every new one.
Using it
It's a Claude Code plugin in plrojek/agent-skills, under MIT: run /plugin marketplace add plrojek/agent-skills, then /plugin install transcript-review@plrojek. It needs a Mac with Apple Silicon, uv, ffmpeg and whisper.cpp, and downloads about 5 GB of models on the first run. It asks who was there and which moments matter, runs the models and opens the page; after you correct the test passages, it tells you which model is best on your recording, not mine.