I didn't think I could get another local transcription model to run faster than Parakeet but alas, 4x faster 2x memory size and a minuscule error rate difference
Will test more.
5
2
596
And then FluidInference decides to go and do this
further benchmarking
xitter.twitterapi.xyz/Alex_tra_memory/status…
NVIDIA Nemotron 3.5 Transcribe, ported to Apple's Core ML and Core AI on M5 Pro zero day launch.
Core ML (video): a 49.5-min, 3-person meeting → speaker-labeled transcript with word timestamps in 6.0s , 500 rtfx .
- Speech encoder + CTC head on the GPU
- Nemotron-3-Diarization on the Neural Engine, in parallel
- ~1.5 GB, no LLM
Core AI: same encoder + CTC head as a 1.3 GB .aimodel. Core AI, Core ML and PyTorch give the same transcripts on LibriSpeech (4.62 / 4.64 / 4.64% WER, 400-utt subset). 234x vs 238x on GPU.
Audio: AMI EN2002c (CC BY 4.0).
1
40
@desertantlabs thanks for an awesome release! I compared Voz with Parakeet TDT 0.6B v2 running through Core ML on my M1 Air. Voz was much faster, while v2 had slightly lower WER on my samples.
Have you tried applying your optimizations to v2 (English-only)? If so pls release :)
The best quality of life improvement this year has got to be Parakeet TDT v2 (English only) running as a local paralleled service on multiple M-series MacBooks.
I use Cloudflare for my router, and if all my devices are offline it uses a third party service as a fallback (Whisper v3 Turbo via Groq)
The cherry on top is setting up the two side-buttons on my mouse. One is a boolean toggle for record/stop record, the other executes "Enter".
3
3
468
Not only can I use this for my individual daily usage, the system is setup to be a durable service via Cloudflare so that I can use my local compute in production environments.
161
Terra 6.1 running at ~120 tokens per second is a breath of fresh air since OpenAI nerfed Astra and Sol speeds.
However, a manufactured improvement after degrading a service is not an actual improvement.
4
577
I have 100+ tear downs just from this week.
They all look like this, and worse once I actually probe into them with my toolkit.
Static file analysis reaps artifacts bruv no CVE necessary
2
49
