How accurate is Apple's new transcription AI? We tested it against Whisper and Parakeet

“Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table, 'How they did')

“the fact that Apple’s new transcription API is faster than Whisper is great news” (Introduction)

“Parakeet v2 2 seconds 5.8% 12.3% Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table)

“even if the exact numbers would differ under slightly different methods” (Methodology section)

“Parakeet v2 2 seconds 5.8% 12.3% Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table, 'How they did')

“For Apple’s speech API, I used Finn Vorhees’ excellent Yap project. I ran them on my M2 Pro MacBook Pro with 16GB of RAM.” (How I did it)

“Parakeet v2 2 seconds 5.8% 12.3% Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table)

“Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table, 'How they did')

“Parakeet v2 2 seconds 5.8% 12.3% Whisper Large V3 Turbo 40 seconds 0.2% 1.5% Apple 9 seconds 1.9% 10.3%” (Results table)