{"dataset":{"name":"On-device speech recognition in background talk (babble noise)","title":"Dictation in Noise: Speech Recognition Accuracy Tested","question":"How much worse does speech recognition get when other people are talking in the background?","url":"https://yapapp.cc/data/asr-noise-robustness","measure":{"label":"Word error rate","unit":"percent","short":"%","decimals":2,"lowerIsBetter":true},"dimensions":[{"key":"engine","label":"Engine","values":["Parakeet TDT v3","Whisper large-v3-turbo","Apple SpeechTranscriber"]},{"key":"maker","label":"Maker"},{"key":"condition","label":"Background talk","values":["No noise","20 dB","10 dB","5 dB","0 dB"]},{"key":"language","label":"Language"}],"include":"Each engine on the 100 English recordings of our main benchmark, once without noise and once at each babble level.","exclude":"Other languages and other noise types (traffic, music), for now.","method":"Babble for each recording is four other recordings of the same selection at equal loudness, summed and mixed at a fixed ratio below the speaker (20, 10, 5 and 0 dB); the mixing code and every file's checksum are in the full record. Engines and scoring are the same as the main benchmark.","limitations":"Synthetic babble from read speech is steadier than a real room, and one language and one Mac are tested. The ranking of engines and the shape of the curve are the finding.","sources":0,"updated":"2026-10-19","license":"https://creativecommons.org/licenses/by/4.0/","credit":"Yap (2026). On-device speech recognition in background talk (babble noise) [Data set]. https://yapapp.cc/data/asr-noise-robustness"},"rows":[{"row":"Whisper large-v3-turbo, No noise","value":3.78,"low":null,"high":null,"as_written":"3.78","unit":"percent","engine":"Whisper large-v3-turbo","maker":"OpenAI","condition":"No noise","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 3.78 percent on 100 English FLEURS recordings with no noise added.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-whisper-clean-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Parakeet TDT v3, No noise","value":4.44,"low":null,"high":null,"as_written":"4.44","unit":"percent","engine":"Parakeet TDT v3","maker":"NVIDIA","condition":"No noise","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 4.44 percent on 100 English FLEURS recordings with no noise added.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-parakeet-clean-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Apple SpeechTranscriber, No noise","value":6.73,"low":null,"high":null,"as_written":"6.73","unit":"percent","engine":"Apple SpeechTranscriber","maker":"Apple","condition":"No noise","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Apple SpeechTranscriber had a word error rate of 6.73 percent on 100 English FLEURS recordings with no noise added.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-apple-clean-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Whisper large-v3-turbo, 20 dB","value":4.2,"low":null,"high":null,"as_written":"4.2","unit":"percent","engine":"Whisper large-v3-turbo","maker":"OpenAI","condition":"20 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 4.20 percent on 100 English FLEURS recordings with background talk 20 dB quieter than the speaker (a quiet office).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-whisper-snr20-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Parakeet TDT v3, 20 dB","value":4.44,"low":null,"high":null,"as_written":"4.44","unit":"percent","engine":"Parakeet TDT v3","maker":"NVIDIA","condition":"20 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 4.44 percent on 100 English FLEURS recordings with background talk 20 dB quieter than the speaker (a quiet office).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-parakeet-snr20-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Apple SpeechTranscriber, 20 dB","value":7.61,"low":null,"high":null,"as_written":"7.61","unit":"percent","engine":"Apple SpeechTranscriber","maker":"Apple","condition":"20 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Apple SpeechTranscriber had a word error rate of 7.61 percent on 100 English FLEURS recordings with background talk 20 dB quieter than the speaker (a quiet office).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-apple-snr20-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Parakeet TDT v3, 10 dB","value":4.48,"low":null,"high":null,"as_written":"4.48","unit":"percent","engine":"Parakeet TDT v3","maker":"NVIDIA","condition":"10 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 4.48 percent on 100 English FLEURS recordings with background talk 10 dB quieter than the speaker (a busy office or cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-parakeet-snr10-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Whisper large-v3-turbo, 10 dB","value":5.51,"low":null,"high":null,"as_written":"5.51","unit":"percent","engine":"Whisper large-v3-turbo","maker":"OpenAI","condition":"10 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 5.51 percent on 100 English FLEURS recordings with background talk 10 dB quieter than the speaker (a busy office or cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-whisper-snr10-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Apple SpeechTranscriber, 10 dB","value":13.87,"low":null,"high":null,"as_written":"13.87","unit":"percent","engine":"Apple SpeechTranscriber","maker":"Apple","condition":"10 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Apple SpeechTranscriber had a word error rate of 13.87 percent on 100 English FLEURS recordings with background talk 10 dB quieter than the speaker (a busy office or cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-apple-snr10-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Parakeet TDT v3, 5 dB","value":7.85,"low":null,"high":null,"as_written":"7.85","unit":"percent","engine":"Parakeet TDT v3","maker":"NVIDIA","condition":"5 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 7.85 percent on 100 English FLEURS recordings with background talk 5 dB quieter than the speaker (a loud cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-parakeet-snr5-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Whisper large-v3-turbo, 5 dB","value":13.92,"low":null,"high":null,"as_written":"13.92","unit":"percent","engine":"Whisper large-v3-turbo","maker":"OpenAI","condition":"5 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 13.92 percent on 100 English FLEURS recordings with background talk 5 dB quieter than the speaker (a loud cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-whisper-snr5-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Apple SpeechTranscriber, 5 dB","value":31.34,"low":null,"high":null,"as_written":"31.34","unit":"percent","engine":"Apple SpeechTranscriber","maker":"Apple","condition":"5 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Apple SpeechTranscriber had a word error rate of 31.34 percent on 100 English FLEURS recordings with background talk 5 dB quieter than the speaker (a loud cafe).","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-apple-snr5-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Parakeet TDT v3, 0 dB","value":44.47,"low":null,"high":null,"as_written":"44.47","unit":"percent","engine":"Parakeet TDT v3","maker":"NVIDIA","condition":"0 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Parakeet TDT 0.6B v3 had a word error rate of 44.47 percent on 100 English FLEURS recordings with background talk as loud as the speaker.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-parakeet-snr0-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Whisper large-v3-turbo, 0 dB","value":66.09,"low":null,"high":null,"as_written":"66.09","unit":"percent","engine":"Whisper large-v3-turbo","maker":"OpenAI","condition":"0 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Whisper large-v3-turbo had a word error rate of 66.09 percent on 100 English FLEURS recordings with background talk as loud as the speaker.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-whisper-snr0-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"},{"row":"Apple SpeechTranscriber, 0 dB","value":71.88,"low":null,"high":null,"as_written":"71.88","unit":"percent","engine":"Apple SpeechTranscriber","maker":"Apple","condition":"0 dB","language":"English","conversion":null,"claim":"In our October 2026 test on a Mac with an M2 Pro, Apple SpeechTranscriber had a word error rate of 71.88 percent on 100 English FLEURS recordings with background talk as loud as the speaker.","quote":null,"where_in_source":null,"fact_url":"https://yapapp.cc/facts/m-noise-2026-10-apple-snr0-wer","source_title":null,"source_publisher":null,"source_year":null,"source_url":null,"source_doi":null,"checked":"2026-10-19"}]}