How to Translate Audio and Video to English Text

You can get English text from a recording in another language by one of two routes. A single model can translate the speech straight into English, or you can transcribe the recording in its own language and then translate the transcript. The free route is Whisper, which can translate on your own computer. How good the English is depends heavily on the language. In one published test of read sentences, Whisper's large-v2 model translated German speech into English with a score of 34.6 BLEU ¹, while Yoruba speech scored 1.4 BLEU ². Check any translation that matters against the original recording.

If you are new to the subject, our overview of dictation languages explains which tools support which languages, and how to change the dictation language covers the settings. This guide is about turning a recording into English text.

Translating a recording takes two steps, and some tools do both at once

Translating speech means two jobs: recognizing the words in the original language, then rendering them in English. Some tools hide both behind one button. Whisper's paper describes one model that performs several tasks on the same audio, including transcription and translation ³. Either way, a mistake in recognition becomes a mistake in the English. This is called error propagation, and the SeamlessM4T paper lists it as a known weakness of cascaded systems, which run recognition and translation as separate models ⁴.

A single model reduces that risk without removing it. SeamlessM4T improved into-English speech-to-text translation by 1.3 BLEU points on average over strong cascaded systems, according to its builders ⁵. BLEU is the usual score in these papers. It compares a machine translation with a human reference translation, and a higher score is closer. It counts word overlap, so it is a proxy for meaning, not a measure of it.

The table below sets the main routes side by side. The costs come from the sources we checked, and some routes are not priced in those sources at all.

The table is a starting point, not a ranking. The right route depends on the language, the length of the recording and how private it is.

For a gist, a direct translation is often enough. For anything you will quote, sign, publish or act on, transcribe first, check the transcript against the audio, and only then translate it. The rest of this guide explains why that order matters and how to do each step.

Whisper translates speech from its supported languages straight into English

Whisper is the one tool in the table you can run yourself, and its translate option is the most direct way to get English from a recording. OpenAI's guide says Whisper supports 98 languages, but that accuracy varies by language ¹⁰. The paper reports 125,000 hours of translation data into English ¹¹, which is one reason a single model can handle both transcription and translation.

Run the translate task on your own computer

These steps use the command-line tool from the Whisper README. They assume a Mac or Windows computer with Python installed.

1. Install the package with pip install -U openai-whisper ¹².
2. Install ffmpeg, a command-line tool that the package requires ¹³.
3. Run the translate task. Name the spoken language and use a model that is not turbo: whisper japanese.wav --model medium --language Japanese --task translate ¹⁴.
4. Open the output and check it against the recording, paying attention to names, numbers and any passage you cannot follow.

The turbo model returns the original language even when you ask for a translation ⁷. If a translate run gives you Japanese text when you wanted English, switch to medium or large, as the example above does.

For Japanese recordings specifically, see the question on translating Japanese audio to English.

Use the API when you do not want to install anything

OpenAI's translations endpoint takes an audio file and returns English text. The guide says to use it with whisper-1 ¹⁵, and it notes that the endpoint supports translation into English only ⁸. You send the file, wait for the English text, and read it. Nothing needs installing on your computer, but the audio is sent to OpenAI's servers, so the privacy section below applies.

Not every model that advertises translation works in your direction, and that matters for this job. Before you choose a tool, read its documentation and confirm that it translates from the language you have into English.

Transcribe first and translate second when you need to check the original

Transcribing first costs one extra step, but it gives you the words the speaker used, which you can check against the audio before you translate. That check matters most in the languages where direct translation is weakest. The table below sets the transcription error rate against the translation score for the same model, Whisper large-v2, on the Fleurs test of read sentences. A lower error rate is better, and a higher BLEU score is better.

A word error rate counts the words a transcript gets wrong, as substitutions, deletions and insertions, against a reference transcript, and divides by the number of reference words. It is defined in our glossary entry on word error rate. A character error rate does the same job by character, which suits scripts without spaces between words. The rows above are word-based.

The ranking is plain. In this table, the two languages with the highest transcription error rates, Swahili and Yoruba, have the two lowest translation scores. Four rows are a small sample, and BLEU and error rate measure different things, so the table shows a pattern and does not prove a rule. The chart ranks the translation scores on their own.

Transcripts also carry a risk that the error rate does not show. A 2024 study of recordings from speakers with and without aphasia found that roughly 1 percent of audio transcriptions from the 2023 version of Whisper contained entire hallucinated phrases or sentences that did not exist in the audio ²². Read the output against the recording, and listen closely to passages of silence and background noise.

Once the transcript is checked, any text translator can take it into English. Keep a short list of names, places and numbers from the original, and compare them with the English after translation. For a broader view of the transcription side, see our guide to transcribing audio in other languages.

Videos get translated subtitles from their audio track

For a video, the most useful output is a subtitle file. Whisper's command-line tool writes srt and vtt files directly, alongside plain text and other formats ²³. This command translates a Spanish video into English and writes an SRT file:

whisper interview.mp4 --model medium --language Spanish --task translate --output_format srt

The ffmpeg install from the steps above is required before this command can run. Run the command, then open the SRT file in a plain text editor to check it. An SRT file is plain text, so you can correct it line by line.

A practical order for a subtitle project is to translate, save the SRT, play the video with the subtitles on, and then fix the timing and the line length of each cue. Line length decides how easy a subtitle is to read, so shorten any line that runs long, even when the translation itself is right.

YouTube offers a different route. Its 2008 launch post says auto translate adds machine translation to the caption tracks that uploaders add to a video ⁹. The post describes translating caption tracks, so a video without captions needs a transcript first. Before you download a video to run it through any tool, check the site's terms, because many video sites restrict downloading. Our guide to captions and subtitles covers the file formats in more depth, and the question on translating a video into English subtitles gives the single-video steps.

Voice notes and messages can be transcribed and translated on the phone

On a phone, the simplest route is the recorder app. Samsung's Voice Recorder transcribes a memo and then lets you translate it into other languages, share it or move it to Samsung Notes ²⁴. Before you rely on a translation at scale, check whether the recorder's translation is free or part of a paid AI plan.

On any phone, the steps are the same. A voice note is short, so it is quick to check before you act on it:

1. Record the note, or import it into the recorder app.
2. Transcribe it in the language the speaker used, not in English. Choosing the right language first avoids a transcript that is wrong from the start.
3. Translate the transcript into English.
4. Listen again while you read the English, one phrase at a time, and correct any word that does not match what you heard.

Keep the transcript even when an app hides it, because the transcript is the step that makes the translation checkable. Our guide to changing the dictation language covers the language settings on each platform, and the question on translating a voice note to text gives the phone steps on their own.

Low-resource languages translate worst, so check important passages with a native speaker

The languages with the least training data get the least accurate English, and the gap is large. On the CoVoST 2 test of speech translated into English, zero-shot Whisper averaged 36.2 BLEU for high-resource languages ²⁵ and 25.2 BLEU for low-resource languages ²⁶. Those are averages, and the Yoruba result in the earlier table, at 1.4 BLEU ², shows how far an individual language can fall below them. A low-resource language is one with little recorded speech and text to train a model on.

Scores are only as reliable as the data behind them. A low average tells you the model had little to learn from, and a high average tells you very little about the one language you care about. Check the language you actually need, on the kind of audio you actually have, rather than trusting a headline number.

Zulu and Xhosa have little support for translation into English

SeamlessM4T's language table labels Zulu as low-resource for speech translation into English ²⁷ and marks Xhosa as zero-shot for the same task ²⁸. The table gives no translation score for either language. A 2025 study of isiXhosa speech states that Whisper's training data includes no isiXhosa and no closely related languages ²⁹, so do not expect Whisper to translate Xhosa well.

Whether a tool can transcribe a language is a separate question from whether it can translate it into English, so check both. In the sources we checked, we did not find a published English translation score for Zulu or Xhosa speech. For Zulu and Xhosa recordings specifically, see the question on translating Zulu or Xhosa audio to English.

Read the transcript with a speaker of the language before you translate it. For a recording that matters, such as evidence, a medical history or a contract, use a qualified human translator and do not rely on machine output alone. Our guide to speech recognition and accents covers how accent and dialect change the accuracy of the same tools.

Check the English before you rely on it, whatever the route

Start with the language. Set the language yourself rather than leaving the choice to detection, as the examples above do. A wrong language setting produces output that looks like a translation and is not one.

Then work through a short checklist:

1. Keep the original recording and the transcript together, so you can show what was said and what the English is meant to mean.
2. Check names, places, dates and numbers against the audio first. They are the easiest words to check and the costliest to get wrong.
3. Listen to the original while you read any passage that sounds smooth in English. Fluent English can hide a wrong word, so smooth is not the same as correct.
4. For subtitles, play the video with the subtitle file loaded, and check the timing and the line length in each cue.
5. For anything that matters, have a speaker of the language check the passage, as the low-resource section explains.

Keep confidential recordings on your own device

A recording of a client, patient or family member should stay on your device when you can manage it. Sending audio to a cloud service means it leaves your device, and the provider's policy governs what happens to it after that. Read that policy before you send anything confidential, and ask whether your organization allows it at all.

Local tools keep the audio on the device. The whisper.cpp project shows Whisper running fully offline on an iPhone 13 ³⁰, so the same model family can work without a connection. Our guide to running Whisper on your own computer covers setup, and the guide to Whisper and privacy covers what the cloud services do with audio.

Yap, which we make, is an on-device dictation app for Mac and Windows, so the speech you dictate stays on your device. It is built for dictating your own text, and the download page has the details for each platform.

Before you record anyone, check the law where you are. The rules differ by place, and some places require consent from everyone who is recorded. Our overview of recording consent laws lists the main rules, and it is not legal advice.