How do you translate a video into English subtitles?

Run Whisper's translate task on the video with the medium or large model, and ask for SRT output. The result is a numbered, timed subtitle file in English. Play the video with that file loaded and check each subtitle against the speech before you publish it.

Whisper writes an SRT subtitle file from the translate task

Whisper's command-line tool accepts srt and vtt as output formats, along with plain text and other formats ¹. For a Spanish interview, the command is:

whisper interview.mp4 --model medium --language Spanish --task translate --output_format srt

The --task translate flag produces English. Do not use turbo for this, because the README says turbo returns the original language even when the translate flag is set ². Use medium or large, which the README recommends for the best translation results ³.

Whisper decodes the input file with ffmpeg, which must be installed and on your path ⁴. That describes the open-source Whisper code. Apps that wrap Whisper may handle video files in their own way, so check the app's documentation if you use one.

Check each subtitle against the speech before you publish

Open it in a text editor and correct it line by line. Each entry is numbered in order, starting at 1 ⁵. An SRT file is plain text, so you can edit it in any editor.

1. Run the command and open the SRT file in a plain text editor.
2. Play the video with the subtitle file loaded, in a player that supports SRT, and watch the first few minutes.
3. Correct names, numbers and any line that does not match the speech.
4. Shorten any line that runs too long to read on screen, then check the timing of each subtitle.
5. Play the whole video once more with the final file before you publish.

Whisper can also produce text that is not in the audio. A 2024 study found that roughly 1 percent of its audio transcriptions contained entire hallucinated phrases or sentences that did not exist in the recording ⁶. That study tested Whisper as it stood in 2023 ⁷. A later study found that when non-speech sounds were played to the model, a set of hallucinated phrases appeared frequently ⁸. So check the stretches with no speech first.

Where the video comes from changes what you can do

Before you download anything, check the rules for the site and the video.

• Downloading. YouTube's terms say you may not download any part of the service or its content unless the service expressly authorizes it ⁹. Get the file from its owner, or use a video you have the right to download.
• Captions that already exist. YouTube's 2008 launch post says auto translate adds machine translation to the caption tracks that uploaders add to a video ¹⁰. It translates existing captions rather than listening to the audio, so a video without captions needs a transcript first. That description is from 2008, so check YouTube's current help before you rely on it.
• Burned-in subtitles. FFmpeg's subtitles filter draws subtitles onto the frames of a video ¹¹. Burned-in text cannot be removed later without re-editing the video, so keep the SRT file to fix a line and burn the subtitles in again.

It depends on the language, the speech and the audience

• The language. Translation quality depends on the spoken language. The question on translating Japanese audio and the question on translating Zulu or Xhosa audio show how two languages differ.
• The speakers. Several voices, crosstalk and music make the transcript harder to check. Read the output against the video for each speaker change.
• The audience. Broadcast and regulated captioning can have rules beyond a translated subtitle file. This guide does not cover them, so check the rules that apply to your audience.
• The stakes. For legal, medical or other high-stakes video, have a qualified human translator review the subtitles.

Where to go next

The guide to translating audio and video to English covers the same method for audio files and the choice of model. For the wider subject of subtitle formats, see our guide to captions and subtitles. If the video is a recording on a phone, the question on translating a voice note to text gives those steps. The guide to running Whisper on your own computer explains how to keep the video on your device.