Can Whisper run in the browser?

Yes. Whisper can run inside a web page with Transformers.js, a JavaScript library from Hugging Face. The smallest models are the practical choice for most pages, because the largest model needs about 10 GB of video memory ¹ and a long download. The page runs the model on the CPU by default, and it can use the graphics chip through WebGPU when the page asks for it ².

Whisper is OpenAI's open speech recognition model. The guide to the Web Speech API compares it with the browser's built-in feature. By default, the browser's built-in recognition sends audio to a service ³, while Whisper running in the page keeps the processing in the browser. That is the main reason to use it, and it comes with the cost of a model download.

The model size decides what a page can handle

Whisper comes in several sizes. The smallest, tiny, has 39 million parameters ⁴, and the largest, large, has 1,550 million ⁵. Larger models need more memory and time to run. OpenAI's model table lists about 10 GB of video memory for the large model ¹. Those are OpenAI's estimates for running the model, so use them to choose a size, not to predict one laptop's speed.

For a browser page, start with tiny or base and measure accuracy in the languages your visitors speak. Only move up when the smaller model is not good enough for your audio.

Transformers.js runs on the CPU unless the page asks for WebGPU

Transformers.js runs a model on the CPU through WebAssembly by default in the browser. It runs on the GPU through WebGPU only when the page sets the device option ². WebGPU needs a browser and a graphics driver that support it.

Support has varied by platform. Google's Chrome team said in 2023 that the first release, in Chrome 113, covered ChromeOS devices with Vulkan, Windows devices with Direct3D 12 and macOS, and that Linux and Android would follow ⁶. That post is old, so check current support in each browser you target before you rely on it.

Here is the loading code, using the tiny multilingual model from Hugging Face. We did not run this in a browser, so test it before you ship:

import { pipeline } from '@huggingface/transformers';

const device = navigator.gpu ? 'webgpu' : 'wasm';
const transcriber = await pipeline('automatic-speech-recognition', 'Xenova/whisper-tiny', { device });
const result = await transcriber(audioData);
console.log(result.text);

The audioData value is the audio you pass in, such as samples from a microphone or a file. The check for navigator.gpu falls back to the CPU path when the browser has no WebGPU, which avoids a failure on older machines.

The download is the first cost

A page that runs Whisper must download the model before its first transcription. A model repository on Hugging Face splits the model into several files, such as an encoder and a decoder, and can offer quantized versions of each. Open the repository's file list and add up the files your page will load. Quantized files are smaller than the full-precision ones, but they can change accuracy, so test the variant you ship on your own audio.

It depends: the model, the license and the device

• The model you pick. Tiny is light and fast to load, while large needs a capable graphics card. Choose by the accuracy your audio needs, not by the name of the biggest model.
• The license. The OpenAI Whisper repository says that its code and model weights use the MIT License ⁷. Check the license of the converted model you download, since a community conversion may carry its own terms.
• The device. A phone or an older laptop may load the model slowly or fail to run it. Show a progress message during the download, and offer a typed fallback.

What to do

1. Start with the tiny or base model. Measure accuracy on your own audio, in each language you support.
2. Use WebGPU when it is available, and the CPU path when it is not. Test both.
3. Show the download progress. A model download can take a while, so it needs a visible message, not a blank page.
4. Keep the audio on the page. Once the model is loaded, the transcription runs in the browser, with no server call in the path you wrote.
5. Use a desktop route if the page is too heavy. The guide to running Whisper locally covers the options on a computer, and free audio to text in the browser transcribes a file without any setup.

For the built-in browser feature and its privacy trade-offs, read whether the Web Speech API sends audio to Google. For Whisper on a Mac or on Linux, see Whisper on Mac and Whisper dictation on Linux.