Why is Whisper not using my GPU?

Whisper uses your GPU only when the program can detect a CUDA-capable NVIDIA card. If it cannot, openai-whisper falls back to the processor without stopping, so the run works but is slow. The check happens in PyTorch, the library Whisper runs on, so that is where to look first. Run the check, install a CUDA build of PyTorch if the answer is no, and read the device warning if the answer is yes.

Why the run quietly uses the CPU

The openai-whisper command line runs on an NVIDIA GPU, through CUDA, only when PyTorch can detect one. Otherwise it falls back to the CPU ¹. The detection comes from PyTorch's torch.cuda.is_available() function, which returns a bool saying whether CUDA is currently available ².

There is one warning to know about. When the model is on the CPU and CUDA is available, Whisper prints "Performing inference on CPU when CUDA is available" ³. A slow run with no such message suggests that CUDA was not found.

Step 1: Check what PyTorch can see

Run this in the same Python environment that runs Whisper, activating a virtual environment first if you use one:

python -c "import torch; print(torch.cuda.is_available())"

It prints True or False. Use the answer to choose the next step.

Step 2: If the answer is False, install a CUDA build of PyTorch

A False answer means PyTorch cannot use CUDA on this computer. Install a PyTorch build with CUDA support, following the install instructions on PyTorch's website for your system, and then run the check again. If you use faster-whisper rather than openai-whisper, its GPU mode has its own requirement: NVIDIA libraries must be installed first ⁴.

Step 3: If the answer is True, look for a CPU override

When CUDA is available, the command line already defaults to the GPU. If the run still uses the CPU, look for a --device option set to cpu in your command, since an explicit option overrides the default. The CPU warning above will appear in that case, which confirms the cause.

Macs never reach the CUDA route

A Mac has no NVIDIA card, so the check above returns False and openai-whisper runs on the CPU. That is expected, not a broken install. For Apple silicon, the whisper.cpp README describes the platform as a first-class target and lists Metal among its optimizations ⁵. Our guide to running Whisper locally covers the Mac routes.

Other errors that look like a GPU problem

• A warning about FP16. When openai-whisper runs on a CPU, it warns that FP16 is not supported there and uses FP32 instead ⁶. This is a warning, not a failure, so the run continues at CPU speed.
• An out-of-memory error. The large model needs about 10 GB of video memory ⁷. Choose turbo, which needs about 6 GB ⁸, or a smaller model.

What to do, in order

1. Run the one-line check and note whether it prints True or False.
2. If False, install a CUDA build of PyTorch and check again.
3. If True and the run is still on the CPU, remove any --device cpu option from your command.
4. If the GPU is found but memory runs out, choose a smaller model. Our answer on the best GPU for Whisper lists the memory each model needs.

If the run never reaches the GPU and you do not need one, our answer on running Whisper without a GPU shows what to expect on the processor. For the install steps themselves, see how to install Whisper AI.