Is Whisper AI Safe? Privacy and Security Explained

Whisper is safe for private audio when you run the open-source model on a computer you control, because the audio never leaves that computer. The hosted OpenAI API is a different case. It sends your audio to OpenAI, and OpenAI's data terms decide how long it is kept and whether it can train models. Most privacy problems with "Whisper" come from mixing up the two, because many apps carry the name and do not say which one they use.

Whisper the model runs locally, Whisper the API sends audio to OpenAI

OpenAI released Whisper as an open-source speech recognition model, and its code and weights are under the MIT License ¹. Anyone can download the model, run it on their own hardware and transcribe without sending anything to OpenAI. The name also belongs to a paid service, OpenAI's hosted transcription API, where your recording is uploaded and transcribed on OpenAI's servers.

The two paths have different privacy properties, so the first question to ask about any Whisper product is which path it takes. The table shows the four setups you are most likely to meet.

The last two rows matter most, because an app's name rarely tells you which row you are in. The checks in the section on apps named after Whisper show how to find out.

Running Whisper on your own machine keeps audio off any server

Running Whisper locally is the strongest privacy option, and it needs the most setup. The whisper.cpp project shows the model running fully offline and on the device, in a demonstration on an iPhone 13 ². A demonstration shows the design works, not how fast it runs on your machine, so test your own setup.

The first run downloads the model file, which is the one part of a local setup that touches the internet. Once the file is on disk, the transcription itself needs no connection. You can confirm that for any tool with the airplane-mode test in our guide to whether voice to text is safe.

A local setup writes files, so decide where they go

A command-line setup also writes transcripts to disk. If you do not choose an output format, the openai-whisper command line produces all available formats ³. Several of those formats hold the full text, so a privacy setup has to cover every copy. Pick one format, choose the output folder yourself, and keep that folder on an encrypted drive. Install the other tools a local setup needs, such as ffmpeg, from sources you trust, and know what each install adds.

Model size decides the memory you need, not the privacy

Larger Whisper models need more memory. The README's figures are video memory (VRAM) on a graphics card. The chart shows the approximate figure for each model, and the text below it cites each one.

The base model needs about 1 GB ⁴, the small model about 2 GB ⁵, and the medium model about 5 GB ⁶. The turbo model needs about 6 GB ⁷, and the large model about 10 GB ⁸. Without a suitable graphics card you can usually run a smaller model on the processor, which these figures do not measure.

Every model in that list runs offline. The choice affects accuracy and speed, not whether your audio leaves the machine. A smaller model is still private, so pick the largest one your computer runs at a comfortable speed and test it on your own recordings.

A model file is loaded like code, so get it from the official source

A PyTorch checkpoint is loaded through an unpickler, so it is not just data. The openai-whisper loader checks a downloaded file against a SHA256 checksum and downloads it again when the checksum does not match ⁹. It also loads checkpoints with weights_only set to True on PyTorch 1.13 and later ¹⁰, which limits what the loader will unpickle.

PyTorch's own documentation is still blunt about the risk. It warns that "Never load data from an untrusted source." ¹¹ In practice, download models through the app or the project's documented source, and avoid model files from forum posts or unknown mirrors. A file you cannot inspect deserves the same caution as any other software download.

Local setups need upkeep, and updates can change what happens

An app that was local last month can add a cloud feature in an update, so read the release notes and check the settings after each update. Re-run the airplane-mode test after any change to the app, the model or the operating system. Keep the model folder and the transcript folder under your control, and delete transcripts you no longer need, since a transcript left in a downloads folder is the same audio in text form.

OpenAI's API terms govern retention and training when you use the hosted model

When you send audio to OpenAI's API, OpenAI's data controls decide what happens to it. Two defaults matter most.

The first is training. OpenAI says that data sent to its API has not been used to train or improve its models since March 1, 2023, unless you explicitly opt in to share it ¹².

The second is retention. By default, abuse monitoring logs are kept for up to 30 days ¹³. The page says OpenAI keeps them longer only where the law requires it or to protect its services.

There is one nuance you should know about. OpenAI's per-endpoint table lists the transcription endpoint with no abuse monitoring retention and no application state retention, and marks it as eligible for Zero Data Retention ¹⁴. The general statement above that table says all API feature usage generates logs kept for up to 30 days ¹³. The page does not reconcile the two, so we cannot tell from it which applies to your account. Treat the table as OpenAI's stated behaviour for that endpoint, and ask OpenAI in writing before you rely on it for records that carry legal or medical weight.

Zero retention is not a setting anyone can switch on. OpenAI says its Zero Data Retention and Modified Abuse Monitoring controls currently require prior approval by OpenAI and acceptance of additional requirements ¹⁵. Until an organization has that approval, assume the 30-day default.

Before you send recordings to the API, decide which kinds of recording you would never send without a signed agreement. Then check which settings apply to your own account, not only the defaults on a documentation page, and keep a written note of the setting you chose and the date you chose it. If a vendor's documentation and its own account screens disagree, ask the vendor, and keep the reply.

For the full question of what OpenAI stores, see our answer on whether OpenAI stores Whisper audio and the guide to voice data retention, which compares the other large vendors.

Many apps named after Whisper use a cloud API, so check before you dictate

Searching for Whisper turns up desktop dictation tools, Android keyboards and Mac transcription utilities. Their privacy depends on how each one is built, and the same app can offer both paths.

Superwhisper is the clearest example. On its Free plan, dictation uses local Whisper models without AI processing, and the company says nothing dictated leaves the device ¹⁶. Its Pro plan adds cloud models and AI processing ¹⁷. If you enter your own API keys, the company says it acts as a client of your account, and your agreement with the provider governs the data ¹⁸.

Whisper To Input, an Android keyboard, sits at the other end. Its recognition runs either on the OpenAI API with your own key, or on a Whisper server you host yourself ¹⁹. Both routes are legitimate, but only the second keeps the audio on infrastructure you run.

MacWhisper says it can "Process sensitive content locally without data ever leaving your Mac" ²⁰. That is the vendor's claim, which we have not tested, so run the check yourself.

Four checks tell you which path an app takes:

1. Look for an API key field. If the app asks for an OpenAI or other provider key, your audio goes to that provider under your account.
2. Read the processing statement, then the settings. Find out whether local and cloud modes exist, and which one is on by default.
3. Run the airplane-mode test on a real recording. Text that still appears with the network off is produced on the device.
4. Do not rely on the App Store privacy label alone. A 2023 study found that Apple does not validate the labels developers fill in, but requires developers to provide a privacy policy for comparison ²¹. A 2023 study found that 97 percent of apps labelled Data Not Collected had privacy policies indicating otherwise ²². Read the policy.

Our question on whether Whisper apps send audio to OpenAI walks through these checks app by app, and the guide is voice to text safe explains the places your words can go.

Whisper can invent text, which matters when a transcript is a record

Privacy is not the only security question. Whisper can produce text that nobody said. In a 2024 study of Whisper transcriptions, roughly 1 percent contained entire hallucinated phrases or sentences that did not exist in the audio ²³. The study's analysis found that 38 percent of those hallucinations included explicit harms, such as perpetuating violence or implying false authority ²⁴. It used one dataset and an earlier version of the model, so the rate on your recordings may be different.

The Whisper authors list complete hallucination as a known failure, in which the model outputs a transcript entirely unrelated to the actual audio ²⁵. That is a failure of the whole output, not a single wrong word, so a transcript can look fluent and still be wrong.

Why this matters for security: once a transcript becomes a record, such as a clinical note, a legal file or meeting minutes, an invented sentence is an integrity failure, and encryption does not fix it. Review the transcript against the audio before you file it. Compare each sentence with the recording at the point where it was spoken, and mark any passage you cannot match. Keep the audio until the review is finished, because the transcript is the only copy you can check against it. Our question on why Whisper makes up words explains the causes, and the guide to how accurate speech recognition is gives the wider picture.

Whisper is safe for sensitive work in a setup you control or one covered by an agreement

Regulated data needs a documented arrangement, and the model itself does not provide one. HHS says that the covered entity and a cloud service provider holding its health information must enter into a HIPAA-compliant business associate agreement ²⁶. The HIPAA Privacy Rule allows a covered entity to let a business associate handle protected health information only if the entity obtains satisfactory assurance that it will be safeguarded ²⁷. So the agreement, not the model, decides the answer.

For the OpenAI API, the data controls page says that customers who have executed an OpenAI Business Associate and Healthcare Addendum can use BAA-eligible endpoints for processing PHI, once their organization is provisioned with Private Retention and Private Safety Processing ²⁸. The page ties this to a signed agreement and a provisioned organization, so a personal account does not meet the condition. Ask OpenAI to confirm the specific endpoint you plan to use. Our answer on whether Whisper AI is HIPAA compliant applies this to common setups.

Local Whisper avoids a third party, but it does not remove your duties. The HIPAA Security Rule requires an accurate and thorough risk analysis of the risks to electronic protected health information ²⁹. Decide how the device holding dictated notes is encrypted, and document that decision. A laptop with unencrypted notes is still a risk, even if no server ever received the audio. The guide to HIPAA-compliant dictation covers the rule in more detail.

Superwhisper's documentation recommends its bring-your-own-key setup for organizations that already have HIPAA business associate agreements with an AI provider ³⁰. That is the vendor's recommendation, not a legal finding that the setup complies, and the agreement with the provider is still yours to put in place. For terms like this, see the glossary entries on business associate agreement, protected health information and bring your own key.

A setup you can defend for sensitive audio looks like this:

1. Use local Whisper, or an app whose local mode you have tested offline, for content that must not leave the device.
2. If you use the API, sign the provider's agreement, confirm your organization is provisioned, and check the endpoint's eligibility in writing.
3. Encrypt the device, limit who can open the transcripts, and keep audio only as long as your rules require.
4. Review each transcript against its audio before it becomes a record.
5. Ask a lawyer or compliance officer in your jurisdiction to confirm the arrangement. This is a description of the rules, not individual legal advice.

If you want the local setup without building it yourself, Yap, which we make, is an on-device dictation app for Mac and Windows. Read its privacy terms before you rely on it, as you would for any app in this list. For the wider picture of where dictated audio goes, start with the overview of speech to text privacy, and see the guide to how to run Whisper locally for setup steps and the question on running Whisper offline.