How accurate are AI medical scribes?

Most AI medical scribe drafts are free of significant errors, but not all of them, so every draft needs checking before it is signed. In a 2024 pilot, most reviewed notes had no significant errors, while a small share contained errors that could cause harm if left uncorrected. Accuracy depends on the product, the audio and how the errors are counted, so no single figure applies to every scribe.

Most drafts had no significant errors, but a small share carried serious risk

In a 2024 pilot at UC Davis, physicians reviewed 356 notes made with an ambient scribe, and 94.7 percent were free from significant errors ¹. The remaining notes contained errors, and 19 notes, or 5.3 percent, contained an error that physicians rated as posing serious or imminent risk if it was not corrected ².

The counts describe the notes that physicians reviewed, a sample of all the notes the scribe created, so they are not an audit of every note. The true rate of errors could be higher or lower.

Omissions are the most common error, and invented content is the most worrying

The UC Davis pilot sorted errors by type. Accidental omissions were the most frequent, in 18 percent of the reviewed notes ³. Hallucinations, meaning content the recording did not support, appeared in 11.5 percent ⁴.

The two kinds of error carry different risks. A dropped medication change or a missed symptom can alter the plan. An invented sentence can record a finding the patient never reported. A reviewer has to check the draft against the conversation, not only against the draft itself.

A sentence-by-sentence evaluation published in npj Digital Medicine reported a 1.47 percent hallucination rate for LLM-generated clinical notes ⁵. The authors evaluated their own configurations, and the paper's authors were employees of Tortus AI, which sells the system tested. Treat that rate as one team's result under its own prompts and workflows, not an independent benchmark of every scribe.

Products make different numbers of errors on the same audio

Accuracy varies between products. A Rochester Regional Health audit recorded simulated visits and gave identical recordings to three commercial scribes. The vendor with the fewest errors had 3, while the other two had 8 and 9 ⁶. The authors found that clinician preference did not track those audit counts, and they call the sample exploratory.

That audit used simulated recordings, which can differ from real rooms. Ask a vendor for results on audio like your own.

Dictation shows the value of review, though its errors are measured differently

Dictation has its own error profile. In a 2018 analysis of dictated clinical notes, the speech-recognition drafts had an error rate of 7.4 percent of words in clinical notes dictated in 2016 ⁷. In the notes physicians signed, the rate was 0.3 percent of words ⁸, after a medical transcriptionist had edited the drafts.

These dictation numbers are not directly comparable with scribe audits, since the notes were dictated and recognized years earlier. The lesson they share is that review by a person makes the biggest difference to what reaches the record. Error rates here are usually reported as word error rate, so check how each study defines it before you compare numbers.

It depends on how errors are counted and who checks the draft

• The definition. An error might be a wrong word, a missing finding or an invented one. Studies count these differently, so compare them only by type.
• The reviewer. Ask who checked each draft, and whether that person had the recording to compare against.
• The audio. Results depend on the recordings used, so a test on simulated visits may not match your clinic.
• The product. Results differ between named products and versions, and products change over time.

What to do: test accuracy on your own visits before you rely on a draft

1. Ask the vendor which product and version you are evaluating and how it measured errors.
2. Compare a sample of drafts with the recordings, with a reviewer who did not write the notes.
3. Count errors by type, and track the serious ones separately, since they matter most.
4. Check medications, doses, allergies and the plan on every draft before you sign.
5. Report errors to the vendor and keep a record of the review, so you can show what you checked.

This is general information about published studies, not clinical advice. For how the tools work, see what an ambient AI scribe is. Related questions are how much time doctors spend on documentation and whether AI scribes reduce physician burnout. For the wider subject, see the guide to AI medical scribes and the overview of dictation for work and study.