Timestamps and Speaker Labels in Transcripts

A timestamp tells you when a word or line starts in the recording. A speaker label tells you whose voice said it. Modern speech-to-text tools can add both automatically. Timestamps are the easier part to get right. Speaker labels are harder, because speaker diarization separates the voices and numbers them, but it cannot know who the people are. You supply the names, and you check the places where the software is most likely to be wrong.

This guide covers the timestamp styles you will meet, how diarization works and where it fails, and how to name and check the labels. Our overview of transcription covers the whole subject, including transcribing audio to text and meeting transcription.

Timestamps let a reader jump from any line back to the moment in the audio

A timestamp turns a transcript into an index of the recording. A reader who sees a quote can find the exact moment it was said and listen to it. Meeting platforms already work this way. Microsoft says Teams live transcription "includes each speaker's name and time stamp" ¹.

How often should a human transcriber add a stamp? Oral history practice gives one common answer. Oregon State University Libraries tells transcribers to "Add timestamps at five-minute intervals" ². Five minutes is a convention for long interviews, not a rule for every kind of transcript. Our question on how often to add timestamps covers when a shorter interval helps.

The format of a timestamp also matters when you export to a subtitle file. The two formats most people meet differ in one character.

A file with the wrong separator can fail to load in a player, so check the extension before you import. The SRT file and WebVTT entries in our glossary define both formats. Our guide to captions and subtitles covers where each format is used.

Timestamps come per word, per segment, at fixed intervals or at each speaker change

Timestamps come at four different grains, and the grain decides what you can do with them. A per-word timestamp places a single word. A per-segment timestamp places a sentence or chunk. A fixed interval places a stamp every few minutes. A speaker-change timestamp starts a new line each time the voice changes.

Per-word timing is the most precise, but it is not automatic. In the open-source Whisper command-line tool, the --word_timestamps option is marked experimental: the source describes it as "(experimental) extract word-level timestamps and refine the results based on them" ⁸. A 2024 Interspeech study found that carefully adjusting the tokenizer of a fine-tuned, more verbatim Whisper model "significantly improves the precision of word-level timestamps" ⁹. Plain Whisper output stamps utterances rather than words ⁷, so per-word precision is a setting you have to ask for.

Choose the grain by the job. A segment stamp is enough to find a passage you want to quote, because you only need to land near it. It is not enough to cut a clip or to line up a caption with a word, because a segment stamp can be inaccurate by several seconds ⁷, and that error is visible there. Speaker-change stamps suit reading a meeting, and per-word stamps suit editing from the audio. If you do not know which you need, ask for word-level timing: you can always show fewer stamps in the reading copy, but you cannot recover timing that the tool never produced.

Speaker diarization works out who spoke when, not who the speakers are

Diarization answers one question: when did each voice speak? It gives every voice a label such as "Speaker 1". Google Cloud says its speaker diarization "detects when speakers change and labels by number the individual voices detected in the audio" ¹⁰. The labels are arbitrary numbers. Nothing in the audio tells the software that Speaker 1 is a particular colleague.

Naming is a different task. In speaker identification, the IEEE Technology Navigator says, no identity is claimed. The system instead searches a database of known speakers and selects the closest match ¹¹. Naming a speaker therefore needs a set of voices that are already known, or a person who tells the software who is who. Diarization alone does not provide that.

Many diarization systems run in stages. The DKU-LENOVO system in the second DIHARD challenge was built from "voice activity detection (VAD), segmentation, speaker embedding extraction, similarity scoring, clustering, resegmentation and overlap detection" ¹². The diagram shows the path from audio to labels.

Each stage can make a mistake, and the mistakes compound. Open toolkits show how the stages are built. The pyannote.audio toolkit is "an open-source toolkit written in Python for speaker diarization" ¹³. The segmentation model in Bredin and Laurent's 2021 paper works on "short audio chunks (5 seconds) but at a much higher temporal resolution (every 16ms)" ¹⁴, which is why it can place a change in speaker more finely than a whole sentence.

Commercial products differ on whether they return labels at all. The table shows what each vendor's documentation says, as of the pages we read in October 2026.

The lesson is simple: a tool that does not return speaker labels cannot give you them, and a tool that does return them still gives numbers, not names. Check the product's documentation for the language you need, because support can vary. Our guide to transcription software explains how to compare these features before you buy.

Overlapping speech and short turns are where speaker labels fail

Speaker labels fail where voices overlap and where turns are short. The clearest measurement comes from the DIHARD challenges, shared benchmarks for diarization. In the second DIHARD challenge, the DKU-LENOVO team's system reached a diarization error rate (DER) of 18.84 percent on Track 1 and 27.90 percent on Track 2 ^{17 18}. These are the team's own reported scores. A lower DER is better, so the same system scored worse on Track 2.

Scores are only comparable when the scoring rules match, and papers differ. Most earlier diarization papers scored a modified DER that leaves out speech within 250 milliseconds of a speaker boundary and all overlapped speech ¹⁹. The second DIHARD challenge used DER "without use of collars or exclusion of overlapping speech" ²⁰. A lower DER in one paper may come from leaving out the hardest speech, so read the scoring method before you compare numbers.

Overlap is the hardest part to get right, and handling it well helps. On the AMI meeting corpus, which holds "100 hours of meeting recordings" ²¹, overlap-aware resegmentation of overlapped speech reduced the baseline's DER by 20 percent relative ²². A relative reduction of 20 percent means the error rate fell by a fifth of its old value. It does not mean twenty percentage points.

Conditions also matter. Diarization systems are reported to be more accurate "in balanced conversations with long turns and little turn overlap" ²³. Short turns are the weak point of that condition. In a study of question-and-answer turns in conversation across ten languages, the mean gap between one speaker finishing a turn and the next speaker starting was 208 milliseconds ²⁴. A short reply such as "yes" or "right" can fall into a gap like that, where a diarizer has very little audio to work with. This is an inference from those figures, so treat it as a reason to check, not as a measured error rate. The labels on short replies are the ones to check first.

Errors can be counted by the speaker label as well as by the words. The word diarization error rate counts a correctly recognized word as an error when it carries the wrong speaker label ²⁵. In a 2019 Google study on medical conversations, a joint speech recognition and diarization model improved word-level diarization error rate from 15.8 percent to 2.2 percent ²⁶.

The two charts use different metrics, so do not compare their bars. The second chart is one study's result on its own data, and it is not a general rate for every recording.

Name the speakers once, then check every label change against the audio

Naming and checking are the human steps, and they are where a transcript becomes trustworthy. The software will not do them for you.

1. Turn on speaker labels and timestamps together. Labels without timestamps are hard to check, because you cannot find the moment a voice changed.
2. Name each speaker once. Replace "Speaker 1" with the person's name everywhere, using the first time that voice speaks to confirm who it is. Editing this way is faster than correcting each line.
3. Check every label change. At each change, play the audio at the timestamp and confirm the voice. Labels are most often wrong at changes, so these are the places to spend your time.
4. Mark overlap you cannot separate. GoTranscript's style advice is to "Use [crosstalk mm:ss] if people talk at once and you can't separate their words" ²⁷. A marked overlap is more honest than a guessed speaker.
5. Recheck the short turns and the start. Short replies and the first minute, before the labels have settled, are the lines most likely to be wrong.

The editor Larry Jordan reported a similar limit in Premiere Pro. He wrote that "Premiere does not reliably identify speakers", so he planned to review the transcript by hand ²⁸. That was one editor's experience with a few videos, not a test, but the review step is the one that makes the transcript usable.

Accurate labels also matter for accessibility rules. The FCC rule for captioning television programming says offline captioning must be "a complete textual representation of the audio, including speaker identification and non-speech information" ²⁹. A captioned TV program with unlabeled speakers may not meet that rule. Our guide to captions and subtitles covers the requirements in more detail.

Word-level timestamps are what make captions and audio-linked editors possible

Captions need timing at the level of lines and cues, and an editor that plays audio as you read needs timing at the level of each word. Forced alignment is the technique that produces word-level timing: you give the system both the audio and the words, and it finds where each word falls. The 2017 Interspeech paper introduced the Montreal Forced Aligner as "a new open-source system for speech-text alignment" ³⁰. The forced alignment entry in our glossary defines the term.

WhisperX adds word-level timestamps to Whisper output, using "voice activity detection and forced phoneme alignment" ⁶. This is the approach that gives per-word timing from plain Whisper output, and it is why a per-word stamp is a separate step rather than a default.

Once the words carry timing, a caption file is the same lines with start and end times. The SRT and WebVTT formats in the table above store those times as numbered entries or cues, so the labels and times you checked carry through to the caption file. Check the first and last lines of each caption against the audio once more after export, because a file that looks correct in a text editor can still drift from the picture.

For a free start, Whisper's command-line tool runs on your own computer, and its word timestamps are an experimental option ⁸. Our question on transcribing audio with timestamps for free walks through the options. If you only need to transcribe a file, the free file transcription in the browser is the simplest place to begin. For several speakers, see how to transcribe audio with multiple speakers.

Timestamps and labels are cheap to add and expensive to fix later. Turn them on at the start, check the label changes against the audio, and keep the word-level timing for any file you will caption or edit.