Speaker diarization

A diarization pipeline segments the audio at speaker-change points, computes an embedding for each segment that characterises the voice, then clusters those embeddings so segments from the same person share a label. The output is speaker-relative — "speaker 1" and "speaker 2" — not identity; matching a label to a named person is speaker identification, a separate task with very different privacy implications.

The reliably hard cases are overlapping speech, short turns, and voices with similar pitch and timbre. Crosstalk during an interruption often gets attributed wholesale to one speaker, and single-word turns such as "yes" carry too little signal to embed confidently — which is unfortunate, because those are precisely the turns where consent and confirmation live.

Telephony offers a shortcut worth taking when it is available. If each party is recorded on a separate channel, speaker attribution is exact and free, with no clustering step and no error. Diarization is only necessary when the recording has been mixed down to a single mono channel, which is why recording configuration is worth deciding before the first call rather than after.