Method and system for correcting speaker diarization using speaker change detection based on text
A method and system for correcting speaker diarization using a text-based speaker change detection. A speaker diarization correction method may include performing speaker diarization on an input audio stream; recognizing speech included in the input audio stream and converting the speech to text; detecting a speaker change based on the converted text; and correcting the speaker diarization based on the detected speaker change.
1. A speaker diarization correction method of a computer apparatus comprising at least one processor, the method, which uses the at least one processor, comprising:
performing speaker diarization on an input audio stream;
recognizing a speech included in the input audio stream and converting the speech to text;
detecting a speaker change based on the converted text; and
correcting the speaker diarization based on the detected speaker change,
wherein the detecting of the speaker change comprises:
receiving a speech recognition result for each utterance section, wherein each utterance section consists of at least one word unit, and further wherein each word unit comprises a single word of text;
encoding text included in the speech recognition result for each utterance section to one or more word units of text, wherein the encoding of the text to the one or more word units of text comprises encoding an EndPoint Detection (EPD) unit text included in the speech recognition result for each utterance section to the one or more word units of text using sentence Bidirectional Encoder Representations from Transformers (sBERT);
encoding each of the word units of text to consider a conversation context; and
determining whether a speaker change compared to a previous word unit of text is present for each word unit of text, individually, in which the conversation context is considered.
2. The method of claim 1 , wherein the detecting of the speaker change comprises recognizing a speaker change status for each word unit of text using a module that is trained to receive a speech recognition result for each utterance section and to output a speaker change probability of a word unit.
3. The method of claim 1 , wherein the encoding of the word unit of text to consider the conversation context comprises encoding the word unit of text to consider the conversation context using dialog Bidirectional Encoder Representations from Transformers (dBERT).
4. The method of claim 1 , wherein the correcting comprises correcting the speaker diarization based on the word unit depending on whether the speaker change is present for each word unit of text.
5. A non-transitory computer-readable record medium storing instructions that, when executed by a processor, cause the processor to perform the following method:
performing speaker diarization on an input audio stream;
recognizing a speech included in the input audio stream and converting the speech to text;
detecting a speaker change based on the converted text; and
correcting the speaker diarization based on the detected speaker change,
wherein the detecting of the speaker change comprises:
receiving a speech recognition result for each utterance section, wherein each utterance section consists of at least one word unit, and further wherein each word unit comprises a single word of text;
encoding text included in the speech recognition result for each utterance section to one or more word units of text, wherein the encoding of the text to the one or more word units of text comprises encoding an EndPoint Detection (EPD) unit text included in the speech recognition result for each utterance section to the one or more word units of text using sentence Bidirectional Encoder Representations from Transformers (sBERT);
encoding each of the word units of text to consider a conversation context; and
determining whether a speaker change compared to a previous word unit of text is present for each word unit of text, individually, in which the conversation context is considered.
6. A computer apparatus comprising:
at least one processor configured to execute computer-readable instructions,
wherein the at least one processor causes the computer apparatus to:
perform speaker diarization on an input audio stream,
recognize speech included in the input audio stream and convert the speech to text,
detect a speaker change based on the converted text, and
correct the speaker diarization based on the detected speaker change,
wherein the detecting of the speaker change comprises:
receiving a speech recognition result for each utterance section wherein each utterance section consists of at least one word unit, and further wherein each word unit comprises a single word of text;
encoding text included in the speech recognition result for each utterance section to one or more word units of text, wherein the encoding of the text to the one or more word units of text comprises encoding an EndPoint Detection (EPD) unit text included in the speech recognition result for each utterance section to the one or more word units of text using sentence Bidirectional Encoder Representations from Transformers (sBERT);
encoding each of the word units of text to consider a conversation context; and
determining whether a speaker change compared to a previous word unit of text is present for each word unit of text, individually, in which the conversation context is considered.
7. The computer apparatus of claim 6 , wherein, to detect the speaker change, the at least one processor causes the computer apparatus to recognize a speaker change status for each word unit of text using a module that is trained to receive a speech recognition result for each utterance section and to output a speaker change probability of a word unit.
8. The computer apparatus of claim 6 , wherein the encoding of the word unit of text to consider the conversation context comprises encoding the word unit of text to consider the conversation context using dialog Bidirectional Encoder Representations from Transformers (dBERT).