System and method for unsupervised and active learning for automatic speech recognition
View Patent ↗A system and method is provided for combining active and unsupervised learning for automatic speech recognition. This process enables a reduction in the amount of human supervision required for training acoustic and language models and an increase in the performance given the transcribed and un-transcribed data.
1. A computer-implemented method performed by a processor, the method comprising:
identifying a plurality of un-transcribed utterances and acoustic and language models trained to recognize speech;
for each un-transcribed utterance of the plurality of the un-transcribed utterances, computing a corresponding confidence score that indicates a likelihood of the un-transcribed utterance being correctly recognized by the acoustic and language models;
identifying a first set of utterances selected from the un-transcribed utterances, each utterance in the first set of utterances having a corresponding confidence score indicating at least a first likelihood of the un-transcribed utterance being correctly recognized;
transcribing, via the processor and the acoustic and language models, the first set of utterances to generate machine-transcribed utterances
identifying a second set of utterances selected from the un-transcribed utterances, each utterance in the second set of utterances having a corresponding confidence score indicating a likelihood of the un-transcribed utterance being correctly recognized that is lower than the first likelihood;
receiving human-transcribed utterances for the second set of utterances; and
re-training the acoustic and language models using the machine-transcribed utterances and the human-transcribed utterances.
2. The method of claim 1 , further comprising continuing the identifying, the transcribing, and the re-training until a word error rate has converged.
3. The method of claim 1 , further comprising:
adding the machine-transcribed utterances and the human-transcribed utterances to the database of utterances; and
upon adding the additional transcribed utterances to the database of utterances, removing the additional utterances from the un-transcribed utterances.
4. The method of claim 3 , wherein the adding of the machine-transcribed utterances and the human-transcribed utterances to the database of utterances is used to create an automatic speech recognition module.
5. The method of claim 1 , wherein the confidence scores of the un-transcribed utterances are associated with an arithmetic mean of confidences scores of words contained within each transcription candidate utterance.
6. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, result in the processor performing operations comprising:
identifying a plurality of un-transcribed utterances and acoustic and language models trained to recognize speech;
for each un-transcribed utterance of the plurality of the un-transcribed utterances, computing a corresponding confidence score that indicates a likelihood of the un-transcribed utterance being correctly recognized by the acoustic and language models;
identifying a first set of utterances selected from the un-transcribed utterances, each utterance in the first set of utterances having a corresponding confidence score indicating at least a first likelihood of the un-transcribed utterance being correctly recognized;
transcribing, via the processor and the acoustic and language models, the first set of utterances to generate machine-transcribed utterances
identifying a second set of utterances selected from the un-transcribed utterances, each utterance in the second set of utterances having a corresponding confidence score indicating a likelihood of the un-transcribed utterance being correctly recognized that is lower than the first likelihood;
receiving human-transcribed utterances for the second set of utterances; and
re-training the acoustic and language models using the machine-transcribed utterances and the human-transcribed utterances.
7. The system of claim 6 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising continuing the identifying, the transcribing, and the re-training until a word error rate has converged.
8. The system of claim 6 , the computer-readable storage medium having additional instructions stored which, when executed by the processor, result in operations comprising:
adding the machine-transcribed utterances and the human-transcribed utterances to the database of utterances; and
upon adding the additional transcribed utterances to the database of utterances, removing the additional utterances from the un-transcribed utterances.
9. The system of claim 8 , wherein the adding of the machine-transcribed utterances and the human-transcribed utterances to the database of utterances is used to create an automatic speech recognition module.
10. The system of claim 6 , wherein the confidence scores of the un-transcribed utterances are associated with an arithmetic mean of confidences scores of words contained within each transcription candidate utterance.
11. A computer-readable storage device having instructions stored which, when executed by a computing device, result in the computing device performing operations comprising:
identifying a plurality of un-transcribed utterances and acoustic and language models trained to recognize speech;
for each un-transcribed utterance of the plurality of the un-transcribed utterances, computing a corresponding confidence score that indicates a likelihood of the un-transcribed utterance being correctly recognized by the acoustic and language models;
identifying a first set of utterances selected from the un-transcribed utterances, each utterance in the first set of utterances having a corresponding confidence score indicating at least a first likelihood of the un-transcribed utterance being correctly recognized;
transcribing, via the processor and the acoustic and language models, the first set of utterances to generate machine-transcribed utterances
identifying a second set of utterances selected from the un-transcribed utterances, each utterance in the second set of utterances having a corresponding confidence score indicating a likelihood of the un-transcribed utterance being correctly recognized that is lower than the first likelihood;
receiving human-transcribed utterances for the second set of utterances; and
re-training the acoustic and language models using the machine-transcribed utterances and the human-transcribed utterances.
12. The computer-readable storage device of claim 11 , having additional instructions stored which, when executed by the processor, result in operations comprising continuing the identifying, the transcribing, and the re-training until a word error rate has converged.
13. The computer-readable storage device of claim 11 , having additional instructions stored which, when executed by the processor, result in operations comprising:
adding the machine-transcribed utterances and the human-transcribed utterances to the database of utterances; and
upon adding the additional transcribed utterances to the database of utterances, removing the additional utterances from the un-transcribed utterances.
14. The computer-readable storage device of claim 11 , wherein the confidence scores of the un-transcribed utterances are associated with an arithmetic mean of confidences scores of words contained within each transcription candidate utterance.