System and method for unsupervised and active learning for automatic speech recognition
View Patent ↗A system and method is provided for combining active and unsupervised learning for automatic speech recognition. This process enables a reduction in the amount of human supervision required for training acoustic and language models and an increase in the performance given the transcribed and un-transcribed data.
1. A method comprising:
identifying, in a database of utterances, first transcribed utterances and first un-transcribed utterances;
identifying transcription candidate utterances from the first un-transcribed utterances;
predicting, via a processor, likely recognition error utterances from the transcription candidate utterances;
forwarding the likely recognition error utterances to a transcriber, to yield additional transcribed utterances;
adding the additional transcribed utterances to the database of utterances, to yield an updated database of second transcribed utterances and second un-transcribed utterances; and
if word accuracy has converged, modifying an automatic speech recognition system by providing both the second transcribed utterances and the second un-transcribed utterances to the automatic speech recognition system.
2. The method of claim 1 , wherein identifying transcription candidate utterances further comprises using confidence scores of the un-transcribed utterances.
3. The method of claim 2 , the method further comprising:
determining the confidence scores using an acoustic model and a language model.
4. The method of claim 1 , wherein word posterior probability estimates are used for confidence scores associated with the database of utterances.
5. The method of claim 1 , wherein a word is considered to be correctly recognized if the word has a confidence score higher than a threshold value.
6. The method of claim 5 , wherein the confidence score is based on at least one of a scaling function and a geometric mean.
7. The method of claim 1 , further comprising:
upon adding the additional transcribed utterances to the database of utterances, removing the additional transcribed utterances from the first un-transcribed utterances to yield the second un-transcribed utterances.
8. A system comprising:
a processor;
a memory storing instructions for controlling the processor to perform steps comprising:
identifying, in a database of utterances, first transcribed utterances and first un-transcribed utterances;
identifying transcription candidate utterances from the first un-transcribed utterances;
predicting likely recognition error utterances from the transcription candidate utterances;
forwarding the likely recognition error utterances to a transcriber, to yield additional transcribed utterances;
adding the additional transcribed utterances to the database of utterances, to yield an updated database of second transcribed utterances and second un-transcribed utterances; and
if word accuracy has converged, modifying an automatic speech recognition system by providing both the second transcribed utterances and the second un-transcribed utterances to the automatic speech recognition system.
9. The system of claim 8 , wherein identifying transcription candidate utterances further comprises using confidence scores of the un-transcribed utterances.
10. The system of claim 9 , the steps further comprising:
determining the confidence scores using an acoustic model and a language model.
11. The system of claim 8 , wherein word posterior probability estimates are used for confidence scores associated with the database of utterances.
12. The system of claim 8 , wherein a word is considered to be correctly recognized if the word has a confidence score higher than a threshold value.
13. The system of claim 12 , wherein the confidence score is based on at least one of a scaling function and a geometric mean.
14. The system of claim 8 , the steps further comprising:
upon adding the additional transcribed utterances to the database of utterances, removing the additional transcribed utterances from the first un-transcribed utterances to yield the second un-transcribed utterances.
15. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform steps comprising:
identifying, in a database of utterances, first transcribed utterances and first un-transcribed utterances;
identifying transcription candidate utterances from the first un-transcribed utterances;
predicting likely recognition error utterances from the transcription candidate utterances;
forwarding the likely recognition error utterances to a transcriber, to yield additional transcribed utterances;
adding the additional transcribed utterances to the database of utterances, to yield an updated database of second transcribed utterances and second un-transcribed utterances; and
if word accuracy has converged, modifying an automatic speech recognition system by providing both the second transcribed utterances and the second un-transcribed utterances to the automatic speech recognition system.
16. The non-transitory computer-readable storage medium of claim 15 , wherein identifying transcription candidate utterances further comprises using confidence scores of the un-transcribed utterances.
17. The non-transitory computer-readable storage medium of claim 16 , the steps further comprising:
determining the confidence scores using an acoustic model and a language model.
18. The non-transitory computer-readable storage medium of claim 15 , wherein word posterior probability estimates are used for confidence scores associated with the database of utterances.
19. The non-transitory computer-readable storage medium of claim 15 , wherein a word is considered to be correctly recognized if the word has a confidence score higher than a threshold value.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the confidence score is based on at least one of a scaling function and a geometric mean.