System and method for unsupervised and active learning for automatic speech recognition
View Patent ↗A system and method is provided for combining active and unsupervised learning for automatic speech recognition. This process enables a reduction in the amount of human supervision required for training acoustic and language models and an increase in the performance given the transcribed and un-transcribed data.
1. A method for reducing the transcription effort for training an automatic speech recognition module, the method comprising:
(1) training acoustic and language models using a first set of transcribed data;
(2) recognizing utterances in a set of candidates for transcription using the acoustic and language models;
(3) computing by a processor confidence scores of the utterances;
(4) selecting k utterances that have the smallest confidence scores from the set of candidates and transcribing them into a first additional transcribed set;
(5) adding the first additional transcribed set to the first set of transcribed data to produce a second set of transcribed data;
(6) removing the first additional transcribed set from the set of candidates;
(7) retrieving a set of un-transcribed data from the set of candidates;
(8) training the acoustic and language models using the second set of transcribed data and speech recognition and word confidence scores for the retrieved set of un-transcribed data; and
(9) returning to step (1) if word accuracy has not converged.
2. The method of claim 1 , wherein k is more than one.
3. The method of claim 1 , wherein step (4) comprises leaving out utterances with confidence scores indicating that the utterances were correctly recognized.
4. The method of claim 1 , wherein word posterior probability estimates are used for word confidence scores associated with the utterances.
5. The method of claim 1 , wherein a word is considered to be correctly recognized if it has a confidence score higher than a threshold value.
6. The method of claim 1 , wherein step (7) comprises selecting a sample of un-transcribed data.
7. A tangible computer-readable medium that stores a program for controlling a computer device to perform a method to reduce the transcription effort for training an automatic speech recognition module, the method comprising:
(1) training acoustic and language models using a first set of transcribed data;
(2) recognizing utterances in a set of candidates for transcription using the acoustic and language models;
(3) computing confidence scores of the utterances;
(4) selecting k utterances that have the smallest confidence scores from the set of candidates and transcribing them into a first additional transcribed set;
(5) adding the first additional transcribed set to the first set of transcribed data to produce a second set of transcribed data;
(6) removing the first additional transcribed set from the set of candidates;
(7) retrieving a set of un-transcribed data from the set of candidates;
(8) training the acoustic and language models using the second set of transcribed data and speech recognition and word confidence scores for the retrieved set of un-transcribed data; and
(9) returning to step (1) if word accuracy has not converged.
8. A method for training an automatic speech recognition module, the method comprising:
(1) generating a first set of transcribed data;
(2) recognizing by a processor utterances in the first set of transcribed data;
(3) augmenting said first set of transcribed data with a second set of transcribed data that include those utterances whose speech recognition has low confidence scores;
(4) retrieving a set of un-transcribed data;
(5) recognizing by a processor utterances in the set of un-transcribed data; and
(6) training one of an acoustic model and language model using the augmented set of transcribed data and speech recognition scores and word confidence scores for the retrieved set of un-transcribed data.
9. The method of claim 8 , further comprising returning to step (1) if word accurance has not converged.
10. The method of claim 8 , wherein step (4) comprises selecting a sample of un-transcribed data.
11. The method of claim 8 , wherein step (6) comprises training both said acoustic model and said language model using the augmented set of transcribed data and the set of un-transcribed data.
12. A tangible computer-readable medium that stores a program for controlling a computer device to perform a method to train an automatic speech recognition module, the method comprising:
(1) generating a first set of transcribed data;
(2) recognizing by a processor utterances in the first set of transcribed data;
(3) augmenting said first set of transcribed data with a second set of transcribed data that include those utterances whose speech recognition has low confidence scores;
(4) retrieving a set of un-transcribed data;
(5) recognizing by a processor utterances in the set of un-transcribed data; and
(6) training one of an acoustic model and a language model using the augmented set of transcribed data and speech recognition scores and word confidence scores for the retrieved set of un-transcribed data.
13. An automatic-speech recognition module trained using a method of reducing the transcription effort for training an automatic speech recognition module and stored in a memory storage device, the method comprising:
(1) generating a first set of transcribed data;
(2) recognizing by a processor utterances in the first set of transcribed data;
(3) augmenting said first set of transcribed data with a second set of transcribed data that include those utterances whose speech recognition has low confidence scores;
(4) retrieving a set of un-transcribed data;
(5) recognizing by a processor utterances in the set of un-transcribed data; and
(6) training one of an acoustic model and language model using the augmented set of transcribed data and speech recognition scores and word confidence scores for the retrieved set of un-transcribed data.
14. A spoken dialog system, comprising:
an automatic-speech recognition module trained using a method of reducing the transcription effort for training an automatic speech recognition module and stored in a memory storage device, the method comprising:
(1) generating a first set of transcribed data;
(2) recognizing by a processor utterances in the first set of transcribed data;
(3) augmenting said first set of transcribed data with a second set of transcribed data that include those utterances whose speech recognition has low confidence scores;
(4) retrieving a set of un-transcribed data;
(5) recognizing by a processor utterances in the set of un-transcribed data; and
(6) training one of an acoustic model and language model using the augmented set of transcribed data and speech recognition scores and word confidence scores for the retrieved set of un-transcribed data.