Document transcription system training
View Patent ↗A system is provided for training an acoustic model for use in speech recognition. In particular, such a system may be used to perform training based on a spoken audio stream and a non-literal transcript of the spoken audio stream. Such a system my identify text in the non-literal transcript which represents concepts having multiple spoken forms. The system may attempt to identify the actual spoken form in the audio stream which produced the corresponding text in the non-literal transcript, and thereby produce a revised transcript which more accurately represents the spoken audio stream. The revised, and more accurate, transcript may be used to train the acoustic model, thereby producing a better acoustic model than that which would be produced using conventional techniques, which perform training based directly on the original non-literal transcript.
1. A method performed by a computer processor executing computer program instructions stored on a non-transitory computer-readable medium, the method for use with a system, the system including a first document, the first document containing at least some information in common with a spoken audio stream, the method comprising:
(A) identifying text in the first document, wherein the text represents a concept;
(B) identifying, based on the identified text and a repository of finite state grammars, a plurality of spoken forms of the concept, including at least one spoken form not contained in the first document, wherein all of the plurality of spoken forms have the same content as each other;
(C) replacing the identified text with a finite state grammar specifying the plurality of spoken forms of the concept to produce a second document;
(D) generating a document-specific language model based on the second document, comprising generating at least some of the document-specific language model based on the finite state grammar.
2. The method of claim 1 , further comprising:
(E) using the document-specific language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a third document.
3. The method of claim 2 , further comprising:
(F) using the third document and the spoken audio stream to train an acoustic model.
4. The method of claim 3 , wherein (F) comprises:
(F)(1) filtering text from the third document by reference to the second document to produce a filtered document; and
(F)(2) using the filtered document and the spoken audio stream to train the acoustic model.
5. The method of claim 2 , wherein (E) comprises:
(E)(1) interpolating the first language model with a second language model to produce a third language model; and
(E)(2) using the third language model in the speech recognition process to recognize the spoken audio stream and thereby to produce the third document.
6. The method of claim 1 , further comprising:
(F) before (A), generating the first document based on the spoken audio stream.
7. The method of claim 1 , wherein the concept comprises a semantic concept.
8. The method of claim 1 , wherein the concept comprises a syntactic concept.
9. The method of claim 1 , further comprising:
(E) before (D), normalizing the second document to produce a normalized document.
10. The method of claim 1 , wherein (C) comprises:
(C)(1) generating probabilities for the plurality of spoken forms specified by the finite state grammar;
(C)(2) including the probabilities in the finite state grammar; and
(C)(3) including the plurality of spoken forms in the finite state grammar.
11. A non-transitory computer-readable medium comprising computer program instructions executable to perform a method for use with a system, the system including a first document, the first document containing at least some information in common with a spoken audio stream, the method comprising:
(A) identifying text in the first document, wherein the text represents a concept;
(B) identifying, based on the identified text and a repository of finite state grammars, a plurality of spoken forms of the concept, including at least one spoken form not contained in the first document, wherein all of the plurality of spoken forms have the same content as each other;
(C) replacing the identified text with a finite state grammar specifying the plurality of spoken forms of the concept to produce a second document;
(D) generating a document-specific language model based on the second document, comprising generating at least some of the document-specific language model based on the finite state grammar.
12. The non-transitory computer-readable medium of claim 11 , wherein the method further comprises:
(E) using the document-specific language model in a speech recognition process to recognize the spoken audio stream and thereby to produce a third document.
13. The non-transitory computer-readable medium of claim 12 , wherein the method further comprises:
(F) using the third document and the spoken audio stream to train an acoustic model.
14. The non-transitory computer-readable medium of claim 13 , wherein (F) comprises:
(F)(1) filtering text from the third document by reference to the second document to produce a filtered document; and
(F)(2) using the filtered document and the spoken audio stream to train the acoustic model.
15. The non-transitory computer-readable medium of claim 12 , wherein (E) comprises:
(E)(1) interpolating the first language model with a second language model to produce a third language model; and
(E)(2) using the third language model in the speech recognition process to recognize the spoken audio stream and thereby to produce the third document.
16. The non-transitory computer-readable medium of claim 11 , wherein the method further comprises:
(F) before (A), generating the first document based on the spoken audio stream.
17. The non-transitory computer-readable medium of claim 11 , wherein the concept comprises a semantic concept.
18. The non-transitory computer-readable medium of claim 11 , wherein the concept comprises a syntactic concept.
19. The non-transitory computer-readable medium of claim 11 , wherein the method further comprises:
(E) before (D), normalizing the second document to produce a normalized document.
20. The non-transitory computer-readable medium of claim 11 , wherein (C) comprises:
(C)(1) generating probabilities for the plurality of spoken forms specified by the finite state grammar;
(C)(2) including the probabilities in the finite state grammar; and
(C)(3) including the plurality of spoken forms in the finite state grammar.