Domain adaptive speech recognition using artificial intelligence
View Patent ↗Methods, systems, and computer program products for domain adaptive speech recognition using artificial intelligence are provided herein. A computer-implemented method includes generating a set of language data candidates, each language data candidate comprising one or more graphemes, by processing a sequence of phonemes related to input speech data using an artificial intelligence-based data conversion model; determining, for a target pair of phonemes and graphemes, a subset of graphemes from the set of language data candidates; generating a first speech recognition output by processing the subset of graphemes using at least one biasing language model and an artificial intelligence-based speech recognition model; generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the graphemes from the target pair; and performing automated actions based on the second speech recognition output.
1. A computer-implemented method comprising:
generating a set of language data candidates, each language data candidate comprising one or more graphemes, by processing a sequence of phonemes related to user-provided domain-specific input speech data using an artificial intelligence-based data conversion model comprising a neural network model sharing a prediction network from a recurrent neural network transducer;
determining, for a target pair of one or more phonemes and one or more graphemes, a subset of graphemes from the set of language data candidates;
training at least one biasing language model using at least a portion of the subset of graphemes;
generating a first speech recognition output by processing the at least a portion of the subset of graphemes using the at least one biasing language model and an artificial intelligence-based speech recognition model comprising the recurrent neural network transducer, including the prediction network shared by the artificial intelligence-based data conversion model;
generating a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and
performing one or more automated actions based at least in part on the second speech recognition output;
wherein the method is carried out by at least one computing device.
2. The computer-implemented method of claim 1 , wherein the artificial intelligence-based data conversion model comprises at least one encoder-decoder neural network.
3. The computer-implemented method of claim 1 , wherein the artificial intelligence-based speech recognition model comprises at least one end-to-end automatic speech recognition model.
4. The computer-implemented method of claim 1 , further comprising:
training the artificial intelligence-based data conversion model using one or more graphemes-phonemes pairs.
5. The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises automatically training at least one of the artificial intelligence-based data conversion model and the artificial intelligence-based speech recognition model using feedback data related to the second speech recognition output.
6. The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises outputting, to at least one user, the second speech recognition output in response to the user-provided domain-specific input speech data.
7. The computer-implemented method of claim 1 , wherein the user-provided domain-specific input speech data comprise one or more of one or more individual words, one or more sentences, and one or more compound words.
8. The computer-implemented method of claim 1 , wherein software implementing the method is provided as a service in a cloud environment.
9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:
generate a set of language data candidates, each language data candidate comprising one or more graphemes, by processing a sequence of phonemes related to user-provided domain-specific input speech data using an artificial intelligence-based data conversion model comprising a neural network model sharing a prediction network from a recurrent neural network transducer;
determine, for a target pair of one or more phonemes and one or more graphemes, a subset of graphemes from the set of language data candidates;
train at least one biasing language model using at least a portion of the subset of graphemes;
generate a first speech recognition output by processing the at least a portion of the subset of graphemes using the at least one biasing language model and an artificial intelligence-based speech recognition model comprising the recurrent neural network transducer, including the prediction network shared by the artificial intelligence-based data conversion model;
generate a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and
perform one or more automated actions based at least in part on the second speech recognition output.
10. The computer program product of claim 9 , wherein the artificial intelligence-based data conversion model comprises at least one encoder-decoder neural network.
11. The computer program product of claim 9 , wherein the artificial intelligence-based speech recognition model comprises at least one end-to-end automatic speech recognition model.
12. The computer program product of claim 9 , wherein performing one or more automated actions comprises outputting, to at least one user, the second speech recognition output in response to the user-provided domain-specific input speech data.
13. The computer program product of claim 9 , wherein the program instructions executable by the computing device further cause the computing device to:
train the artificial intelligence-based data conversion model using one or more graphemes- phonemes pairs.
14. The computer program product of claim 9 , wherein performing one or more automated actions comprises automatically training at least one of the artificial intelligence-based data conversion model and the artificial intelligence-based speech recognition model using feedback data related to the second speech recognition output.
15. A system comprising:
a memory configured to store program instructions; and
a processor operatively coupled to the memory to execute the program instructions to:
generate a set of language data candidates, each language data candidate comprising one or more graphemes, by processing a sequence of phonemes related to user-provided domain-specific input speech data using an artificial intelligence-based data conversion model comprising a neural network model sharing a prediction network from a recurrent neural network transducer;
determine, for a target pair of one or more phonemes and one or more graphemes, a subset of graphemes from the set of language data candidates;
train at least one biasing language model using at least a portion of the subset of graphemes;
generate a first speech recognition output by processing the at least a portion of the subset of graphemes using the at least one biasing language model and an artificial intelligence-based speech recognition model comprising the recurrent neural network transducer, including the prediction network shared by the artificial intelligence-based data conversion model;
generate a second speech recognition output by replacing at least a portion of the subset of graphemes in the first speech recognition output with at least one of the one or more graphemes from the target pair; and
perform one or more automated actions based at least in part on the second speech recognition output.
16. The system of claim 15 , wherein the artificial intelligence-based data conversion model comprises at least one encoder-decoder neural network.
17. The system of claim 15 , wherein performing one or more automated actions comprises outputting, to at least one user, the second speech recognition output in response to the user-provided domain-specific input speech data.
18. The system of claim 15 , wherein the artificial intelligence-based speech recognition model comprises at least one end-to-end automatic speech recognition model.
19. The system of claim 15 , wherein the processor is operatively coupled to the memory to further execute the program instructions to:
train the artificial intelligence-based data conversion model using one or more graphemes-phonemes pairs.
20. The system of claim 15 , wherein performing one or more automated actions comprises automatically training at least one of the artificial intelligence-based data conversion model and the artificial intelligence-based speech recognition model using feedback data related to the second speech recognition output.