System for enhancing speech understanding with effective and efficient integration of automated speech recognition error correction, out-of-domain detection, and/or domain classification
The systems and methods described herein are related to a new speech understanding system for domain-specific voice interaction. The systems and methods described herein combine the automatic correction of an automatic speech recognition errors with a natural language understanding model in a way that optimizes the recognition and understanding of a received speech input. The systems and methods described herein may further support out-of-domain detection or domain classification by jointly learning and/or performing automatic speech recognition error correction and domain-related classification. Through the joint learning with automatic speech recognition error correction, the out-of-domain detection or domain classification may be conducted based on a plurality of possible speech recognition results with shared feature inputs and shared neural layers. The systems and methods described herein may achieve robust performance with high computational efficiency.
1. A method for identifying a text string associated with a speech input, the method comprising:
receiving a speech input from a speech-based dialogue interface;
generating, by at least one speech recognition model, first and second text transcription predictions by converting the speech input into respective text strings;
generating a first confidence score for the first text transcription prediction and a second confidence score for the second text transcription prediction;
determining a slot type for respective text strings of the first text transcription prediction and the second text transcription prediction;
applying tags to the respective text strings of the first text transcription prediction and the second text transcription prediction based on the slot type for each respective text string, wherein each tag corresponds to one of the slot types or to an empty tag type that does not include a description;
generating at least one tagged text group by extracting at least one text string of the first text transcription prediction and grouping the at least one text string with at least one text string of the second text transcription prediction based on a tag associated with the text strings;
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to one of the slot types:
determining, using a natural language model, a relevance ranking of the text strings within the at least one tagged text group based on, at least, the slot type of the tag associated with the tagged text group, the first confidence score, and the second confidence score; and
identifying a text string of the at least one tagged text group having a highest relevance ranking, wherein the text string of the at least one tagged text group having the highest relevance ranking is provided to the speech-based dialogue interface; and
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to the empty tag type, discard the at least one tagged text group.
2. The method of claim 1 , wherein the natural language model is one of a class-based natural language understanding model and a word-based natural language understanding model.
3. The method of claim 2 , further comprising:
selecting the class-based natural language understanding model in response to a tag value of the tagged text group indicating an essential tag type; and
selecting the word-based natural language understanding model in response to a tag value of the tagged text group not indicating an essential tag type.
4. The method of claim 1 , further comprising:
applying multiple tags for a respective text string; and
associating a copy of the respective text string to the tagged text group associated with a respective tag.
5. The method of claim 1 , wherein the natural language model includes at least one layer configured to identify text that does not correspond to a tag type.
6. The method of claim 1 , wherein the text strings associated with a tagged text group include a similar linguistic usage.
7. A system for outputting, at a speech-based dialogue interface, a text string associated with a speech input, the system comprising:
a processor; and
a memory including instructions that, when executed by the processor, cause the processor to:
receive a speech input from a speech-based dialogue interface;
generate, by at least one speech recognition model, first and second text transcription predictions by converting the speech input into respective text strings;
generate a first confidence score for the first text transcription prediction and a second confidence score for the second text transcription prediction;
determine a slot type for respective text strings of the first text transcription prediction and the second text transcription prediction;
apply tags to the respective text strings of the first text transcription prediction and the second text transcription prediction based on the slot type for each respective text string, wherein each tag corresponds to one of the slot types or to an empty tag type that does not include a description;
generate at least one tagged text group by extracting at least one text string of the first text transcription prediction and grouping the at least one text string with at least one text string of the second text transcription prediction based on a tag associated with the text strings;
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to one of the slot types:
determine, using a natural language model, a relevance ranking of the text strings within the at least one tagged text group based on, at least, the slot type of the tag associated with the tagged text group, the first confidence score, and the second confidence score;
identify a text string of the at least one tagged text group having a highest relevance ranking; and
output the text string of the at least one tagged text group having the highest relevance ranking to the speech-based dialogue interface; and
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to the empty tag type, discard the at least one tagged text group.
8. The system of claim 7 , wherein the natural language model is one of a class-based natural language understanding model and a word-based natural language understanding model.
9. The system of claim 8 , wherein the instructions further cause the processor to:
select the class-based natural language understanding model in response to a tag value of the tagged text group indicating an essential tag type; and
select the word-based natural language understanding model in response to a tag value of the tagged text group not indicating an essential tag type.
10. The system of claim 7 , wherein at least one tag corresponds to an empty tag type that does not include a description.
11. The system of claim 10 , wherein a tagged text group related to the empty tag type is discarded.
12. The system of claim 7 , wherein the instructions further cause the processor to:
apply multiple tags for a respective text string; and
associate a copy of the respective text string to the tagged text group associated with a respective tag.
13. An apparatus for identifying a text string associated with a speech input, the apparatus comprising:
a processor; and
a memory including instructions that, when executed by the processor, cause the processor to:
receive a speech input from a speech-based dialogue interface captured by at least one input device;
generate, by at least one speech recognition model, first and second text transcription predictions by converting the speech input into respective text strings;
generate a first confidence score for the first text transcription prediction and a second confidence score for the second text transcription prediction;
determine a slot type for respective text strings of the first text transcription prediction and the second text transcription prediction;
apply tags to the respective text strings of the first text transcription prediction and the second text transcription prediction based on the slot type for each respective text string, wherein each tag corresponds to one of the slot types or to an empty tag type that does not include a description;
generate at least one tagged text group by extracting at least one text string of the first text transcription prediction and grouping the at least one text string with at least one text string of the second text transcription prediction based on a tag associated with the text strings;
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to one of the slot types:
determine, using a natural language model, a relevance ranking of the text strings within the at least one tagged text group based on, at least, the slot type of the tag associated with the tagged text group, the first confidence score, and the second confidence score; and
identify a text string of the at least one tagged text group having a highest relevance ranking; and
in response to a determination that the tag of the at least one text string of the first text transcription prediction and the tag of the at least one text string of the second text transcription prediction corresponds to the empty tag type, discard the at least one tagged text group.
14. The apparatus of claim 13 , wherein the natural language model is one of a class-based natural language understanding model and a word-based natural language understanding model.
15. The apparatus of claim 13 , wherein the at least one input device includes at least one microphone, and wherein the at least one input device is associated with at least one of a manufacturing machine, a power tool, an automated personal assistant, a domestic appliance, surveillance system, and a medical imaging system.