Automatic grammar augmentation for robust voice command recognition
Various embodiments include methods and devices for implementing automatic grammar augmentation for improving voice command recognition accuracy in systems with a small footprint acoustic model. Alternative expressions that may capture acoustic model decoding variations may be added to a grammar set. An acoustic model-specific statistical pronunciation dictionary may be derived by running the acoustic model through a large general speech dataset and constructing a command-specific candidate set containing potential grammar expressions. Greedy based and cross-entropy-method (CEM) based algorithms may be utilized to search the candidate set for augmentations with improved recognition accuracy.
1. A method of improving voice command recognition, comprising:
applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary, wherein the statistical pronunciation dictionary comprises variations of each utterance in the general speech dataset with corresponding maximum probability decoding;
generating an augmented grammar candidate set based on the statistical pronunciation dictionary and an original grammar set, wherein:
the original grammar set comprises predefined voice commands to be recognized; and
each element of the augmented grammar candidate set comprises a variation of one of the predefined voice commands to be recognized, wherein the variation is obtained by replacing one of the predefined voice commands, from the original grammar set, by corresponding variations based on the maximum probability decoding in the statistical pronunciation dictionary; and
generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set.
2. The method of claim 1 , wherein applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining a greedy decoding sequence for each utterance in the general speech dataset;
calculating a minimum-edit-path of a corresponding ground-truth to the greedy decoding sequence; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
3. The method of claim 1 , wherein applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining one or more sequences from beam-searching algorithms;
calculating a minimum-edit-path of a corresponding ground-truth to the obtained one or more sequences; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
4. The method of claim 1 , further comprising determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprising the variation of one of the predefined voice commands to be recognized.
5. The method of claim 4 , wherein determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprises:
determining the command recognition accuracy for each element of the augmented grammar candidate set based on a command-specific data set, the command-specific data set comprising, for each element of the augmented grammar candidate set, an audio waveform and a corresponding target command; and
determining the false-alarm rate for each element of the augmented grammar candidate set based on an out-of-domain data set comprising a set of utterances that do not correspond to any one of the predefined voice commands to be recognized.
6. The method of claim 1 , wherein generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a method selected from one of:
a naïve greedy search;
a greedy search with refinement; or
a beam-search, and
wherein the selected one or more elements of the augmented grammar candidate set has a particular weighted sum of a mis-detection rate and a mis-classification rate with a fixed false-alarm target.
7. The method of claim 1 , wherein generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a cross entropy method.
8. A computing device, comprising:
a memory; and
a processor coupled to the memory and configured with processor executable instructions to perform operations comprising:
applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary, wherein the statistical pronunciation dictionary comprises variations of each utterance in the general speech dataset with corresponding maximum probability decoding;
generating an augmented grammar candidate set based on the statistical pronunciation dictionary and an original grammar set, wherein:
the original grammar set comprises predefined voice commands to be recognized; and
each element of the augmented grammar candidate set comprises a variation of one of the predefined voice commands to be recognized, wherein the variation is obtained by replacing one of the predefined voice commands, from the original grammar set, by corresponding variations based on the maximum probability decoding in the statistical pronunciation dictionary; and
generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set.
9. The computing device of claim 8 , wherein the processor is configured with processor-executable instructions to perform operations such that applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining a greedy decoding sequence for each utterance in the general speech dataset;
calculating a minimum-edit-path of a corresponding ground-truth to the greedy decoding sequence; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
10. The computing device of claim 8 , wherein the processor is configured with processor-executable instructions to perform operations such that applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining one or more sequences from beam-searching algorithms;
calculating a minimum-edit-path of a corresponding ground-truth to the obtained one or more sequences; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
11. The computing device of claim 8 , wherein the processor is configured with processor-executable instructions to perform operations further comprising determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprising the variation of one of the predefined voice commands to be recognized.
12. The computing device of claim 11 , wherein the processor is configured with processor-executable instructions to perform operations such that determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprises:
determining the command recognition accuracy for each element of the augmented grammar candidate set based on a command-specific data set, the command-specific data set comprising, for each element of the augmented grammar candidate set, an audio waveform and a corresponding target command; and
determining the false-alarm rate for each element of the augmented grammar candidate set based on an out-of-domain data set comprising a set of utterances that do not correspond to any one of the predefined voice commands to be recognized.
13. The computing device of claim 8 , wherein the processor is configured with processor-executable instructions to perform operations such that generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a method selected from one of:
a naïve greedy search;
a greedy search with refinement; or
a beam-search, and
wherein the selected one or more elements of the augmented grammar candidate set has a particular weighted sum of a mis-detection rate and a mis-classification rate with a fixed false-alarm target.
14. The computing device of claim 8 , wherein the processor is configured with processor-executable instructions to perform operations such that generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a cross entropy method.
15. A computing device, comprising:
a memory;
means for applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary, wherein the statistical pronunciation dictionary comprises variations of each utterance in the general speech dataset with corresponding maximum probability decoding;
means for generating an augmented grammar candidate set based on the statistical pronunciation dictionary and an original grammar set, wherein:
the original grammar set comprises predefined voice commands to be recognized; and
each element of the augmented grammar candidate set comprises a variation of one of the predefined voice commands to be recognized, wherein the variation is obtained by replacing one of the predefined voice commands, from the original grammar set, by corresponding variations based on the maximum probability decoding in the statistical pronunciation dictionary; and
means for generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set.
16. A non-transitory processor-readable medium having stored thereon processor-executable instructions configured to cause a processor to perform operations comprising:
applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary, wherein the statistical pronunciation dictionary comprises variations of each utterance in the general speech dataset with corresponding maximum probability decoding;
generating an augmented grammar candidate set based on the statistical pronunciation dictionary and an original grammar set, wherein:
the original grammar set comprises predefined voice commands to be recognized; and
each element of the augmented grammar candidate set comprises a variation of one of the predefined voice commands to be recognized, wherein the variation is obtained by replacing one of the predefined voice commands, from the original grammar set, by corresponding variations based on the maximum probability decoding in the statistical pronunciation dictionary; and
generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set.
17. The non-transitory processor-readable medium of claim 16 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations such that applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining a greedy decoding sequence for each utterance in the general speech dataset;
calculating a minimum-edit-path of a corresponding ground-truth to the greedy decoding sequence; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
18. The non-transitory processor-readable medium of claim 16 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations such that applying an acoustic model to a general speech dataset to generate a statistical pronunciation dictionary comprises:
obtaining one or more sequences from beam-searching algorithms;
calculating a minimum-edit-path of a corresponding ground-truth to the obtained one or more sequences; and
obtaining a mapping of each utterance to the corresponding maximum probability decoding.
19. The non-transitory processor-readable medium of claim 16 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprising the variation of one of the predefined voice commands to be recognized.
20. The non-transitory processor-readable medium of claim 19 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations such that determining a command recognition accuracy, a false-alarm rate, and a mis-detection rate for each element of a given augmented grammar candidate set comprises:
determining the command recognition accuracy for each element of the augmented grammar candidate set based on a command-specific data set, the command-specific data set comprising, for each element of the augmented grammar candidate set, an audio waveform and a corresponding target command; and
determining the false-alarm rate for each element of the augmented grammar candidate set based on an out-of-domain data set comprising a set of utterances that do not correspond to any one of the predefined voice commands to be recognized.
21. The non-transitory processor-readable medium of claim 16 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations such that generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a method selected from one of:
a naïve greedy search;
a greedy search with refinement; or
a beam-search, and
wherein the selected one or more elements of the augmented grammar candidate set has a particular weighted sum of a mis-detection rate and a mis-classification rate with a fixed false-alarm target.
22. The non-transitory processor-readable medium of claim 16 , wherein the stored processor-executable instructions are configured to cause a processor to perform operations such that generating an augmented grammar set by adding one or more elements of the augmented grammar candidate set to the original grammar set comprises selecting the one or more elements of the augmented grammar candidate set utilizing a cross entropy method.