IP Library › Granted Patent US 11,462,208
Granted Patent B2
US 11,462,208 · App. 16/992,291 · Granted Oct 4, 2022

Implementing a correction model to reduce propagation of automatic speech recognition errors

Inventors: Thanh Long Duong (Seabrook, AU); Mark Edward Johnson (Castle Cove, AU)
Assignee: ORACLE INTERNATIONAL CORPORATION
G10L15/01G10L15/16G10L15/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,462,208
App. No.
16/992,291
Granted
Oct 4, 2022
Kind
B2
Abstract

Some techniques described herein determine a correction model for a dialog system, such that the correction model corrects output from an automatic speech recognition (ASR) subsystem in the dialog system. A method described herein includes accessing training data. A first tuple of the training data includes an utterance, where the utterance is a textual representation of speech. The method further includes using an ASR subsystem of a dialog system to convert the utterance to an output utterance. The method further includes storing the output utterance in corrective training data that is based on the training data. The method further includes training a correction model based on the corrective training data, such that the correction model is configured to correct output from the ASR subsystem during operation of the dialog system.

Claims (66)

1. A method comprising:

accessing training data comprising tuples, a first tuple of the tuples comprising an utterance, wherein the utterance is a textual representation of speech;

utilizing an automatic speech recognition (ASR) subsystem of a dialog system to convert the utterance to first output utterance comprising text of a first speech variant of the utterance;

utilizing the ASR subsystem of the dialog system to convert the utterance to a second output utterance comprising text of a second speech variant of the utterance;

storing the first output utterance and the second output utterance in corrective training data based on the training data; and

training a correction model based on the corrective training data to generate a trained correction model, the trained correction model configured to process output of the ASR subsystem of the dialog system.

2. The method of claim 1 , wherein utilizing the ASR subsystem of the dialog system to convert the utterance to the first output utterance comprises:

generating a speech comprising an audio version of the utterance in the first tuple; and

determining the first output utterance by applying the ASR subsystem to the speech.

3. The method of claim 2 , wherein utilizing the ASR subsystem of the dialog system to convert the utterance to the second output utterance comprises:

generating a second speech comprising a second audio version of the utterance in the first tuple; and

determining the second output utterance by applying the ASR subsystem to the second speech.

4. The method of claim 2 , wherein generating the speech comprises applying a text-to-speech subsystem to the utterance in the first tuple to determine the speech.

5. The method of claim 1 , wherein:

the first tuple further comprises a logical form corresponding to the utterance, wherein the logical form is a syntactic expression of the utterance according to an established grammar;

storing the first output utterance and the second output utterance in the corrective training data based on the training data comprises combining the first output utterance and the logical form into a corrective tuple of the corrective training data; and

the correction model is integrated with a corrective semantic parser configured to map utterances to logical forms.

6. The method of claim 5 , wherein storing the first output utterance and the second output utterance in the corrective training data based on the training data further comprises:

storing the second output utterance and the logical form of the first tuple into a second corrective tuple of the corrective training data.

7. The method of claim 1 , wherein storing the first output utterance and the second output utterance in the corrective training data based on the training data comprises:

combining the first output utterance and the utterance into a first corrective tuple of the corrective training data; and

combining the second output utterance and the utterance into a second corrective tuple of the corrective training data.

8. The method of claim 7 , further comprising including the correction model in the dialog system between the ASR subsystem and a semantic parser of the dialog system, the ASR subsystem configured to map speeches to utterances, and the semantic parser configured to map utterances to logical forms.

9. The method of claim 8 , further comprising:

receiving, by the dialog system, a first utterance output by the ASR subsystem;

providing, by the dialog system, the first utterance from the ASR subsystem to the correction model to determine a corrected utterance; and

inputting, by the dialog system, the corrected utterance to the semantic parser.

10. A system comprising:

a data-generation subsystem configured to:

access training data comprising tuples, a first tuple of the tuples comprising an utterance, wherein the utterance is a textual representation of speech;

generate a first audio version of the utterance, wherein the first audio version comprises a first speech variant of the utterance;

convert the first audio version of the utterance to a first output utterance;

generate a second audio version of the utterance, wherein the second audio version comprises a second speech variant of the utterance; and

convert the second audio version of the utterance to a second output utterance; and

a training subsystem configured to train a correction model, based on the the utterance, the first output utterance, and the second output utterance, to generate a trained correction model, the trained correction model configured to process output of an automatic speech recognition (ASR) subsystem of a dialog system.

11. The system of claim 10 , wherein, to generate the first audio version, the data-generation subsystem is further configured to apply a text-to-speech subsystem to the utterance in the first tuple to determine the first audio version.

12. The system of claim 10 , wherein:

the first tuple further comprises a logical form corresponding to the utterance, wherein the logical form is a syntactic expression of the utterance according to an established grammar; and

the data-generation subsystem is further configured to:

combine the first output utterance and the logical form into a first corrective tuple of the corrective training data; and

combine the second output utterance and the logical form into a second corrective tuple of the corrective training data.

13. The system of claim 12 , wherein the correction model is integrated with a corrective semantic parser configured to map utterances to logical forms of corrected versions of the utterances.

14. The system of claim 10 , wherein the data-generation subsystem is further configured to:

combine the first output utterance and the utterance into a first corrective tuple of the corrective training data; and

combine the second output utterance and the utterance into a second corrective tuple of the corrective training data.

15. The system of claim 14 , further comprising a dialog system comprising:

the ASR subsystem configured, as part of the dialog system, to map speeches to utterances, wherein the ASR subsystem is a first prediction model;

a semantic parser configured to map utterances to logical forms, wherein the semantic parser is a second prediction model; and

the correction model configured, as part of the dialog system, to:

receive a first utterance output by the ASR subsystem; and

generate a corrected utterance for use by the semantic parser.

16. The system of claim 10 , wherein the correction model is a sequence-to-sequence prediction model.

17. The system of claim 10 , wherein:

to convert the first audio version of the utterance to the first output utterance, the data-generation subsystem is configured to apply the ASR subsystem to the first audio version of the utterance; and

to convert the second audio version of the utterance to the second output utterance, the data-generation subsystem is configured to apply the ASR subsystem to the second audio version of the utterance.

18. A computer-program product for configuring a correction model to correct output from an automatic speech recognition (ASR) subsystem of a dialog system, the computer-program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions executable by one or more processors to cause the one or more processors to perform a method comprising:

accessing training data comprising tuples, a first tuple of the tuples comprising an utterance, wherein the utterance is a textual representation of speech;

generating a first output utterance comprising text of a first speech variant of the utterance;

generating a second output utterance comprising text of a second speech variant of the utterance, wherein the second output utterance differs from the first output utterance; and

storing the utterance and the first output utterance in a first corrective tuple of corrective training data based on the training data;

storing the utterance the second output utterance in a second corrective tuple of the corrective training data based on the training data; and

training the correction model based on the corrective training data to generate a trained correction model, the trained correction model configured to map utterances output by the ASR subsystem to corrected utterances.

19. The computer-program product of claim 18 , the method further comprising including the correction model in the dialog system between the ASR subsystem and a semantic parser of the dialog system, the ASR subsystem configured to map speeches to utterances, and the semantic parser configured to map utterances to logical forms.

20. The computer-program product of claim 19 , wherein:

the first tuple of the training data further comprises a logical form corresponding to the utterance, wherein the logical form is a syntactic expression of the utterance according to an established grammar; and

the method further comprises training the semantic parser based on the training data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2020
From: DUONG, THANH LONG; JOHNSON, MARK EDWARD
To: ORACLE INTERNATIONAL CORPORATION
Reel/Frame 053484/0179 →
Continuity (2)
Provisional Application 62898677 · Sep 11, 2019
Related Publication 20210074262A1 · Mar 11, 2021