IP Library Granted Patent US 11,482,213
Granted Patent B2
US 11,482,213 · App. 16/261,511 · Granted Oct 25, 2022

Automatic speech recognition correction

Inventors: Karthik Raghunathan (Sunnyvale, CA); Arushi Raghuvanshi (San Francisco, CA); Vijay Ramakrishnan Thimmaiyah (San Francisco, CA); Lucien Serapio Carroll (San Leandro, CA); Varsha Ravikumar Embar (Bangalore, IN)
Assignee: CISCO TECHNOLOGY, INC.
G10L15/187G10L15/02G10L15/063G10L15/22G10L2015/025
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,482,213
App. No.
16/261,511
Granted
Oct 25, 2022
Kind
B2
Abstract

Systems, methods, and computer-readable media for correcting transcriptions created through automatic speech recognition. A transcription of speech created using an automatic speech recognition system can be received. One or more domain-specific contexts associated with the speech can be identified and a text span that includes a mistranscribed entry can be recognized from the speech based on the one or more domain-specific contexts. Additionally, features can be extracted from the mistranscribed entry and the extracted features can be matched against an index of domain-specific entries to identify a correct entry of the mistranscribed entry. Subsequently, the transcription can be corrected by replacing with the mistranscribed entry with the correct entry.

Claims (46)

1. A method comprising:

receiving a transcription of speech created using an automatic speech recognition system;

identifying one or more domain-specific contexts associated with the speech;

detecting a text span from the transcription that includes a mistranscribed entry of the speech based on a user-specific context included in the one or more domain-specific contexts associated with the speech;

extracting features from the mistranscribed entry by applying both a user-specific extractor associated with the user-specific context and a domain-specific sequence to sequence grapheme-to-phoneme model, wherein the features include one or more user-specific contextual features of the mistranscribed entry associated with the user-specific context and one or more phonetic features extracted through application of the sequence to sequence grapheme-to-phoneme model;

matching the extracted features against an index of domain-specific entries to identify a correct entry of the mistranscribed entry, based at least in part on both a user-specific contextual feature of the mistranscribed entry and a phonetic feature; and

correcting the transcription by replacing the mistranscribed entry with the correct entry.

2. The method of claim 1 , wherein the text span is detected by applying one or more domain-specific extractors trained based on text spans in the one or more domain-specific contexts and corresponding entries of the text spans in the one or more domain-specific contexts to the transcription.

3. The method of claim 2 , further comprising applying the one or more domain-specific extractors to the text span to identify the mistranscribed entry of the speech from the text span.

4. The method of claim 1 , wherein the mistranscribed entry is matched to the correct entry based on the one or more phonetic features of the mistranscribed entry and phonetic features of the correct entry using term frequency-inverse document frequency-based information retrieval.

5. The method of claim 1 , wherein the domain-specific grapheme-to-phoneme model is trained using phonetic features of the correct entry.

6. The method of claim 1 , further comprising:

identifying textual features of the mistranscribed entry, as part of extracting the features from the mistranscribed entry, based on the one or more domain-specific contexts by applying a domain-specific textual model to the mistranscribed entry; and

matching the mistranscribed entry to the correct entry of the mistranscribed entry based on similarities between the textual features of the mistranscribed entry and textual features of the correct entry in the index of domain-specific entries.

7. The method of claim 6 , wherein the textual features include one or a combination of character n-grams, word n-grams, edge n-grams, and normalized keywords identified based on the one or more domain-specific contexts by applying the domain-specific textual model to the mistranscribed entry.

8. The method of claim 6 , wherein the mistranscribed entry is matched to the correct entry based on the textual features of the mistranscribed entry and the textual features of the correct entry using term frequency-inverse document frequency-based information retrieval.

9. The method of claim 1 , further comprising:

identifying contextual features of the mistranscribed entry, as part of extracting the features from the mistranscribed entry, based on the one or more domain-specific contexts by applying a domain-specific contextual model to the mistranscribed entry; and

matching the mistranscribed entry to the correct entry based on similarities between the contextual features of the mistranscribed entry and contextual features of the correct entry in the index of domain-specific entries.

10. The method of claim 9 , wherein the contextual features of the mistranscribed entry include contextual features of a speaker who uttered the text span of the speech that includes the mistranscribed entry.

11. The method of claim 9 , wherein the mistranscribed entry is matched to the correct entry based on the contextual features of the mistranscribed entry and the contextual features of the correct entry using term frequency-inverse document frequency-based information retrieval.

12. The method of claim 1 , wherein the index of domain-specific entries is selected based on the one or more domain-specific contexts associated with the speech.

13. A system comprising:

one or more processors; and

at least one computer-readable storage medium having stored therein instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving a transcription of speech created using an automatic speech recognition system;

identifying one or more domain-specific contexts associated with the speech;

applying one or more domain-specific extractors to the transcription to detect a text span from the transcription that includes a mistranscribed entry of the speech based on a user-specific context included in the one or more domain-specific contexts associated with the speech;

extracting features from the mistranscribed entry by applying a user-specific extractor associated with the user-specific context and a domain-specific sequence to sequence grapheme-to-phoneme model, wherein the features include one or more user-specific contextual features of the mistranscribed entry associated with the user-specific context and one or more phonetic features extracted through application of the sequence to sequence grapheme-to-phoneme model;

matching the extracted features against an index of domain-specific entries to identify a correct entry of the mistranscribed entry, based at least in part on both a user-specific contextual feature of the mistranscribed entry and a phonetic feature; and

correcting the transcription by replacing the mistranscribed entry with the correct entry.

14. The system of claim 13 , wherein the mistranscribed entry is matched to the correct entry based on the one or more phonetic features of the mistranscribed entry and phonetic features of the correct entry using term frequency-inverse document frequency-based information retrieval.

15. The system of claim 13 , wherein the instructions which, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

identifying textual features of the mistranscribed entry, as part of extracting the features from the mistranscribed entry, based on the one or more domain-specific contexts by applying a domain-specific textual model to the mistranscribed entry; and

matching the mistranscribed entry to the correct entry of the mistranscribed entry based on similarities between the textual features of the mistranscribed entry and textual features of the correct entry in the index of domain-specific entries.

16. The system of claim 13 , wherein the instructions which, when executed by the one or more processors, further cause the one or more processors to perform operations comprising:

identifying contextual features of the mistranscribed entry, as part of extracting the features from the mistranscribed entry, based on the one or more domain-specific contexts by applying a domain-specific contextual model to the mistranscribed entry; and

matching the mistranscribed entry to the correct entry based on similarities between the contextual features of the mistranscribed entry and contextual features of the correct entry in the index of domain-specific entries.

17. The system of claim 13 , wherein the index of domain-specific entries is selected based on the one or more domain-specific contexts associated with the speech.

18. A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform operations comprising:

receiving a transcription of speech created using an automatic speech recognition system;

identifying one or more domain-specific contexts associated with the speech;

detecting a text span from the transcription that includes a mistranscribed entry of the speech based on a user-specific context included in the one or more domain-specific contexts associated with the speech;

extracting features from the mistranscribed entry by applying a user-specific extractor associated with the user-specific context and a domain-specific sequence to sequence grapheme-to-phoneme model, wherein the features include one or more user-specific contextual features of the mistranscribed entry associated with the user-specific context and one or more phonetic features extracted through application of the sequence to sequence grapheme-to-phoneme model;

matching the extracted features against an index of domain-specific entries to identify a correct entry of the mistranscribed entry using term frequency-inverse document frequency-based information retrieval and based at least in part on both a user-specific contextual feature of the mistranscribed entry and a phonetic feature; and

correcting the transcription by replacing the mistranscribed entry with the correct entry.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2019
From: RAGHUNATHAN, KARTHIK; RAGHUVANSHI, ARUSHI; THIMMAIYAH, VIJAY RAMAKRISHNAN; CARROLL, LUCIEN SERAPIO; EMBAR, VARSHA RAVIKUMAR
To: CISCO TECHNOLOGY, INC.
Reel/Frame 048181/0190 →
Continuity (2)
Provisional Application 62701060 · Jul 20, 2018
Related Publication 20200027445A1 · Jan 23, 2020
Cited By (2)
US 12,300,225 US 12,633,290