IP Library Granted Patent US 8,924,212
Granted Patent B1
US 8,924,212 · App. 11/213,458 · Granted Dec 30, 2014

System and method for robust access and entry to large structured data using voice form-filling

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,924,212
App. No.
11/213,458
Granted
Dec 30, 2014
Kind
B1
Abstract

A method, apparatus and machine-readable medium are provided. A phonotactic grammar is utilized to perform speech recognition on received speech and to generate a phoneme lattice. A document shortlist is generated based on using the phoneme lattice to query an index. A grammar is generated from the document shortlist. Data for each of at least one input field is identified based on the received speech and the generated grammar.

Claims (60)

1. A method comprising:

performing, via a processor, a first pass of entries in a database to generate a shortlist of the entries, the first pass comprising:

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar;

removing silence and filler words from the phoneme lattice to yield a revised phoneme lattice;

normalizing costs in the revised phoneme lattice, to yield a normalized phoneme lattice with a best path having a cost of zero;

identifying N-best lowest cost indices of an index for the database based on the normalized phoneme lattice; and

generating the shortlist of the entries in the database based on the N-best lowest cost indices;

performing a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising:

generating a grammar from the shortlist of the entries in the database to yield a generated grammar, having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints; and

populating an input field of a form using the final result.

2. The method of claim 1 , wherein the index is previously generated based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, the factors of interest comprising trigrams, and wherein the database is a name/address database.

3. The method of claim 2 , wherein the factors of interest further comprise N-grams based on the valid entries in the database.

4. The method of claim 1 , wherein the generated grammar comprises common variations of the input field.

5. The method of claim 1 , wherein the index is divided into two sub-indices.

6. The method of claim 1 , wherein the phonotactic grammar is an N-gram phonotactic grammar.

7. The method of claim 6 , wherein the N-gram phonotactic grammar is unsmoothed, recognizing only N-grams which have been seen in data used to train the N-gram phonotactic grammar.

8. The method of claim 1 , wherein the generated grammar is used to: perform one of another speech recognition on the received speech, constrain recognition for each of the input field, and rescore a result of performing the speech recognition.

9. The method of claim 1 , wherein generating the shortlist of the entries in the database further comprises:

utilizing only factors of interest from the index to generate a cost-normalized query based on the normalized lattice; and

selecting the N-best lowest indices from indices of the index containing the factors of interest and the cost-normalized query.

10. A system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

performing a first pass of entries in a database to generate a shortlist of the entries, the first pass comprising:

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar;

removing silence and filler words from the phoneme lattice to generate a revised phoneme lattice;

normalizing costs in the revised phoneme lattice, to yield a normalized phoneme lattice with a best path having a cost of zero;

identifying N-best lowest cost indices of an index for the database based on the normalized phoneme lattice; and

generating the shortlist of the entries in the database based on the N-best lowest cost indices;

perform a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising:

generating a grammar from the shortlist of the entries in the database to yield a generated grammar, having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints; and

populating an input field of a form using the final result.

11. The system of claim 10 , wherein the index is previously generated based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, the factors of interest comprising trigrams, and wherein the database is a name/address database.

12. The system of claim 11 , wherein the factors of interest comprise N-grams based on the valid entries in the database.

13. The system of claim 10 , wherein the generated grammar comprises common variations of the input field.

14. The system of claim 10 , wherein the index is divided into two sub-indices.

15. The system of claim 10 , wherein the phonotactic grammar is an N-gram phonotactic grammar.

16. The system of claim 10 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising performing one of another speech recognition on the received speech, constrain recognition for the input field, and rescore a result of performing the speech recognition.

17. The speech processing device of claim 10 , the computer-readable storage medium having additional instructions stored which result in the operations further comprising:

utilizing only factors of interest from the index to generate a cost-normalized query based on the normalized query; and

selecting the N-best lowest indices from indices of the index containing the factors of interest and the cost-normalized query.

18. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

instructions for performing, via a processor, a first pass of entries in a database to generate a shortlist of the entries, the first pass comprising:

utilizing a phonotactic grammar to perform speech recognition on received speech and to generate a phoneme lattice, wherein the utilizing ignores inter-field constraints of the phonotactic grammar;

removing silence and filler words from the phoneme lattice to yield a revised phoneme lattice;

normalizing costs in the revised phoneme lattice, to yield a normalized phoneme lattice with a best path having a cost of zero;

identifying N-best lowest cost indices of an index for the database based on the normalized phoneme lattice; and

generating the shortlist of the entries in the database based on the N-best lowest cost indices; and

instructions for performing a second pass on the shortlist of the entries in the database to obtain a final result, the second pass comprising:

generating a grammar from the shortlist of the entries in the database to yield a generated grammar, having inter-field constraints; and

identifying data from the database for each of the input field based on the received speech, the generated grammar, and the inter-field constraints; and populating an input field of a form using the final result.

19. The computer-readable storage device of claim 18 , wherein the index is previously generated based on a plurality of training phoneme lattices and factors of interest from valid entries in the database, the factors of interest comprising trigrams, and wherein the database is a name/address database.

20. The computer-readable storage device of claim 19 , wherein the factors of interest comprise N-grams based on the valid entries in the database.

21. The computer-readable storage device of claim 18 , wherein the generated grammar comprises common variations of the input field.

22. The computer-readable storage device of claim 18 , wherein the instructions for utilizing a phonotactic grammar to perform speech recognition on the received speech and to generate a phoneme lattice further comprise instructions for generating the phoneme lattice by utilizing an N-gram phonotactic grammar.

23. The computer-readable storage device of claim 18 , wherein generating the shortlist of the entries in the database further comprises:

utilizing only factors of interest from the index to generate a cost-normalized query based on the normalized lattice; and

selecting the N-best lowest indices from indices of the index containing the factors of interest and the cost-normalized query.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065578/0676 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2017
From: AT&T INTELLECTUAL PROPERTY II, L.P.
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 041512/0608 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2015
From: AT&T CORP.
To: AT&T PROPERTIES, LLC
Reel/Frame 035579/0487 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2015
From: AT&T PROPERTIES, LLC
To: AT&T INTELLECTUAL PROPERTY II, L.P.
Reel/Frame 035579/0541 →