System and method for tightly coupling automatic speech recognition and search
View Patent ↗Disclosed herein are systems, methods, and computer-readable storage media for performing a search. A system configured to practice the method first receives from an automatic speech recognition (ASR) system a word lattice based on speech query and receives indexed documents from an information repository. The system composes, based on the word lattice and the indexed documents, at least one triple including a query word, selected indexed document, and weight. The system generates an N-best path through the word lattice based on the at least one triple and re-ranks ASR output based on the N-best path. The system aggregates each weight across the query words to generate N-best listings and returns search results to the speech query based on the re-ranked ASR output and the N-best listings. The lattice can be a confusion network, the arc density of which can be adjusted for a desired performance level.
1. A method comprising:
receiving, from an automatic speech recognition system, a word lattice based on a speech query;
receiving indexed documents from an information repository;
composing, based on the word lattice and the indexed documents, a triple comprising a query word, a selected indexed document, and a weight;
generating an N-best path through the word lattice based on the triple;
re-ranking automatic speech recognition output based on the N-best path, to yield a re-ranked automatic speech recognition output;
aggregating each weight across the query words to generate N-best listings; and
returning search results to the speech query based on the re-ranked automatic speech recognition output and the N-best listings.
2. The method of claim 1 , wherein the word lattice is a word confusion network.
3. The method of claim 2 , further comprising adjusting an arc density of the word confusion network based on a desired performance level.
4. The method of claim 2 , further comprising adjusting an arc density of the word confusion network based on a desired accuracy level.
5. The method of claim 1 , wherein automatic speech recognition and searching is performed simultaneously.
6. The method of claim 1 , wherein the weight is determined based on a plurality of relevance metrics.
7. The method of claim 1 , wherein a mobile device receives the speech query.
8. The method of claim 1 , wherein re-ranking the automatic speech recognition output is further based on constraints encoded by relevance metrics and the information repository.
9. The method of claim 1 , wherein the indexed documents are represented as a search finite state machine.
10. The method of claim 9 , wherein the search finite state machine represents an index of the indexed documents.
11. A system comprising:
a processor; and
a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:
receiving, from an automatic speech recognition system, a word lattice based on a speech query;
receiving indexed documents from an information repository;
composing, based on the word lattice and the indexed documents, a triple comprising a query word, a selected indexed document, and a weight;
generating an N-best path through the word lattice based on the triple;
re-ranking automatic speech recognition output based on the N-best path, to yield a re-ranked automatic speech recognition output;
aggregating each weight across the query words to generate N-best listings; and
returning search results to the speech query based on the re-ranked automatic speech recognition output and the N-best listings.
12. The system of claim 11 , wherein the indexed documents are represented as a search finite state machine.
13. The system of claim 11 , wherein the weight is determined based on a plurality of relevance metrics.
14. The system of claim 11 , wherein a mobile device receives the speech query.
15. The system of claim 11 , wherein re-ranking automatic speech recognition output is further based on constraints encoded as relevance metrics.
16. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:
receiving from an automatic speech recognition system a word lattice based on speech query;
receiving indexed documents from an information repository;
composing, based on the word lattice and the indexed documents, a triple comprising a query word, a selected indexed document, and a weight;
generating an N-best path through the word lattice based on the triple;
re-ranking automatic speech recognition output based on the N-best path, to yield a re-ranked automatic speech recognition output;
aggregating each weight across the query words to generate N-best listings; and
returning search results to the speech query based on the re-ranked automatic speech recognition output and the N-best listings.
17. The computer-readable storage device of claim 16 , wherein the word lattice is a word confusion network.
18. The computer-readable storage device of claim 17 , the computer-readable storage device having additional instructions stored which result in the operations further comprising adjusting an arc density of the word confusion network based on a desired performance level.
19. The computer-readable storage device of claim 17 , the computer-readable storage device having additional instructions stored which result in the operations further comprising adjusting an arc density of the word confusion network based on a desired accuracy level.
20. The computer-readable storage device of claim 16 , wherein re-ranking ranked automatic speech recognition output is further based on constraints encoded as relevance metrics.