Retrieval and presentation of network service results for mobile device using a multimodal browser
View Patent ↗A method of obtaining information using a mobile device can include receiving a request including speech data from the mobile device, and querying a network service using query information extracted from the speech data, whereby search results are received from the network service. The search results can be formatted for presentation on a display of the mobile device. The search results further can be sent, along with a voice grammar generated from the search results, to the mobile device. The mobile device then can render the search results.
1. A method carried out by at least one computer, the method comprising acts of:
receiving a request comprising speech data from a mobile device;
querying a network service using query information obtained from the speech data, whereby search results are received from the network service;
formatting the search results for presentation on a display of the mobile device;
generating a voice grammar based at least in part on the search results; and
sending the search results and the voice grammar generated from the search results to the mobile device.
2. The method of claim 1 , further comprising acts of:
selecting a term from the speech data; and
dynamically building a query and including the term selected from the speech data within the query, wherein the act of querying further comprises sending the query to the network service.
3. The method of claim 2 , wherein the speech data comprises digitized audio, the method further comprising an act of converting the speech data into a textual representation of the speech data.
4. The method of claim 1 , wherein the act of sending further comprises resending, to the mobile device, a different voice grammar from a prior transaction with the mobile device, with the search results and the voice grammar.
5. The method of claim 1 , wherein the act of generating the voice grammar comprises selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein formatting the search results comprises creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.
6. The method of claim 5 , further comprising acts of:
receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;
building a subsequent query using the term specified in the speech data of the subsequent request; and
sending the subsequent query to the network service.
7. The method of claim 2 , wherein the speech data comprises a textual representation of user speech captured by the mobile device.
8. The method of claim 5 , wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.
9. The method of claim 1 , wherein the at least one computer comprises a proxy server.
10. The method of claim 1 , wherein the search results represent a subset of information queried
11. An article of manufacture, comprising:
at least one computer-readable non-transitory storage medium having encoded thereon instructions which, when executed by at least one processor, perform a method comprising acts of:
receiving a request comprising speech data from a mobile device;
querying a network service using query information obtained from the speech data, whereby search results are received from the network service;
formatting the search results for presentation on a display of the mobile device;
generating a voice grammar based at least in part on the search results; and
sending the search results and the voice grammar generated from the search results to the mobile device.
12. The article of manufacture of claim 11 , wherein the method further comprises acts of:
selecting a term from the speech data; and
dynamically building a query and including the term selected from the speech data within the query, wherein the act of querying further comprises sending the query to the network service.
13. The article of manufacture of claim 12 , wherein the speech data comprises digitized audio, and wherein the method further comprises an act of converting the speech data into a textual representation of the speech data.
14. The article of manufacture of claim 12 , whrein the speech data comprises a textual respresentation of user speech captured by the mobile device.
15. The article of manufacture of claim 11 , wherein the act of sending further comprises resending, to the mobile device, a different voice grammar from a prior transaction with the mobile device, with the search results and the voice grammar.
16. The article of manufacture of claim 11 , wherein the act of generating the voice grammar comprises selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein formatting the search results comprises creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.
17. The article of manufacture of claim 16 , wherein the method further comprises acts of:
receiving a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;
building a subsequent query using the term specified in the speech data of the subsequent request; and
sending the subsequent query to the network service.
18. The article of manufacture of claim 16 , wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.
19. The article of manufacture of claim 11 , wherein the search results represent a subset of information queried when querying the network service.
20. A system comprising at least one proxy server programmed to:
receive a request comprising speech data from a mobile device;
query a network service using query information obtained from the speech data, whereby search results are received from the network service;
format the search results for presentation on a display of the mobile device;
generate a voice grammar based at least in part on the search results; and
send the search results and the voice grammar generated from the search results to the mobile device.
21. The system of claim 20 , wherein the at least one proxy server is further programmed to:
select a term from the speech data;
dynamically build a query and include the term selected from the speech data within the query; and
send the query to the network service.
22. The system of claim 21 , wherein the speech data comprises digitized audio, and wherein the at least one proxy server is further programmed to convert the speech data into a textual representation of the speech data.
23. The system of claim 21 , wherein the speech data comprises a textual representation of user speech captured by the mobile device.
24. The system of claim 20 , wherein the at least one proxy server is further programmed to send to the mobile device a different voice grammar from a prior transaction with the mobile device.
25. The system of claim 20 , wherein the at least one proxy server is further programmed to generate the voice grammar at least in part by selecting a term from the search results and dynamically generating the voice grammar to include the term selected from the search results, and wherein the at least one proxy server is further programmed to format the search results at least in part by creating a multi-modal markup language document which, upon execution, causes the term from the search results to be displayed upon a display screen of the mobile device, wherein the voice grammar is part of the multi-modal markup language document.
26. The system of claim 25 , wherein the at least one proxy server is further programmed to:
receive a subsequent request comprising speech data from the mobile device, wherein the subsequent request specifies the term from the search results;
build a subsequent query using the term specified in the speech data of the subsequent request; and
send the subsequent query to the network service.
27. The system of claim 25 , wherein, upon execution of the multi-modal markup language document by the mobile device, a user utterance comprising the term selected from the search results causes the mobile device to initiate a request for further information regarding the term selected from the search results.
28. The system of claim 20 , wherein the search results represent a subset of information queried when querying the network service.