Enabling grammars in web page frames
View Patent ↗Enabling grammars in web page frames, including receiving, in a multimodal application on a multimodal device, a frameset document, where the frameset document includes markup defining web page frames; obtaining by the multimodal application content documents for display in each of the web page frames, where the content documents include navigable markup elements; generating by the multimodal application, for each navigable markup element in each content document, a segment of markup defining a speech recognition grammar, including inserting in each such grammar markup identifying content to be displayed when words in the grammar are matched and markup identifying a frame where the content is to be displayed; and enabling by the multimodal application all the generated grammars for speech recognition.
1. A method of enabling grammars in web page frames, the method comprising:
receiving, in a multimodal application on a multimodal device, a frameset document, the frameset document comprising markup defining web page frames;
obtaining, by the multimodal application, content documents for display in each of the web page frames, the content documents comprising navigable markup elements;
generating, by the multimodal application, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar and comprises markup identifying content to be displayed when words in the grammar are matched and markup identifying a frame where the content is to be displayed; and
enabling, by the multimodal application, all the generated grammars so that any of the generated grammars is usable for speech recognition at any one time.
2. The method of claim 1 wherein:
the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames; and
obtaining at least two content documents further comprises iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.
3. The method of claim 1 wherein enabling the generated grammars further comprises:
dynamically generating a markup language fragment specifying a grammar; and
providing the markup language fragment to an automated voice markup language interpreter.
4. The method of claim 1 wherein:
the multimodal device further comprises an automated voice markup language interpreter; and
enabling the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more application programming interface (‘API’) calls from the multimodal application to the automated voice markup language interpreter.
5. The method of claim 1 wherein:
the multimodal device is coupled for data communications to a voice server, the voice server comprising an automated voice markup language interpreter; and
enabling all the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more data communications protocol messages from the multimodal device to the automated voice markup language interpreter on the voice server.
6. The method of claim 1 further comprising:
providing, by the multimodal application to an automated voice markup language interpreter, speech for recognition from a user;
matching, by the automated voice markup language interpreter with an enabled grammar, at least part of the speech for recognition; and
returning, from the automated voice markup language interpreter to the multimodal application, an event indicating an instruction representative of the matched speech.
7. A system for enabling grammars in web page frames, the system comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions capable of:
receiving, in a multimodal application on a multimodal device, a frameset document, the frameset document comprising markup defining web page frames;
obtaining, by the multimodal application, content documents for display in each of the web page frames, the content documents comprising navigable markup elements;
generating, by the multimodal application, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar and comprises markup identifying content to be displayed when words in the grammar are matched and markup identifying a frame where the content is to be displayed; and
enabling, by the multimodal application, all the generated grammars so that any of the generated grammars is usable for speech recognition at any one time.
8. The system of claim 7 wherein:
the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames; and
obtaining at least two content documents further comprises iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.
9. The system of claim 7 wherein enabling the generated grammars further comprises:
dynamically generating a markup language fragment specifying a grammar; and
providing the markup language fragment to an automated voice markup language interpreter.
10. The system of claim 7 wherein:
the multimodal device further comprises an automated voice markup language interpreter; and
enabling the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more application programming interface (‘API’) calls from the multimodal application to the automated voice markup language interpreter.
11. The system of claim 7 wherein:
the multimodal device is coupled for data communications to a voice server, the voice server comprising an automated voice markup language interpreter; and
enabling all the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more data communications protocol messages from the multimodal device to the automated voice markup language interpreter on the voice server.
12. The system of claim 7 further comprising computer program instructions capable of:
providing, by the multimodal application to an automated voice markup language interpreter, speech for recognition from a user;
matching, by the automated voice markup language interpreter with an enabled grammar, at least part of the speech for recognition; and
returning, from the automated voice markup language interpreter to the multimodal application, an event indicating an instruction representative of the matched speech.
13. At least one computer-readable storage medium having instructions encoded thereon which, when executed, perform a method comprising:
receiving, in a multimodal application on a multimodal device, a frameset document, the frameset document comprising markup defining web page frames;
obtaining by the multimodal application, content documents for display in each of the web page frames, the content documents comprising navigable markup elements;
generating, by the multimodal application, for each navigable markup element in each content document, a segment of markup that defines a speech recognition grammar and comprises markup identifying content to be displayed when words in the grammar are matched and markup identifying a frame where the content is to be displayed; and
enabling, by the multimodal application, all the generated grammars so that any of the generated grammars is usable for speech recognition at any one time.
14. The at least one computer-readable storage medium of claim 13 wherein:
the web page frames are organized in a hierarchy according to one or more framesets, the hierarchy characterized by a topmost frame and one or more child frames; and
obtaining at least two content documents further comprises iteratively obtaining, for the topmost frame and for each child frame, a separate content document for display in each frame.
15. The at least one computer-readable storage medium of claim 13 wherein enabling the generated grammars further comprises:
dynamically generating a markup language fragment specifying a grammar; and
providing the markup language fragment to an automated voice markup language interpreter.
16. The at least one computer-readable storage medium of claim 13 wherein:
the multimodal device further comprises an automated voice markup language interpreter; and
enabling the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more application programming interface (‘API’) calls from the multimodal application to the automated voice markup language interpreter.
17. The at least one computer-readable storage medium of claim 13 wherein:
the multimodal device is coupled for data communications to a voice server, the voice server comprising an automated voice markup language interpreter; and
enabling all the generated grammars further comprises providing the grammars to the automated voice markup language interpreter by one or more data communications protocol messages from the multimodal device to the automated voice markup language interpreter on the voice server.
18. The at least one computer-readable storage medium of claim 13 further comprising computer program instructions capable of:
providing, by the multimodal application to an automated voice markup language interpreter, speech for recognition from a user;
matching, by the automated voice markup language interpreter with an enabled grammar, at least part of the speech for recognition; and
returning, from the automated voice markup language interpreter to the multimodal application, an event indicating an instruction representative of the matched speech.