User interface for multimodal information system
View Patent ↗A method and system for providing a user interface for information services related to multimodal information on a computer system is presented. A system for providing a user interface for augmenting multimodal information with information services and interacting with the information services is described.
1. A graphical user interface displayed on a portable wireless electronic device for providing context-based information services by interpreting visual imagery on the portable wireless electronic device comprising:
a display displaying a first tab comprising a first tab pane comprising a video display region, a first plurality of video controls below the video display region, and a first text box having a first area size, wherein the first text box overlays the video display region, the video display region comprises a video, and the first text box comprises at least a portion of an augmentation information associated with the video, wherein the augmentation information is generated by:
a user selecting text displayed in the video,
recognizing the selected text in the video,
generating a context based on the selected text, and
using the generated context in a search for relevant augmentation information to present in the first text box; and
a second tab comprising a second tab pane comprising a second text box having a second area size, greater than the first area size, wherein the second text box comprises the augmentation information displayed in the first text box, and when the second tab is selected, the second tab pane overlays the first tab pane, wherein the augmentation displayed in the first text box is read-only and the augmentation displayed in the second text box can be edited by the user.
2. The graphical user interface of claim 1 wherein the video display region comprises a plurality of horizontal reference lines overlaying the video.
3. The graphical user interface of claim 1 wherein the video comprises visual imagery captured live.
4. The graphical user interface of claim 1 wherein the video comprises a prerecorded visual imagery.
5. The graphical user interface of claim 1 wherein the first plurality of video controls comprises a play button, a stop button, a zoom button, and a record button.
6. The graphical user interface of claim 1 wherein the first tab comprises a first tab label, the second tab comprises a second tab label, and the first and second tab labels are nearer to a top edge of the display of the portable wireless electronic device than a bottom edge of the display.
7. The graphical user interface of claim 1 wherein the portable wireless electronic device is a camera phone.
8. The graphical user interface of claim 1 wherein the augmentation information comprises links to on-line information.
9. The graphical user interface of claim 1 wherein the first text box is an editable text box.
10. The graphical user interface of claim 1 wherein when the first tab is selected, the first tab pane overlays the second tab pane.
11. The graphical user interface of claim 1 wherein the annotation is vocalized using an audio player of the portable wireless electronic device.
12. The graphical user interface of claim 1 wherein the first text box is highlighted by a dashed line surrounding the first text box.
13. The graphical user interface of claim 1 wherein the first tab further comprises a second plurality of video controls comprising a marking control, the marking control being arranged to mark or highlight portions of the visual imagery.
14. The graphical user interface of claim 1 wherein the augmentation information is automatically generated without additional user input.
15. The graphical user interface of claim 1 wherein the augmentation information is generated upon explicit request by a user.
16. The graphical user interface of claim 1 further comprising an interface, the interface being arranged to record audio information through a microphone integrated into the wireless electronic device.