IP Library Granted Patent US 9,436,300
Granted Patent B2
US 9,436,300 · App. 13/545,667 · Granted Sep 6, 2016

Method and apparatus for providing a multimodal user interface track

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,436,300
App. No.
13/545,667
Granted
Sep 6, 2016
Kind
B2
Abstract

An approach is provided for providing a multimodal user interface track. A multimodal generation platform determining one or more user interface elements for interacting with at least one media segment. The multimodal generation platform further causes, at least in part, an inclusion of the one or more user interface elements as at least one track of the at least one media segment. Accordingly, when the at least one track is processed during a presentation of the at least one media segment by at least one device, the at least one track causes, at least in part, an enablement of the one or more user interface elements.

Claims (45)

1. A method comprising:

on a device, presenting a user interface including a user interface element of a first track associated with a first user interface modality type of a media segment;

including one or more scripts for initiating one or more actions associated with the media segment in at least one track of the media segment;

determining, with a processor, one or more detected changes in contextual information associated with at least one of the user of the device or the device, wherein the one or more changes are detected during the presentation of the media segment;

identifying an additional user interface modality type associated with the one or more detected changes in the contextual information;

identifying a template associated with the additional user interface modality type;

processing the media segment with respect to the template to modify the user interface element during the presentation based on the template and the identified additional user interface modality type associated with the one or more changes, wherein the user interface modality type comprises at least one of a visual modality, an audio modality, a tactile modality, an olfactory modality or a combination thereof;

identifying one or more semantic elements based on the user interface modality type, wherein the one or more semantic elements describe characteristics or features of the media segment; and

including the one or more semantic elements as at least one other track of the media segment.

2. The method of claim 1 , further comprising:

processing any of the capability information, preference information, contextual information, or a combination thereof associated with the one device to identify the user interface modality type.

3. The method of claim 2 further comprising:

determining that the capability information, preference information, the contextual information, or a combination thereof associated with the device is at least partially incomplete; and

identifying the user interface modality type associated with the one or more changes based on one or more default criteria.

4. The method of claim 1 , further comprising:

processing the media segment with respect to one or more words, one or more tokens, or a combination thereof in an audio modality template to track one or more topics across tracks.

5. The method of claim 4 , wherein the one or more words, the one or more tokens, or a combination thereof are based on one or more datasets located at the device, and the one or more tokens include grammar tokens for speech.

6. The method of claim 1 ,

wherein the one or more scripts are processed during the presentation.

7. The method of claim 1 , further comprising:

in response to receiving an indication of a contextual event, cause a cancelling or limiting of the user interface modality type that requires more battery power or is prohibited based on the presence of a software protection dongle attached to the device, or a combination thereof.

8. The method of claim 1 , wherein the media segment further represents one of a plurality of views from one single location, the track is processed during the presentation of the media segment by the device to cause an enablement of the one or more user interface elements for switching among views from the one single location, and the views from the one single location include one or more wide-angle views, one or more telephotos, one or more front views, one or more rear views, one or more left views, one or more right views, one or more top views, or a combination thereof.

9. The method of claim 1 , wherein the template is associated with the user interface modality.

10. The method of claim 1 , wherein the template is a multi-modal template, and the presentation is modified to include at least two modalities.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:

on a device, present a user interface including a user interface element of a first track associated with a first user interface modality type of a media segment;

include one or more scripts for initiating one or more actions associated with the media segment in at least one track of the media segment;

determine one or more detected changes in contextual information associated with at least one of a user of the device or the device, wherein the one or more changes are detected during the presentation of the media segment;

identify an additional user interface modality type associated with the one or more detected changes in the contextual information by executing the one or more scripts;

identify a template associated with the additional user interface modality type; and

process the media segment with respect to the template to modify the user interface element during the presentation based on the template and the identified user interface modality type associated with the one or more changes, wherein the user interface modality type comprises at least one of a visual modality, an audio modality, a tactile modality, an olfactory modality or a combination thereof;

identify one or more semantic elements based on the user interface modality type, wherein the one or more semantic elements describe characteristics or features of the media segment; and

include the one or more semantic elements as at least one other track of the media segment.

12. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to cause the apparatus to:

process any of the capability information, preference information, contextual information, or a combination thereof associated with the device to identify the user interface modality type.

13. The apparatus of claim 12 , wherein the at least one memory and the computer program code are further configured to cause the apparatus to:

determine that the capability information, preference information, the contextual information, or a combination thereof associated with the device is at least partially incomplete; and

identify the user interface modality type associated with the one or more changes based on one or more default criteria.

14. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to cause the apparatus to:

process the media segment with respect to one or more words, one or more tokens, or a combination thereof in an audio modality template to modify the user interface element.

15. The apparatus of claim 14 , wherein the one or more words, the one or more tokens, or a combination thereof are based on one or more datasets located at the device.

16. The apparatus of claim 11 ,

wherein the one or more scripts are processed during the presentation.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2023
From: PIECE FUTURE PTE LTD
To: FIXAPA SDN BHD
Reel/Frame 064516/0209 →
CHANGE OF NAME Recorded Feb 19, 2019
From: NOKIA TECHNOLOGIES OY
To: PIECE FUTURE PTE LTD
Reel/Frame 048376/0576 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 035231/0785 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2012
From: SATHISH, SAILESH KUMAR; CURCIO, IGOR DANILO DIEGO
To: NOKIA CORPORATION
Reel/Frame 029041/0738 →