IP Library › Granted Patent US 11,647,260
Granted Patent B2
US 11,647,260 · App. 17/729,462 · Granted May 9, 2023

Content event mapping

Inventors: Ian M. Richter (Los Angeles, CA); Daniel Ulbricht (Sunnyvale, CA); Eshan Verma (Mountain View, CA)
Assignee: APPLE INC.
H04N21/8133G06T19/006G06V20/20H04N21/44008H04N21/8456
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,647,260
App. No.
17/729,462
Granted
May 9, 2023
Kind
B2
Abstract

In one implementation, consumption of media content (such as video, audio, or text) is supplemented with an immersive synthesized reality (SR) map based on the media content. In various implementations described herein, the SR map includes a plurality of SR environment representations which, when selected by a user, cause display of a corresponding SR environment.

Claims (42)

1. A method comprising:

at an electronic device including a processor and non-transitory memory:

obtaining media content data characterized by a timeline, wherein the media content data represents a plurality of events;

determining, from the media content data, a plurality of event definitions and a plurality of respective event times in the timeline corresponding to the plurality of events;

obtaining, based on the plurality of event definitions, a plurality of event representations;

determining a path and a plurality of respective locations along the path for the plurality of event representations, wherein the path is defined by an ordered set of locations including the plurality of respective locations in an order based on the plurality of respective event times; and

displaying a map including the plurality of event representations displayed at the plurality of respective locations along the path.

2. The method of claim 1 , wherein the media content data includes at least one of video content data, audio content data, or text content data.

3. The method of claim 1 , wherein determining a first event definition of the plurality of event definitions comprises performing, using the processor, image analysis of video content data.

4. The method of claim 3 , wherein performing image analysis of the video content data includes detecting an object in an image of the video content data.

5. The method of claim 1 , wherein determining a first event definition of the plurality of event definitions comprises performing, using the processor, audio analysis of audio content data.

6. The method of claim 1 , wherein determining a first event definition of the plurality of event definitions comprises detecting, using the processor, one or more words in text content data.

7. The method of claim 1 , wherein obtaining a first event representation of the plurality of event representations based on a first event definition of the plurality of event definitions includes selecting the first event representation from a plurality of stored event representations based on the first event definition.

8. The method of claim 1 , wherein obtaining a first event representation of the plurality of event representations based on a first event definition of the plurality of event definitions includes generating the first event representation based on the media content data.

9. The method of claim 1 , wherein determining the path and the plurality of respective locations includes determining the path before determining the plurality of respective locations and determining the plurality of respective locations after determining the path.

10. The method of claim 1 , wherein determining the path and the plurality of respective locations includes determining the plurality of respective locations before determining the path and determining the path after determining the plurality of respective locations.

11. The method of claim 1 , wherein displaying the map includes concurrently presenting the media content data according to the timeline and displaying respective ones of the plurality of event representations at corresponding ones of the plurality of respective times in the timeline.

12. The method of claim 1 , further comprising:

receiving user input indicative of a selection of a particular event representation of the plurality of event representations; and

in response to receiving the user input, displaying content associated with a particular one of the plurality of events corresponding to the particular one of the plurality of event representations.

13. The method of claim 12 , wherein the content associated with the particular one of the plurality of events is based on the media content data.

14. A device comprising:

a non-transitory memory; and

one or more processors to:

obtain media content data characterized by a timeline, wherein the media content data represents a plurality of events;

determine, from the media content data, a plurality of event definitions and a plurality of respective event times in the timeline corresponding to the plurality of events;

obtain, based on the plurality of event definitions, a plurality of event representations;

determine a path and a plurality of respective locations along the path for the plurality of event representations, wherein the path is defined by an ordered set of locations including the plurality of respective locations in an order based on the plurality of respective event times; and

display a map including the plurality of event representations displayed at the plurality of respective locations along the path.

15. The device of claim 14 , wherein the one or more processors are to determine a first event definition of the plurality of event definitions by performing image analysis of video content data.

16. The device of claim 15 , wherein the one or more processors are to perform image analysis of the video content data by detecting an object in an image of the video content data.

17. The device of claim 14 , wherein the one or more processors are to display the map by concurrently presenting the media content data according to the timeline and displaying respective ones of the plurality of event representations at corresponding ones of the plurality of respective times in the timeline.

18. The device of claim 14 , wherein the one or more processers are further to:

receive user input indicative of a selection of a particular event representation of the plurality of event representations; and

in response to receiving the user input, display content associated with a particular one of the plurality of events corresponding to the particular one of the plurality of event representations.

19. The device of claim 18 , wherein the content associated with the particular one of the plurality of events is based on the media content data.

20. A non-transitory computer-readable medium having instructions encoded thereon which, when executed by one or more processors of a device, cause the device to:

obtain media content data characterized by a timeline, wherein the media content data represents a plurality of events;

determine, from the media content data, a plurality of event definitions and a plurality of respective event times in the timeline corresponding to the plurality of events;

obtain, based on the plurality of event definitions, a plurality of event representations;

determine a path and a plurality of respective locations along the path for the plurality of event representations, wherein the path is defined by an ordered set of locations including the plurality of respective locations in an order based on the plurality of respective event times; and

display a map including the plurality of event representations displayed at the plurality of respective locations along the path.

Continuity (3)
Continuation 17275038
Provisional Application 62737550 · Sep 27, 2018
Related Publication 20220256249A1 · Aug 11, 2022