IP Library › Granted Patent US 11,727,606
Granted Patent B2
US 11,727,606 · App. 17/707,774 · Granted Aug 15, 2023

Method and device for presenting synthesized reality companion content

Inventors: Ian M. Richter (Los Angeles, CA); Daniel Ulbricht (Sunnyvale, CA); Jean-Daniel E. Nahmias (San Francisco, CA); Omar Elafifi (Oakland, CA); Peter Meier (Los Gatos, CA)
Assignee: APPLE INC.
G06T11/00G06F3/04883G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,727,606
App. No.
17/707,774
Granted
Aug 15, 2023
Kind
B2
Abstract

In one implementation, a method includes: obtaining a user input to view SR content associated with video content; if the video content includes a first scene when the user input was detected: obtaining first SR content for a first time period of the video content associated with the first scene; obtaining a task associated with the first scene; and causing presentation of the first SR content and a first indication of the task associated with the first scene; and if the video content includes a second scene when the user input was detected: obtaining second SR content for a second time period of the video content associated with the second scene; obtaining a task associated with the second scene; and causing presentation of the second SR content and a second indication of the task associated with the second scene.

Claims (103)

1. A method comprising:

at a computing system including non-transitory memory and one or more processors, wherein the computing system is communicatively coupled to a display device and one or more input devices:

while causing presentation of video content, obtaining, via the one or more input devices, a user input indicating a request to view synthesized reality (SR) content associated with the video content;

in accordance with a determination that the video content includes a first scene associated with to a first time period of the video content when the user input was detected:

obtaining first SR content for the first time period of the video content, wherein the first SR content is associated with the first scene within the video content;

obtaining one or more missions or tasks associated with the first scene within the video content; and

causing presentation of the first SR content associated with the first scene within the video content and a first indication of the one or more missions or tasks associated with the first scene within the video content via the display device; and

in accordance with a determination that the video content includes a second scene associated with to a second time period of the video content when the user input was detected:

obtaining second SR content for the first time period of the video content, wherein the second SR content is associated with the second scene within the video content;

obtaining one or more missions or tasks associated with the second scene within the video content; and

causing presentation of the second SR content associated with the second scene within the video content and a second indication of the one or more missions or tasks associated with the second scene within the video content via the display device.

2. The method of claim 1 , wherein the first SR content and second SR content are obtained from a library associated with the video content.

3. The method of claim 1 , wherein the first SR content is generated on-the-fly based at least in part on the first scene within the video content, and wherein the second SR content is generated on-the-fly based at least in part on the second scene within the video content.

4. The method of claim 1 , wherein the first SR content continues concurrently while causing presentation of the first SR content, and wherein the second SR content continues concurrently while causing presentation of the second SR content.

5. The method of claim 4 , wherein the first SR content is paused in response to the user breaching a first threshold region associated with the first SR content, and wherein the second SR content is paused in response to the user breaching a second threshold region associated with the second SR content.

6. The method of claim 1 , wherein the first SR content is paused while causing presentation of the first SR content, and wherein the second SR content is paused while causing presentation of the second SR content.

7. The method of claim 1 , further comprising:

while presenting the first SR content and in response to receiving a first command from the user, ceasing to present the first SR content and continuing causing presentation of the first scene associated with to the first time period of the video content; and

while presenting the second SR content and in response to receiving a second command from the user, ceasing to present the second SR content and continuing causing presentation of the second scene associated with to the second time period of the video content.

8. The method of claim 1 , further comprising:

while causing presentation of the video content that includes the first scene, causing display of a first affordance indicating availability of the first SR content associated with the first scene within the video content, wherein the user input corresponds to selection of the first affordance; and

while causing presentation of the video content that includes the second scene, causing display of a second affordance indicating availability of the second SR content associated with the second scene within the video content, wherein the user input corresponds to selection of the second affordance.

9. The method of claim 1 , further comprising:

while causing presentation of the first SR content

obtaining, via the one or more input devices, one or more user interactions with the first SR content; and

adjusting the first SR content in response to receiving the one or more user interactions with the first SR content; and

while causing presentation of the second SR content

obtaining, via the one or more input devices, one or more user interactions with the second SR content; and

adjusting the first SR content in response to receiving the one or more user interactions with the second SR content.

10. The method of claim 9 , wherein the one or more user interactions with the first SR content include at least one of picking up an object associated with the first SR content, physically engaging with an SR character associated with the first SR content, or speaking with an SR character associated with the first SR content.

11. The method of claim 9 , further comprising:

in response to determining that the one or more user interactions with the first SR content satisfy an exit condition for the first scene, ceasing to cause presentation of the first SR content and causing presentation of the first scene associated with to the first time period of the video content, wherein the exit condition for the first scene is satisfied when the one or more user interactions with the first SR content complete a mission or task associated with the first scene within the video content; and

in response to determining that the one or more user interactions with the second SR content satisfy an exit condition for the second scene, ceasing to cause presentation of the second SR content and causing presentation of the second scene associated with to the second time period of the video content, wherein the exit condition for the second scene is satisfied when the one or more user interactions with the second SR content complete a mission or task associated with the second scene within the video content.

12. The method of claim 1 , further comprising:

while causing presentation of the first SR content

obtaining, via the one or more input devices, one or more commands associated with driving a first SR character within the first SR content; and

adjusting the first SR content in response to obtaining the one or more commands associated with driving the first SR character within the first SR content; and

while causing presentation of the second SR content

obtaining, via the one or more input devices, one or more commands associated with driving a second SR character within the second SR content; and

adjusting the second SR content in response to obtaining the one or more commands associated with driving the second SR character within the second SR content.

13. The method of claim 1 , further comprising:

presenting, via the display device, the one or more missions or tasks associated with the first scene within the video content; and

detecting, via the one or more input devices, a user input selecting a respective mission or task among the one or more missions or tasks associated with the first scene within the video content, wherein the first indication corresponds to the respective mission or task selected by the user input.

14. A computing system comprising:

one or more processors;

a non-transitory memory;

an interface for communicating with a display device and one or more input devices; and

one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the computing system to:

while causing presentation of video content, obtain, via the one or more input devices, a user input indicating a request to view synthesized reality (SR) content associated with the video content;

in accordance with a determination that the video content includes a first scene associated with to a first time period of the video content when the user input was detected:

obtain first SR content for the first time period of the video content, wherein the first SR content is associated with the first scene within the video content;

obtain one or more missions or tasks associated with the first scene within the video content; and

cause presentation of the first SR content associated with the first scene within the video content and a first indication of the one or more missions or tasks associated with the first scene within the video content via the display device; and

in accordance with a determination that the video content includes a second scene associated with to a second time period of the video content when the user input was detected:

obtain second SR content for the first time period of the video content, wherein the second SR content is associated with the second scene within the video content;

obtain one or more missions or tasks associated with the second scene within the video content; and

cause presentation of the second SR content associated with the second scene within the video content and a second indication of the one or more missions or tasks associated with the second scene within the video content via the display device.

15. The computing system of claim 14 , wherein the one or more programs further cause the computing system to:

while causing presentation of the first SR content

obtain, via the one or more input devices, one or more user interactions with the first SR content; and

adjust the first SR content in response to receiving the one or more user interactions with the first SR content; and

while causing presentation of the second SR content

obtain, via the one or more input devices, one or more user interactions with the second SR content; and

adjust the first SR content in response to receiving the one or more user interactions with the second SR content.

16. The computing system of claim 15 , wherein the one or more user interactions with the first SR content include at least one of picking up an object associated with the first SR content, physically engaging with an SR character associated with the first SR content, or speaking with an SR character associated with the first SR content.

17. The computing system of claim 15 , wherein the one or more programs further cause the computing system to:

in response to determining that the one or more user interactions with the first SR content satisfy an exit condition for the first scene, cease to cause presentation of the first SR content and cause presentation of the first scene associated with to the first time period of the video content, wherein the exit condition for the first scene is satisfied when the one or more user interactions with the first SR content complete a mission or task associated with the first scene within the video content; and

in response to determining that the one or more user interactions with the second SR content satisfy an exit condition for the second scene, cease to cause presentation of the second SR content and cause presentation of the second scene associated with to the second time period of the video content, wherein the exit condition for the second scene is satisfied when the one or more user interactions with the second SR content complete a mission or task associated with the second scene within the video content.

18. The computing system of claim 14 , wherein the one or more programs further cause the computing system to:

while causing presentation of the first SR content

obtain, via the one or more input devices, one or more commands associated with driving a first SR character within the first SR content; and

adjust the first SR content in response to obtaining the one or more commands associated with driving the first SR character within the first SR content; and

while causing presentation of the second SR content

obtain, via the one or more input devices, one or more commands associated with driving a second SR character within the second SR content; and

adjust the second SR content in response to obtaining the one or more commands associated with driving the second SR character within the second SR content.

19. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a computing system with an interface for communicating with a display device and one or more input devices, cause the computing system to:

while causing presentation of video content, obtain, via the one or more input devices, a user input indicating a request to view synthesized reality (SR) content associated with the video content;

in accordance with a determination that the video content includes a first scene associated with to a first time period of the video content when the user input was detected:

obtain first SR content for the first time period of the video content, wherein the first SR content is associated with the first scene within the video content;

obtain one or more missions or tasks associated with the first scene within the video content; and

cause presentation of the first SR content associated with the first scene within the video content and a first indication of the one or more missions or tasks associated with the first scene within the video content via the display device; and

in accordance with a determination that the video content includes a second scene associated with to a second time period of the video content when the user input was detected:

obtain second SR content for the first time period of the video content, wherein the second SR content is associated with the second scene within the video content;

obtain one or more missions or tasks associated with the second scene within the video content; and

cause presentation of the second SR content associated with the second scene within the video content and a second indication of the one or more missions or tasks associated with the second scene within the video content via the display device.

20. The non-transitory memory of claim 19 , wherein the one or more programs further cause the computing system to:

while causing presentation of the first SR content

obtain, via the one or more input devices, one or more user interactions with the first SR content; and

adjust the first SR content in response to receiving the one or more user interactions with the first SR content; and

while causing presentation of the second SR content

obtain, via the one or more input devices, one or more user interactions with the second SR content; and

adjust the first SR content in response to receiving the one or more user interactions with the second SR content.

21. The non-transitory memory of claim 20 , wherein the one or more user interactions with the first SR content include at least one of picking up an object associated with the first SR content, physically engaging with an SR character associated with the first SR content, or speaking with an SR character associated with the first SR content.

22. The non-transitory memory of claim 20 , wherein the one or more programs further cause the computing system to:

in response to determining that the one or more user interactions with the first SR content satisfy an exit condition for the first scene, cease to cause presentation of the first SR content and cause presentation of the first scene associated with to the first time period of the video content, wherein the exit condition for the first scene is satisfied when the one or more user interactions with the first SR content complete a mission or task associated with the first scene within the video content; and

in response to determining that the one or more user interactions with the second SR content satisfy an exit condition for the second scene, cease to cause presentation of the second SR content and cause presentation of the second scene associated with to the second time period of the video content, wherein the exit condition for the second scene is satisfied when the one or more user interactions with the second SR content complete a mission or task associated with the second scene within the video content.

23. The non-transitory memory of claim 19 , wherein the one or more programs further cause the computing system to:

while causing presentation of the first SR content

obtain, via the one or more input devices, one or more commands associated with driving a first SR character within the first SR content; and

adjust the first SR content in response to obtaining the one or more commands associated with driving the first SR character within the first SR content; and

while causing presentation of the second SR content

obtain, via the one or more input devices, one or more commands associated with driving a second SR character within the second SR content; and

adjust the second SR content in response to obtaining the one or more commands associated with driving the second SR character within the second SR content.

Continuity (4)
Continuation 16963308
Provisional Application 62734050 · Sep 20, 2018
Provisional Application 62620326 · Jan 22, 2018
Related Publication 20220222869A1 · Jul 14, 2022