IP Library Granted Patent US 11,406,896
Granted Patent B1
US 11,406,896 · App. 16/153,658 · Granted Aug 9, 2022

Augmented reality storytelling: audience-side

Inventors: Vincent Charles Cheung (San Carlos, CA); Baback Elmieh (Palo Alto, CA); Connie Yeewei Ho (San Jose, CA); Girish Patangay (Los Altos, CA); Mark Alexander Walsh (Simi Valley, CA)
Assignee: Meta Platforms, Inc.
A63F13/53A63F13/40A63F13/45A63F13/537A63F2300/405A63F2300/632A63F2300/69
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,406,896
App. No.
16/153,658
Granted
Aug 9, 2022
Kind
B1
Abstract

In one embodiment, a technique includes receiving a first story fragment from a storyteller interface of a first computing system. The technique further includes displaying the first story fragment on an audience interface of a second computing system. The technique also includes detecting a trigger on the audience interface of the second computing system. In the technique, the trigger corresponds to the first story fragment. The technique also includes identifying a special effect associated with the trigger. The technique further includes outputting the special effect and the first story fragment on at least one of the storyteller interface displayed on the first computing system and the audience interface displayed on the second computing system.

Claims (62)

1. A method comprising:

by one or more computing systems, receiving a first story fragment from a storyteller interface of a first computing system associated with a storyteller, wherein the first story fragment is displayed within a first background layer depicting a first physical environment of the storyteller;

by the one or more computing systems, displaying the first story fragment on an audience interface of a second computing system associated with an audience, wherein the first story fragment is displayed within a second background layer depicting a second physical environment of the audience;

by the one or more computing systems, detecting a trigger relating to the audience interface of the second computing system, wherein the trigger corresponds to the first story fragment;

by the one or more computing systems, identifying a special effect associated with the trigger; and

by the one or more computing systems, outputting the special effect and the first story fragment on at least one of the storyteller interface displayed on the first computing system and the audience interface displayed on the second computing system, wherein outputting the special effect on the storyteller interface comprises modifying the identified special effect based on the first physical environment, and wherein outputting the special effect on the audience interface comprises modifying the identified special effect based on the second physical environment.

2. The method of claim 1 , wherein a trigger comprises one or more of a gesture, an audio sound, a voice sound, a time interval, a contact event, a proximity, and a context.

3. The method of claim 1 , wherein a special effect comprises one or more of a visual layer, a visual effect, an audio layer, and an audio effect.

4. The method of claim 1 , further comprising:

determining a privacy setting associated with at least one of the storyteller and the audience; and

based on the privacy setting, filtering story fragments and special effects displayed on the audience interface.

5. The method of claim 2 , further comprising:

selecting a default privacy setting based on an age of a user, wherein the user comprises one of the audience and the storyteller and the default privacy setting is strict if the user is a minor; and

based on the privacy setting, turning off one or more triggers.

6. The method of claim 1 , wherein a pre-scripted story is divided into a set of segments of one or more granularity levels, wherein the first story fragment is an element of the set of segments.

7. The method of claim 6 , further comprising:

detecting a second trigger;

responsive to detecting the second trigger, selecting a second story fragment from the set of segments for display; and

displaying the second story fragment on the audience interface.

8. The method of claim 7 , wherein selecting the second story fragment comprises:

identifying a branch point of the pre-scripted story;

selecting a branch of the story based on the second trigger;

selecting a subset of the set of segments corresponding to the selected branch; and

selecting the second story fragment from the selected subset.

9. The method of claim 8 , wherein selecting the second story fragment further comprises:

identifying the branch point of the pre-scripted story;

displaying the storyteller interface on the second computing system;

receiving a selection via the storyteller interface of the branch of the story; and

selecting the second story segment corresponding to the selected branch.

10. The method of claim 1 , wherein the audience interface displays public information and private information, wherein public information is displayed on both the audience interface and the storyteller interface and private information is displayed on one of the audience interface and the storyteller interface.

11. The method of claim 7 , further comprising:

receiving a third trigger;

identifying a previous branch point of the pre-scripted story;

inferring a branch of the pre-scripted story at the previous branch point to select;

selecting a third story fragment from a subset of the set of segments corresponding to the inferred branch; and

displaying the third story fragment.

12. The method of claim 1 , comprising a third computing system, wherein the third computing system displays the audience interface and wherein the audience display of the audience interface displayed on the third computing system displays the self-display of the audience interface displayed on the second computing system and wherein the audience display of the audience interface displayed on the second computing system displays the self-display of the audience interface displayed on the third computing system.

13. The method of claim 12 , further comprising:

receiving a selection menu on the audience interface displayed on the second computing system and the third computing system, wherein the selection menu corresponds to an audience poll;

receiving user data corresponding to the audience poll;

identifying the special effect based on the user data;

outputting the special effect on the audience interface displayed on the second computing system and the third computing system.

14. The method of claim 13 , wherein the user data corresponds to one or more of an activity level, an excitement level, a sound level, a sound pitch, and a majority selection.

15. The method of claim 1 , wherein the audience interface comprises one or more of a story control menu, a visual display, a self-display, a story timeline indicator, and a storyteller display, and wherein a size of a storyteller display on the audience interface is increased responsive to a change in one or more of an activity level, a facial expression, and a sound level detected by the storyteller interface.

16. The method of claim 12 , further comprising increasing a size of an audience display on a second system responsive to a change in one or more of an activity level, a facial expression, and a sound level detected by the audience interface on the third system.

17. The method of claim 12 , wherein one or more of the audience display and the storyteller display displayed on the second computing system are blocked or limited based on a privacy setting of the storyteller interface displayed on the first computing system or a privacy setting of the audience interface displayed on the third computing system.

18. The method of claim 15 , further comprising recording content from one or more of the story control menu, the visual display, the self-display, the story timeline indicator, and the storyteller display.

19. One or more non-transitory computer-readable storage media embodying software that is operable when executed by a server computing system to:

receive a first story fragment from a storyteller interface of a first computing system associated with a storyteller, wherein the first story fragment is displayed within a first background layer depicting a first physical environment of the storyteller;

display the first story fragment on an audience interface of a second computing system associated with an audience, wherein the first story fragment is displayed within a second background layer depicting a second physical environment of the audience;

detect a trigger on the audience interface of the second computing system, wherein the trigger corresponds to the first story fragment;

identify a special effect associated with the trigger; and

output the special effect and the first story fragment on at least one of the storyteller interface displayed on the first computing system and the audience interface displayed on the second computing system, wherein outputting the special effect on the storyteller interface comprises modifying the identified special effect based on the first physical environment, and wherein outputting the special effect on the audience interface comprises modifying the identified special effect based on the second physical environment.

20. A system comprising:

a client system of a user;

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:

receive a first story fragment from a storyteller interface of a first computing system associated with a storyteller, wherein the first story fragment is displayed within a first background layer depicting a first physical environment of the storyteller;

display the first story fragment on an audience interface of a second computing system associated with an audience, wherein the first story fragment is displayed within a second background layer depicting a second physical environment of the audience;

detect a trigger on the audience interface of the second computing system, wherein the trigger corresponds to the first story fragment;

identify a special effect associated with the trigger; and

output the special effect and the first story fragment on at least one of the storyteller interface displayed on the first computing system and the audience interface displayed on the second computing system, wherein outputting the special effect on the storyteller interface comprises modifying the identified special effect based on the first physical environment, and wherein outputting the special effect on the audience interface comprises modifying the identified special effect based on the second physical environment.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2019
From: CHEUNG, VINCENT CHARLES; ELMIEH, BABACK; HO, CONNIE YEEWEI; PATANGAY, GIRISH; WALSH, MARK ALEXANDER
To: FACEBOOK, INC.
Reel/Frame 050671/0012 →
Continuity (1)
Provisional Application 62682818 · Jun 8, 2018
Cited By (19)
US 12,282,592 US 12,405,658 US 12,511,847 US 12,524,124 US 12,524,956 US 12,524,977 US 12,541,280 US 12,566,532 US 12,608,115 US 12,608,877 US 12,608,890 US 12,614,038 US 12,625,608 US 12,632,170 US 12,633,044 US 12,663,907 US 12,675,958 US 12,687,922 US 12,688,660