IP Library Granted Patent US 11,430,157
Granted Patent B1
US 11,430,157 · App. 16/663,163 · Granted Aug 30, 2022

Effects in artificial reality based on environmental features

Inventors: Xiaoyang Gao (Palo Alto, CA); Rahul Prasad (San Francisco, CA); Krishnan Ramnath (Sunnyvale, CA)
Assignee: Meta Platforms Technologies, LLC
G06T11/00G06F3/0482G06F3/04847G06V20/20G10L25/51
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,430,157
App. No.
16/663,163
Granted
Aug 30, 2022
Kind
B1
Abstract

Disclosed herein are related to a device and a method of generating triggers to provide effects to an artificial reality based on features in an environment. In one aspect, a set of candidate triggers for applying effects to the artificial reality provided by the device is determined according to features in an environment. In one aspect, a subset of the set of candidate triggers is determined according to a predetermined policy. In one aspect, corresponding effects to apply to the artificial reality are generated by the device according to the subset of the set of candidate triggers.

Claims (67)

1. A method comprising:

determining, by a device according to features detected in an environment, a set of candidate triggers for applying effects to an artificial reality provided by the device for the environment;

determining, by the device according to a predetermined policy for selecting one or more candidate triggers for applying effects to the artificial reality, a subset of the set of candidate triggers, the predetermined policy specifying (i) a limit of a number of effects to apply to the artificial reality within a time period, and (ii) a defined area within a display region of the artificial reality in which to apply effects to the artificial reality according to one or more effect types for the effects;

generating, by the device according to the subset of the set of candidate triggers, one or more effects corresponding to the features detected in the environment, to apply to the artificial reality; and

applying, by the device, the one or more effects to the artificial reality in the defined area of the displayed region corresponding to an effect type for the one or more effects, according to the limit of the number of effects.

2. The method of claim 1 , further comprising:

detecting, by the device, the features in the environment;

providing, by the device, at least one of the detected features to a remote server to cause the remote server to detect a first candidate trigger for applying effects to the artificial reality; and

determining, by the device, the set of candidate triggers, the set of candidate triggers including the first candidate trigger detected by the remote server.

3. The method of claim 1 , wherein detecting, by the device, the features in the environment includes:

capturing an image of the environment, and

detecting, as one of the features in the environment, a predetermined shape or structure in the captured image.

4. The method of claim 1 , wherein detecting, by the device, the features in the environment includes:

capturing an audio signal of a sound in the environment, and

detecting, as one of the features in the environment, a predetermined pattern in the audio signal.

5. The method of claim 1 , wherein generating the one or more effects to apply to the artificial reality includes:

modifying a setting of a presentation of the artificial reality,

presenting a virtual object in the artificial reality,

presenting information via the artificial reality,

presenting a sound for the artificial reality, or

presenting a profile via the artificial reality.

6. The method of claim 1 , further comprising:

determining rankings of the set of candidate triggers; and

determining the subset of the set of candidate triggers according to the rankings.

7. The method of claim 6 , further comprising determining the rankings according to scores or categories of the set of candidate triggers.

8. The method of claim 1 , wherein determining the subset of the set of candidate triggers includes:

presenting selectable indications of the set of candidate triggers to a user of the artificial reality, and

receiving a selection of one or more of the subset of the set of candidate triggers from the user.

9. A system comprising:

a processor; and

a non-transitory computer readable medium storing instructions when executed by the processor cause the processor to:

determine, according to features detected in an environment, a set of candidate triggers for applying effects to an artificial reality provided for the environment,

determine, according to a predetermined policy for selecting one or more candidate triggers for applying effects to the artificial reality, a subset of the set of candidate triggers, the predetermined policy specifying (i) a limit of a number of effects to apply to the artificial reality within a time period, and (ii) a defined area within a display region of the artificial reality in which to apply effects to the artificial reality according to one or more effect types for the effects,

generate, according to the subset of the set of candidate triggers, one or more effects corresponding to the features detected in the environment, to apply to the artificial reality, and

apply the one or more effects to the artificial reality in the defined area of the displayed region corresponding to an effect type for the one or more effects, according to the limit of the number of effects.

10. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to:

detect the features in the environment,

provide at least one of the detected features to a remote server to cause the remote server to detect a first candidate trigger for applying effects to the artificial reality, and

determine the set of candidate triggers, the set of candidate triggers including the first candidate trigger detected by the remote server.

11. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to:

capture an image of the environment, and

detect, as one of the features in the environment, a predetermined shape or structure in the captured image.

12. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to:

capture an audio signal of a sound in the environment, and

detect, as one of the features in the environment, a predetermined pattern in the audio signal.

13. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to generate the one or more effects to apply to the artificial reality by:

modifying a setting of a presentation of the artificial reality,

presenting a virtual object in the artificial reality,

presenting information via the artificial reality,

presenting a sound for the artificial reality, or

presenting a profile via the artificial reality.

14. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to:

determine rankings of the set of candidate triggers; and

determine the subset of the set of candidate triggers according to the rankings.

15. The system of claim 14 , wherein the instructions when executed by the processor cause the processor to determine the rankings according to scores or categories of the set of candidate triggers.

16. The system of claim 9 , wherein the instructions when executed by the processor cause the processor to determine the subset of the set of candidate triggers by:

presenting selectable indications of the set of candidate triggers to a user of the artificial reality, and

receiving a selection of one or more of the subset of the set of candidate triggers from the user of the artificial reality.

17. A non-transitory computer readable medium storing instructions when executed by a processor cause the processor to:

determine, according to features detected in an environment, a set of candidate triggers for applying effects to an artificial reality provided for the environment;

determine, according to a predetermined policy for selecting one or more candidate triggers for applying effects to the artificial reality, a subset of the set of candidate triggers, the predetermined policy specifying (i) a limit of a number of effects to apply to the artificial reality within a time period, and (ii) a defined area within a display region of the artificial reality in which to apply effects to the artificial reality according to one or more effect types for the effects;

generate, according to the subset of the set of candidate triggers, one or more effects corresponding to the features detected in the environment, to apply to the artificial reality; and

apply the one or more effects to the artificial reality in the defined area of the displayed region corresponding to an effect type for the one or more effects, according to the limit of the number of effects.

18. The non-transitory computer readable medium of claim 17 , further comprising instructions when executed by the processor cause the processor to:

detect the features in the environment,

provide at least one of the detected features to a remote server to cause the remote server to detect a first candidate trigger for applying effects to the artificial reality, and

determine the set of candidate triggers, the set of candidate triggers including the first candidate trigger detected by the remote server.

Assignments (2)
CHANGE OF NAME Recorded Jul 22, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060816/0634 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 26, 2019
From: GAO, XIAOYANG; PRASAD, RAHUL; RAMNATH, KRISHNAN
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 051370/0452 →