Granted Patent B2

US 12,613,938 · App. 18/296,641 · Granted Apr 28, 2026

Inventors: Michal Hlavac (Seattle, WA); Jasper Stevens (London, GB); Arthur Zwiegincew (Medina, WA); Alexander Michael Louie (Milpitas, CA)

Assignee: Meta Platforms Technologies, LLC

G06F18/2178G06F3/011G06F3/017G06F3/0486G06F18/22G06F18/2415G06F21/629G06N20/00G06F3/0482

Abstract

Claims (73)

1 . A method for suggesting, authorizing, and controlling application output in an artificial reality (XR) device, the method comprising:

identifying artificial reality environment anchor points, as points in the real world around the XR device for which virtual objects can be provided;

determining a current XR context of the XR device;

providing suggestions, using the artificial reality environment anchor points, for one or more applications that match the current XR context;

receiving a user action to authorize a selected application of the suggested one or more applications;

in response to the user action, setting permissions for the selected application that allow the selected application to provide virtual object output as overlays on a real-world environment;

identifying a user intent by identifying that a user ignored or closed a threshold amount of virtual object outputs from the selected application; and

in response to identifying the user intent, removing one or more of the permissions set for the selected application.

2 . The method of claim 1 , wherein each of the artificial reality environment anchor points corresponds to a respective particular real-world object identified by the XR device.

3 . The method of claim 1 , wherein the current XR context comprises:

one or more identified objects in the artificial reality environment,

current location data for the XR device, and

environment conditions.

4 . The method of claim 1 , wherein the one or more applications are selected by the determining that a threshold subset of features from the current XR context has been mapped to the particular application.

5 . The method of claim 1 , wherein the one or more applications are selected by:

applying a machine learning model, trained to determine a match score, to features of an XR context and to a type specified for each particular application; and

determining that a resulting match score is above a threshold.

6 . The method of claim 1 , wherein the one or more applications are selected by:

determining a category, of multiple categories, for each of one or more potential applications; and

adding, to the suggestions for one or more applications, a maximum amount of the one or more potential applications in one of the multiple categories, wherein at least one of the one or more potential applications is excluded from the suggestions for one or more applications due to the maximum amount already having been reached for the category of that application.

7 . The method of claim 1 , wherein the received user action includes identifying a user hand gesture that indicates the selected application, from the suggestions, and drags an indication of the selected application to another location in the artificial reality environment.

8 . The method of claim 1 further comprising obtaining components of the selected application by:

determining, based on which aspects of the selected application will be used to provide the output in the artificial reality environment, a priority order for the components of the selected application; and

scheduling download of the components of the selected application according to the priority order.

9 . The method of claim 1 , wherein the permissions set for the selected application include two or more of:

permissions for creating augments in the artificial reality environment,

permissions to provide notifications,

permissions to access input signals,

permissions to receive context features,

permissions to write into a shared XR space,

permissions to exchange objects or other data with other authorized applications, or

any combination thereof.

10 . The method of claim 1 , wherein the XR context comprises determinations based on sensor data, data retrieved from a third party source, and data local to the XR device.

11 . A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for suggesting, authorizing, and controlling application output in an artificial reality (XR) device, the process comprising:

identifying artificial reality environment anchor points, as points in the real world around the XR device for which virtual objects can be provided;

determining a current XR context of the XR device;

providing suggestions, using the artificial reality environment anchor points, for one or more applications that match the current XR context;

receiving a user action to authorize a selected application of the suggested one or more applications;

in response to the user action, setting permissions for the selected application that allow the selected application to provide virtual object output as overlays on a real-world environment;

identifying a user intent by identifying that a user ignored or closed a threshold amount of virtual object outputs from the selected application; and

in response to identifying the user intent, removing one or more of the permissions set for the selected application.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein each of the artificial reality environment anchor points corresponds to a respective particular real-world object identified by the XR device.

13 . The non-transitory computer-readable storage medium of claim 11 , wherein the current XR context comprises:

one or more identified objects in the artificial reality environment,

current location data for the XR device, and

environment conditions.

14 . The non-transitory computer-readable storage medium of claim 11 , wherein the one or more applications are selected by the determining that a threshold subset of features from the current XR context has been mapped to the particular application.

15 . The non-transitory computer-readable storage medium of claim 11 , wherein the one or more applications are selected by:

determining a category, of multiple categories, for each of one or more potential applications; and

adding, to the suggestions for one or more applications, a maximum amount of the one or more potential applications in one of the multiple categories, wherein at least one of the one or more potential applications is excluded from the suggestions for one or more applications due to the maximum amount already having been reached for the category of that application.

16 . A computing system for suggesting, authorizing, and controlling application output in an artificial reality (XR) device, the computing system comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:

identifying artificial reality environment anchor points, as points in the real world around the XR device for which virtual objects can be provided;

determining a current XR context of the XR device;

providing suggestions, using the artificial reality environment anchor points, for one or more applications that match the current XR context;

receiving a user action to authorize a selected application of the suggested one or more applications;

in response to the user action, setting permissions for the selected application that allow the selected application to provide virtual object output;

identifying a user intent by identifying that a user ignored or closed a threshold amount of virtual object outputs from the selected application; and

in response to identifying the user intent, removing one or more of the permissions set for the selected application.

17 . The computing system of claim 16 , wherein the received user action includes identifying a user hand gesture that indicates the selected application, from the suggestions, and drags an indication of the selected application to another location in the artificial reality environment.

18 . The computing system of claim 16 , wherein the process further comprises obtaining components of the selected application by:

determining, based on which aspects of the selected application will be used to provide the output in the artificial reality environment, a priority order for the components of the selected application; and

scheduling download of the components of the selected application according to the priority order.

19 . The computing system of claim 16 , wherein the permissions set for the selected application include two or more of:

permissions for creating augments in the artificial reality environment,

permissions to provide notifications,

permissions to access input signals,

permissions to receive context features,

permissions to write into a shared XR space,

permissions to exchange objects or other data with other authorized applications, or

any combination thereof.

20 . The computing system of claim 16 , wherein the XR context comprises determinations based on sensor data, data retrieved from a third party source, and data local to the XR device.

Assignments (2)

Continuity (2)

References Cited (400)