Granted Patent B1

US 12,321,659 · App. 18/589,088 · Granted Jun 3, 2025

Inventors: Jonathan Lindo (Emerald Hills, CA); Agustin Fonts (Santa Clara, CA); Michael James Armstrong (Bellevue, WA); Nandit Tiku (Sammamish, WA); Biju Mathew (San Ramon, CA); Rukmani Ravisundaram (Newark, CA); Bryce Masatsune Matsumori (San Francisco, CA)

Assignee: Meta Platforms Technologies, LLC

G06F3/1454G06F3/011

Abstract

Claims (57)

1. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for streaming interactive content from an application executing at an artificial reality (XR) device to one or more separate artificial reality devices, the process comprising:

displaying, to a user via an XR device, an XR environment, wherein a shell application executing at the XR device provides the XR environment and the XR device is executing an additional application providing content to the shell application;

streaming, to an other XR device, remote content from the additional application, wherein the other XR device:

in response to the streaming, outputs a version of the remote content; and

receives and processes input relative to the remote content, wherein the processed input is transmitted from the other XR device to the XR device; and

performing functionality for the additional application in response to the transmitted input.

2. The non-transitory computer-readable storage medium of claim 1 ,

wherein the additional application executing at the XR device streams data for a three-dimensional virtual object to the other XR device; and

wherein, in response to the streamed data for the three-dimensional virtual object, the outputting the version of the remote content includes displaying an other three-dimensional virtual object, within an other XR environment, associated with the three-dimensional virtual object.

3. The non-transitory computer-readable storage medium of claim 2 ,

wherein the input relative to the remote content is input relative to the other three-dimensional virtual object;

wherein the input is translated into input for the additional application executing at the XR device before being transmitted from the other XR device to the XR device; and

wherein the performed functionality includes A) altering the three-dimensional virtual object, B) causing an alteration to the other three-dimensional virtual object, or C) both.

4. The non-transitory computer-readable storage medium of claim 1 , wherein the XR device and the other XR device are co-located in a physical room.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the XR environment and an other XR environment provided by the other XR device are a shared XR environment that includes a virtual presence for the user and a virtual presence for an other user of the other XR device.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the process further comprises:

displaying, to the user via the XR device and in addition to the content for the shell application, a representation virtual object within the XR environment that displays the version of the remote content that is being output by the other XR device.

7. The non-transitory computer-readable storage medium of claim 1 , wherein the content for the shell application is provided as a first virtual object by the XR device, the version of the remote content is provided as a second virtual object by the other XR device, and the first virtual object differs in display properties from the second virtual object.

8. The non-transitory computer-readable storage medium of claim 1 ,

wherein the content provided to the shell application is output to a virtual object and the shell application manages a position of the virtual object within the XR environment; and

wherein the other XR devices executes an other shell application that manages a position of an other virtual object that includes the version of the remote content.

9. The non-transitory computer-readable storage medium of claim 8 ,

wherein the shell application manages user input relative to the virtual object received via the XR device and translates the user input for the additional application executing at the XR device; and

wherein the additional application executing at the XR device performs application functionality in response to the translated user input and alters the content displayed at the virtual object.

10. The non-transitory computer-readable storage medium of claim 8 , wherein the other shell application translates the input, received at the other XR device, for the additional application executing at the XR device.

11. A method for streaming interactive content from an application executing at an artificial reality (XR) device to one or more separate artificial reality devices, the method comprising:

displaying, to a user via an XR device, an XR environment, wherein a shell application executing at the XR device provides the XR environment and the XR device is executing an additional application providing content to the shell application;

streaming, to an other XR device, remote content from the additional application, wherein the other XR device:

in response to the streaming, outputs a version of the remote content; and

receives and processes input relative to the remote content, wherein the processed input is transmitted from the other XR device to the XR device; and

performing functionality for the additional application in response to the transmitted input.

12. The method of claim 11 ,

wherein the additional application executing at the XR device streams data for a three-dimensional virtual object to the other XR device; and

wherein, in response to the streamed data for the three-dimensional virtual object, the outputting the version of the remote content includes displaying an other three-dimensional virtual object, within an other XR environment, associated with the three-dimensional virtual object.

13. The method of claim 12 ,

wherein the input relative to the remote content is input relative to the other three-dimensional virtual object;

wherein the input is translated into input for the additional application executing at the XR device before being transmitted from the other XR device to the XR device; and

wherein the performed functionality includes A) altering the three-dimensional virtual object, B) causing an alteration to the other three-dimensional virtual object, or C) both.

14. The method of claim 11 , wherein the the XR device and the other XR device are co-located in a physical room.

15. The method of claim 11 , wherein the XR environment and an other XR environment provided by the other XR device are a shared XR environment that includes a virtual presence for the user and a virtual presence for an other user of the other XR device.

16. The method of claim 11 , wherein the process further comprises:

displaying, to the user via the XR device and in addition to the content for the shell application, a representation virtual object within the XR environment that displays the version of the remote content that is being output by the other XR device.

17. A computing system for streaming interactive content from an application executing at an artificial reality (XR) device to one or more separate artificial reality devices, the computing system comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:

displaying, to a user via an XR device, an XR environment, wherein a shell application executing at the XR device provides the XR environment and the XR device is executing an additional application providing content to the shell application;

streaming, to an other XR device, remote content from the additional application, wherein the other XR device:

in response to the streaming, outputs a version of the remote content; and

receives and processes input relative to the remote content, wherein the processed input is transmitted from the other XR device to the XR device; and

performing functionality for the additional application in response to the transmitted input.

18. The computing system of claim 17 , wherein the content for the shell application is provided as a first virtual object by the XR device, the version of the remote content is provided as a second virtual object by the other XR device, and the first virtual object differs in display properties from the second virtual object.

19. The computing system of claim 17 ,

wherein the content provided to the shell application is output to a virtual object and the shell application manages a position of the virtual object within the XR environment; and

wherein the other XR devices executes an other shell application that manages a position of an other virtual object that includes the version of the remote content.

20. The computing system of claim 19 ,

wherein the shell application manages user input relative to the virtual object received via the XR device and translates the user input for the additional application executing at the XR device; and

wherein the additional application executing at the XR device performs application functionality in response to the translated user input and alters the content displayed at the virtual object.

Continuity (1)

References Cited (205)