Granted Patent B2

US 12,591,304 · App. 18/415,563 · Granted Mar 31, 2026

Inventors: Faizan Muhammad (Medford, WA); Daniel Wetmore (Brooklyn, NY)

Assignee: Meta Platforms Technologies, LLC

G06F3/014G06F3/015G06T7/33G06T19/003G06T19/20G06V10/147G06V10/945G06V20/20G06V20/64G06T2207/10016

Abstract

Claims (55)

1 . A system comprising:

a head-mounted device communicatively coupled with a wearable device that includes a plurality of neuromuscular sensors configured to sense neuromuscular signals from a first user, the plurality of neuromuscular sensors being arranged on the wearable device structured to be worn by the first user;

at least one camera configured to capture information about a physical environment; and

at least one processor programmed to:

determine that the first user is present within the physical environment using the information captured by the camera, wherein the first user has control privileges for a first object and a second object within an extended-reality experience associated with the physical environment;

determine an identity of a second user that is present within the physical environment, wherein the second user has control privileges for the first object and a third object within the extended-reality experience associated with the physical environment;

detect one or more neuromuscular signals of the first user sensed by the plurality of neuromuscular sensors; and

cause presentation, via the head-mounted device, of a control interface for the first user, selected based on the identity of the second user and in accordance with the first user and the second user both having control privileges for the first object within the extended-reality experience, that allows the first user to interact with the second user via the first object based on the detected neuromuscular signals of the first user.

2 . The system of claim 1 , wherein a first user gesture indicated by the neuromuscular signals from the first user is configured to initiate capturing information about the physical environment, a second user gesture indicated by the neuromuscular signals from the first user is configured to terminate capturing information about the physical environment, a third user gesture indicated by the neuromuscular signals from the first user is configured to identify the first object associated with the physical environment, and a fourth user gesture indicated by the neuromuscular signals from the first user is configured to identify the first object associated with the physical environment as a reference object.

3 . The system of claim 1 , wherein the second user's identity is determined by identifying an electronic device that is associated with the second user.

4 . The system of claim 3 , wherein the control interface is specific to the second user and includes one or more controls for controlling the electronic device associated with the second user.

5 . The system of claim 3 , wherein the electronic device comprises at least one of:

a smart device;

a smartphone;

a haptic device;

an electronic gaming device; or

a display device.

6 . The system of claim 5 , wherein the neuromuscular signals of the first user cause at least one of the following:

a personal message to be transmitted to the smartphone of the second user;

an electronic game input to be performed in the electronic gaming device played by the first user;

an activation of the haptic device worn by the first user;

an image to be displayed on the display device viewable by the second user in the environment; or

permit the second user to control a function of the environment.

7 . The system of claim 5 , wherein the processor is further programmed to cause permission to be granted to the second user to control at least one of:

the electronic gaming device;

the display device;

the smartphone; or

the haptic device.

8 . The system of claim 1 , wherein the processor is further programmed to generate an extended reality (XR) environment in which the first user and the second interact with each other.

9 . The system of claim 8 , wherein the processor is programmed to enable the first user to interact with the second user in the XR environment based on neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors.

10 . The system of claim 8 , wherein the XR environment enables the first and second users to share control of each other's electronically controllable objects.

11 . The system of claim 8 , wherein recognized neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors cause an electronic game to be played with the second user in the XR environment.

12 . The system of claim 8 , wherein recognized neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors cause messages to be sent to a smartphone associated with the second user.

13 . The system of claim 8 , wherein recognized neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors cause haptic signals to be sent to the second user.

14 . A wearable electronic device comprising:

a plurality of neuromuscular sensors configured to sense neuromuscular signals from a first user, the plurality of neuromuscular sensors being arranged on the wearable device structured to be worn by the first user, wherein the wearable electronic device is communicatively coupled with a head-mounted device;

at least one camera configured to capture information about a physical environment; and

at least one processor programmed to:

determine that the first user is present within the physical environment using the information captured by the camera, wherein the first user has control privileges for a first object and a second object within an extended-reality experience associated with the physical environment;

determine an identity of a second user that is present within the environment, wherein the second user has control privileges for the first object and a third object within the extended-reality experience associated with the physical environment;

detect one or more neuromuscular signals of the first user sensed by the plurality of neuromuscular sensors; and

cause presentation, via the head-mounted device, of a control interface for the first user, selected based on the identity of the second user and in accordance with the first user and the second user both having control privileges for the first object within the extended-reality experience, that allows the first user to interact with the second user via the first object based on the detected neuromuscular signals of the first user.

15 . The wearable electronic device of claim 14 , wherein the processor is further programmed to generate an extended reality (XR) environment in which the first user and the second interact with each other.

16 . The wearable electronic device of claim 15 , wherein the processor is programmed to enable the first user to interact with the second user in the XR environment based on neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors.

17 . The wearable electronic device of claim 15 , wherein the XR environment enables the first and second users to share control of each user's electronically controllable objects.

18 . The wearable electronic device of claim 15 , wherein recognized neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors cause an electronic game to be played with the second user in the XR environment.

19 . The wearable electronic device of claim 15 , wherein recognized neuromuscular activities recognized from the neuromuscular signals sensed by the plurality of neuromuscular sensors cause messages to be sent to a smartphone associated with the second user.

20 . An apparatus comprising:

a head-mounted device communicatively coupled with a wearable device that includes a plurality of neuromuscular sensors configured to sense neuromuscular signals from a first user, the plurality of neuromuscular sensors being arranged on the wearable device structured to be worn by the first user;

at least one camera configured to capture information about a physical environment; and

at least one processor programmed to:

determine that the first user is present within the physical environment using the information captured by the camera, wherein the first user has control privileges for a first object and a second object within an extended-reality experience associated with the physical environment;

determine an identity of a second user that is present within the physical environment, wherein the second user has control privileges for the first object and a third object within the extended-reality experience associated with the physical environment;

detect one or more neuromuscular signals of the first user sensed by the plurality of neuromuscular sensors; and

cause presentation, via the head-mounted device, of a control interface for the first user, selected based on the identity of the second user and in accordance with the first user and the second user both having control privileges for the first object within the extended-reality experience, that allows the first user to interact with the second user via the first object based on the detected neuromuscular signals of the first user.

Assignments (2)

Continuity (3)

References Cited (400)