IP Library Patent Application 17722128
Patent Application
App. No. 17/722,128

WEARABLE ELECTRONIC DEVICES AND EXTENDED REALITY SYSTEMS INCLUDING NEUROMUSCULAR SENSORS

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
17/722,128
Abstract

The disclosed system for interacting with objects in an extended reality (XR) environment generated by an XR system may include (1) neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user and (2) at least one computer processor programmed to (a) determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment and (b) instruct the XR system to, based on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment. Other embodiments of this aspect include corresponding apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

Claims (54)

1 . A computerized system for interacting with objects in an extended reality (XR) environment generated by an XR system, the computerized system comprising:

a plurality of neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices configured to be worn on the wrist of the user; and

at least one computer processor programmed to:

determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment; and

instruct the XR system to, based at least in part on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment.

2 . The computerized system of claim 1 , wherein the at least one computer processor instructing the XR system to augment the interaction of the user with the object comprises providing augmented properties to the object.

3 . The computerized system of claim 1 , wherein the plurality of neuromuscular sensors includes at least a first pair of neuromuscular sensors and a second pair of neuromuscular sensors, wherein:

the first pair of neuromuscular sensors is configured to sense neuromuscular signals generated by the user at a first wrist location when the one or more wearable devices is worn by the user; and

the second pair of neuromuscular sensors is configured to sense neuromuscular signals generated by the user at a second wrist location when the one or more wearable devices is worn by the user.

4 . The computerized system of claim 1 , wherein the object comprises a physical object.

5 . The computerized system of claim 4 , wherein the physical object comprises at least one of:

a stylus;

a writing implement;

a keyboard;

a stick;

a ruler; or

a surface of a table.

6 . The computerized system of claim 1 , wherein the interaction of the user with the object in the XR environment comprises at least one of:

selection of one or more objects;

control of one or more objects;

activation or deactivation of one or more objects; or

adjustment of settings or features relating to one or more objects.

7 . The computerized system of claim 1 , wherein the determination of the information relating to an interaction of the user with the object is determined based additionally on image data associated with the interaction of the user with the object.

8 . The computerized system of claim 7 , further comprising a camera configured to capture images of a hand of the user to generate the image data.

9 . The computerized system of claim 8 , wherein the camera is mounted on a head-mounted display.

10 . The computerized system of claim 1 , wherein the at least one computer processor is further programmed to:

instruct the XR system to generate feedback based at least in part on the information relating to the interaction of the user with the object; and

instruct the XR system to provide the feedback to the user.

11 . The computerized system of claim 10 , wherein the feedback comprises an indication of an amount of force to be applied to the object by the user, and the amount of force is determined based at least in part on the sensed neuromuscular signals.

12 . A wrist-wearable device, comprising:

a plurality of neuromuscular sensors arranged to sense, at a user's wrist, neuromuscular signals generated by the user when the user interacts with an object in an extended reality (XR) environment;

one or more processors configured to:

determine, based at least in part on the sensed neuromuscular signals, information relating to the interaction of the user with the object in the XR environment;

augment, based at least in part on the determined information relating to the interaction of the user with the object, at least one property of the object in the XR environment; and

determine, based at least in part on the determined information relating to the interaction of the user with the object, a haptic feedback signal to be provided to the user; and

a haptic feedback element configured to provide the haptic feedback signal to the user corresponding to the determined amount of force to be applied to the object by the user.

13 . The wrist-wearable device of claim 12 , wherein the haptic feedback signal comprises one or more of:

a vibration;

a skin-tap;

an electrical jolt; or

an application of force.

14 . The wrist-wearable device of claim 13 , wherein the one or more processors is configured to determine an amount of force to be applied to the object by the user when the user interacts with the object in the XR environment.

15 . The wrist-wearable device of claim 12 , wherein the plurality of neuromuscular sensors comprises a plurality of distinct pairs of neuromuscular sensors.

16 . The wrist-wearable device of claim 15 , wherein the pairs of neuromuscular sensors are arranged circumferentially around a band shaped and sized to be worn on the user's wrist.

17 . The wrist-wearable device of claim 12 , wherein the object comprises a physical object.

18 . The wrist-wearable device of claim 12 , wherein the one or more processors is configured to determine the information relating to the interaction of the user with the object further based on an output of an inference model to which the sensed neuromuscular signals are provided as an input.

19 . An extended reality (XR) system, comprising:

a wrist-wearable device including a plurality of neuromuscular sensors arranged to sense, at a user's wrist, neuromuscular signals generated by the user when the user interacts with an object in an XR environment;

a head-mounted display in communication with the wrist-wearable device, the head-mounted display configured to display information to a user; and

one or more processors configured to:

determine, based at least in part on the sensed neuromuscular signals, information relating to the interaction of the user with the object when the user interacts with the object in the XR environment;

determine, based at least in part on the determined information relating to the interaction of the user with the object, an augmented property to be applied to the object in the XR environment; and

apply the augmented property to the object in the XR environment.

20 . The XR system of claim 19 , wherein the augmented property comprises at least one additional function of the object.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2022
From: REMALEY, MASON
To: COGNESCENT CORPORATION
Reel/Frame 060790/0509 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2022
From: OSBORN, CHRISTOPHER; AWAD, LANA; BERENZWEIG, ADAM; SUSU-MAGO, ARIELLE; ASTOLFI, MICHAEL; WETMORE, DANIEL
To: CTRL-LABS CORPORATION
Reel/Frame 060790/0600 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2022
From: CTRL-LABS CORPORATION
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 061159/0264 →
CHANGE OF NAME Recorded Aug 12, 2022
From: COGNESCENT CORPORATION
To: CTRL-LABS CORPORATION
Reel/Frame 061161/0603 →
CHANGE OF NAME Recorded Jun 7, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060291/0526 →