IP Library Granted Patent US 11,366,527
Granted Patent B1
US 11,366,527 · App. 16/933,307 · Granted Jun 21, 2022

Systems and methods for sensing gestures via vibration-sensitive wearables donned by users of artificial reality systems

Inventors: Hrvoje Benko (Seattle, WA); Wolf Kienzle (Seattle, WA); Neil Weiss (Seattle, WA); Yu-Te Wang (Seattle, WA); Yanjun Ma (Seattle, WA); David R. Perek (Seattle, WA)
Assignee: Facebook Technologies, LLC
G06F3/017G06F3/011G06F3/0346G06N3/08G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,366,527
App. No.
16/933,307
Granted
Jun 21, 2022
Kind
B1
Abstract

A system for vibration-driven sensing may include a wearable dimensioned to be donned by a user of an artificial reality system. The system may also include a vibration sensor that is incorporated into the wearable and generates an electrical response that corresponds to a vibration detected at the wearable. The system may further include at least one processing device communicatively coupled to the vibration sensor. The processing device may determine, based at least in part on the electrical response generated by the vibration sensor, that the user has made a specific gesture with at least one body part. In response to this determination, the processing device may generate an input command for the artificial reality system based at least in part on the specific gesture made by the user. Various other systems and methods are also disclosed.

Claims (62)

1. A wearable electronic device comprising:

a wearable structure dimensioned to be donned by a user of an artificial-reality system;

an accelerometer arranged on the wearable structure, wherein the accelerometer is configured to:

detect vibrations at or near the wearable structure;

generate, an output data rate of at least 200 hertz, electrical responses that correspond to the vibrations detected at or near the wearable structure;

an inertial measurement unit (IMU) sensor that generates data that correspond to the vibrations; and

one or more processors communicatively coupled to the accelerometer, wherein the one or more processors are configured to:

determine, based at least in part on at least one of the electrical responses generated by the accelerometer and the data generated by the IMU, that the user has made a specific gesture with at least one body part; and

in response to determining that the user has made the specific gesture with the body part, generate an input command for the artificial-reality system based at least in part on the specific gesture made by the user.

2. The wearable electronic device of claim 1 , wherein the accelerometer generates the electrical responses by measuring an amount of acceleration caused by the vibrations, the vibrations having propagated from the body part to the wearable electronic device via the body of the user.

3. The wearable electronic device of claim 1 , further comprising at least one additional sensor that generates data indicative of the specific gesture made by the user; and

wherein the one or more processors determine, based at least in part on the at least one of the electrical responses generated by the accelerometer, the data generated by the IMU, and the data generated by the additional sensor, that the user has made the specific gesture with the body part.

4. The wearable electronic device of claim 3 , wherein the additional sensor comprises an electromyography (EMG) sensor.

5. The wearable electronic device of claim 1 , wherein the one or more processors:

capture one or more samples of the electrical responses generated by the accelerometer while the user performs the specific gesture; and

calibrate, based at least in part on the one or more samples of the electrical responses, the wearable electronic device to recognize the specific gesture as performed by the user.

6. The wearable electronic device of claim 1 , wherein the one or more processors:

measure the at least one of the electrical responses generated by the accelerometer; and

determines that the measurement of the at least one of the electrical responses is indicative of the specific gesture made by the user, wherein the specific gesture comprises at least one of:

a tapping together of an index finger of the user and a thumb of the user;

a tapping together of a middle finger of the user and a thumb of the user;

a hand gesture;

a wrist twist;

a physical touching of the body part to another body part; or

a physical touching of the body part to an inanimate object.

7. The wearable electronic device of claim 1 , wherein the accelerometer is positioned proximate to a skin surface of the user within the wearable electronic device.

8. The wearable electronic device of claim 1 , further comprising a transmitter that is incorporated into the wearable electronic device; and

wherein the one or more processors transmit the input command for the artificial-reality system to a head-mounted display via the transmitter, wherein the input command causes the head-mounted display to account for the specific gesture made by the user.

9. The wearable electronic device of claim 1 , wherein the electrical responses generated by the accelerometer are commensurate with the vibrations detected at or near the wearable structure.

10. The wearable electronic device of claim 1 , wherein the one or more processors facilitate modifying at least one virtual component of the artificial-reality system to account for the specific gesture made by the user.

11. The wearable electronic device of claim 1 , wherein the one or more processors sample the electrical responses generated by the accelerometer at a sampling rate of at least 200 hertz.

12. The wearable electronic device of claim 1 , wherein:

the accelerometer generates the electrical responses at an output data rate of at least 800 hertz; or

the one or more processors sample the electrical responses generated by the accelerometer at a sampling rate of at least 800 hertz.

13. The wearable electronic device of claim 1 , wherein:

the accelerometer generates the electrical responses at an output data rate of at least 2,400 hertz; or

the one or more processors sample the electrical responses generated by the accelerometer at a sampling rate of at least 2,400 hertz.

14. The wearable electronic device of claim 1 , wherein the accelerometer is capable of measuring a range of −3 g to +3 g of gravitational acceleration.

15. An artificial-reality system comprising:

a head-mounted display device; and

a wearable device communicatively coupled to the head-mounted display device, wherein the wearable device comprises:

a fastener dimensioned to facilitate securing the wearable device to a user of the artificial-reality system;

an accelerometer arranged on the wearable device, wherein the accelerometer is configured to:

detect vibrations at or near the wearable device;

generate, an output data rate of at least 200 hertz, electrical responses that correspond to the vibrations detected at or near the wearable device;

an inertial measurement unit (IMU) sensor that generates data that correspond to the vibrations; and

one or more processors communicatively coupled to the accelerometer, wherein the one or more processors are configured to:

determine, based at least in part on at least one of the electrical responses generated by the accelerometer and the data generated by the IMU, that the user has made a specific gesture with at least one body part; and

in response to determining that the user has made the specific gesture with the body part, generate an input command for the artificial-reality system based at least in part on the specific gesture made by the user.

16. The artificial-reality system of claim 15 , wherein the accelerometer generates the electrical responses by measuring an amount of acceleration caused by the vibrations, the vibrations having propagated from the body part to the wearable device via the body of the user.

17. The artificial-reality system of claim 15 , further comprising at least one additional sensor that generates data indicative of the specific gesture made by the user; and

wherein the one or more processors determine, based at least in part on the at least one of the electrical responses generated by the accelerometer, the data generated by the IMU, and the data generated by the additional sensor, that the user has made the specific gesture with the body part.

18. The artificial-reality system of claim 17 , wherein the additional sensor comprises an electromyography (EMG) sensor.

19. The artificial-reality system of claim 15 , wherein the one or more processors:

capture one or more samples of the electrical responses generated by the accelerometer while the user performs the specific gesture; and

calibrate, based at least in part on the one or more samples of the electrical responses, the wearable device to recognize the specific gesture as performed by the user.

20. A method comprising:

detecting, by an accelerometer arranged on a wearable device donned by a user of an artificial-reality system, vibrations that arrived at the wearable device via a body of the user;

generating, by the accelerometer at an output data rate of at least 200 hertz, electrical responses that correspond to the vibrations that arrived at the wearable device via the body of the user;

generating, by an inertial measurement unit (IMU) sensor arranged on the wearable device donned by the user of the artificial-reality system, data that correspond to the vibrations;

determining, based at least in part on at least one of the electrical responses generated by the accelerometer and the data generated by the IMU, that the user has made a specific gesture with at least one body part; and

in response to determining that the user has made the specific gesture with the body part, generating an input command for the artificial-reality system based at least in part on the specific gesture made by the user.

Assignments (2)
CHANGE OF NAME Recorded May 27, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060203/0228 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2022
From: BENKO, HRVOJE; KIENZLE, WOLF; WEISS, NEIL; WANG, YU-TE; MA, YANJUN; PEREK, DAVID R.
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 058938/0177 →