Sensors for accurately interacting with objects in an artificial-reality environment, and systems and methods of use thereof
The various implementations described herein include methods and systems for tracking user movements. In one aspect, a method includes, while a user is interacting with a virtual object, obtaining tracking information by tracking, via a sensor, a position of an arm of a user. The method also includes, in conjunction with tracking the position of the arm, obtaining wrist information by detecting, via a neuromuscular-signal sensor, a movement of a wrist of the arm. The method further includes assigning one or more motion characteristics to the virtual object in accordance with the tracking information and the wrist information.
1. A method of tracking user movements, the method comprising:
while a user is interacting with a virtual object displayed to the user via a head-wearable device:
obtaining tracking information by tracking, via a sensor, a position of an arm of the user; and
in conjunction with tracking the position of the arm, obtaining wrist information by detecting, via a neuromuscular-signal sensor, neuromuscular signals indicating respective muscle intensity levels at a wrist of the arm while the user is continuing to interact with the virtual object;
determining, based on one or both of the tracking information and the neuromuscular signals indicating respective muscle intensity levels at the wrist of the arm while the user is continuing to interact with the virtual object, a contact feature characterizing the user's interaction with the virtual object; and
in response to determining the contact feature, assigning one or more motion characteristics to the virtual object in accordance with the tracking information and the wrist information, wherein the one or more motion characteristics include an object velocity and an object trajectory.
2. The method of claim 1 , further comprising determining an angle of release based on the wrist information and a release input from the user and assigning the one or more motion characteristics in accordance with the angle of release.
3. The method of claim 1 , further comprising determining a center of mass based on the wrist information and assigning the one or more motion characteristics in accordance with the center of mass.
4. The method of claim 1 , wherein the sensor is an inertial measurement unit (IMU) sensor.
5. The method of claim 1 , wherein the sensor is an imaging sensor.
6. The method of claim 1 , wherein the sensor and the neuromuscular-signal sensor are arranged on a wrist-wearable device.
7. The method of claim 6 , wherein:
the neuromuscular-signal sensor is arranged to sense signals for a first motor unit in the arm of the user; and
the wrist-wearable device further includes a second neuromuscular-signal sensor arranged to sense signals for a second motor unit in the arm of the user.
8. The method of claim 1 , further comprising, in conjunction with tracking the position of the arm:
obtaining additional tracking information by tracking a position of the other arm of the user; and
obtaining additional wrist information by detecting, via an additional neuromuscular-signal sensor, a movement of a wrist of the other arm, wherein the one or more motion characteristics are based on the additional tracking information and the additional wrist information.
9. The method of claim 1 , wherein the sensor is arranged on a controller.
10. The method of claim 1 , wherein:
the user interacting with the virtual object comprises the user throwing the virtual object; and
the one or more motion characteristics are assigned to the virtual object in response to detecting the user perform a release gesture.
11. The method of claim 10 , wherein a rotational velocity of the virtual object is assigned based on the wrist information.
12. The method of claim 10 , wherein the release gesture comprises the user pressing or releasing a button.
13. The method of claim 10 , wherein during a throwing motion the user is performing a pinch gesture, and wherein the release gesture comprises the user releasing the pinch gesture.
14. The method of claim 1 , wherein:
the virtual object is a virtual ball; and
the user interacting with the virtual object comprises the user striking the virtual ball.
15. The method of claim 1 , wherein feedback information corresponding to the one or more motion characteristics is provided to the user.
16. A computing system, comprising:
control circuitry;
memory; and
one or more sets of instructions stored in the memory and configured for execution by the control circuitry, the one or more sets of instructions comprising instructions for:
while a user is interacting with a virtual object displayed to the user via a head-wearable device:
obtaining tracking information by tracking, via a sensor, a position of an arm of the user; and
in conjunction with tracking the position of the arm, obtaining wrist information by detecting, via a neuromuscular-signal sensor, neuromuscular signals indicating respective muscle intensity levels at a movement of a wrist of the arm while the user is continuing to interact with the virtual object;
determining, based on one or both of the tracking information and the neuromuscular signals indicating respective muscle intensity levels at the wrist of the arm while the user is continuing to interact with the virtual object, a contact feature characterizing the user's interaction with the virtual object; and
in response to determining the contact feature, assigning one or more motion characteristics to the virtual object in accordance with the tracking information and the wrist information, wherein the one or more motion characteristics include an object velocity and an object trajectory.
17. A non-transitory computer-readable storage medium storing one or more sets of instructions configured for execution by a computing device having control circuitry and memory, the one or more sets of instructions comprising instructions for:
while a user is interacting with a virtual object displayed to the user via a head-wearable device:
obtaining tracking information by tracking, via a sensor, a position of an arm of the user; and
in conjunction with tracking the position of the arm, obtaining wrist information by detecting, via a neuromuscular-signal sensor, neuromuscular signals indicating respective muscle intensity levels at a movement of a wrist of the arm while the user is continuing to interact with the virtual object;
determining, based on one or both of the tracking information and the neuromuscular signals indicating respective muscle intensity levels at the wrist of the arm while the user is continuing to interact with the virtual object, a contact feature characterizing the user's interaction with the virtual object; and
in response to determining the contact feature, assigning one or more motion characteristics to the virtual object in accordance with the tracking information and the wrist information, wherein the one or more motion characteristics include an object velocity and an object trajectory.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more sets of instructions further include instructions for: conjunction with tracking the position of the arm:
determining an angle of release based on the wrist information and a release input from the user and assigning the one or more motion characteristics in accordance with the angle of release.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the one or more sets of instructions further include instructions for:
in conjunction with tracking the position of the arm:
obtaining additional tracking information by tracking a position of the other arm of the user; and
obtaining additional wrist information by detecting, via an additional neuromuscular-signal sensor, a movement of a wrist of the other arm, wherein the one or more motion characteristics are based on the additional tracking information and the additional wrist information.