IP Library Granted Patent US 12663864
Granted Patent B2
US 12663864 · App. 18/604,338 · Granted Jun 23, 2026

EMG-based control for interacting with vehicles, and systems and methods of use thereof

Inventors: Andrew Scott Brenner (Sunnyvale, CA); Sean Eric Camacho (Long Beach, CA); Willy Huang (San Francisco, CA); Vicki Chang (Santa Clara, CA)
Assignee: Meta Platforms Technologies, LLC
G06F3/015B60W10/06B60W10/30B60W50/14B60W60/001F02D41/042G06F3/012G06F3/017B60W2540/221B60W2540/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12663864
App. No.
18/604,338
Granted
Jun 23, 2026
Kind
B2
Abstract

Systems and methods for controlling a vehicle system using a wearable device is disclosed. The method includes, while a wrist-wearable device is communicatively coupled with a vehicle system, detecting, via a wrist-wearable device worn by a user, a hand gesture performed by the user. In accordance with a determination that the hand gesture is associated with a vehicle-control action, the method further includes identifying a device of the vehicle system associated with the vehicle-control action, and providing instructions to the vehicle system to cause performance of the vehicle-control action at the device of the vehicle system. The hand gesture is an in-air hand gesture or a surface-contact gesture.

Claims (67)

1 . A non-transitory, computer-readable storage medium including instructions that, when executed by one or more processors of a wrist-wearable device, cause the wrist-wearable device to:

while a wrist-wearable device is communicatively coupled with a vehicle system and a head-wearable device:

detect, via one or more biopotential sensors of the wrist-wearable device while it is worn by a user, a hand gesture performed by the user;

identify a field of view of the user based on data from the head-wearable device;

in accordance with a determination that the hand gesture is associated with a vehicle-control action:

identify a device of the vehicle system associated with the vehicle-control action, and

provide instructions to the vehicle system to cause performance of the vehicle-control action at the device of the vehicle system; and

in accordance with a determination that another device of the vehicle system is within the field of view of the user, provide instructions to the vehicle system to cause performance of the vehicle-control action at the other device of the vehicle system within the field of view of the user.

2 . The non-transitory, computer-readable storage medium of claim 1 , wherein the hand gesture is a first hand gesture and the vehicle-control action is a first vehicle-control action, and the instructions, when executed by the one or more processors of the wrist-wearable device, further cause the wrist-wearable device to:

detect, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a second hand gesture; and

in accordance with a determination that the second hand gesture is associated with a second vehicle-control action, distinct from the first vehicle-control action:

determine a different device of the vehicle system associated with the second vehicle-control action, and

provide instructions to the vehicle system to cause performance of the second vehicle-control action at the different device of the vehicle system.

3 . The non-transitory, computer-readable storage medium of claim 2 , wherein the instructions, when executed by the one or more processors of the wrist-wearable device, further cause the wrist-wearable device to:

detect, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a third hand gesture; and

in accordance with a determination that the third hand gesture is associated with a device-cycle control action that causes the vehicle system to alternate between devices of the vehicle system, provide instructions to the vehicle system to cause selection of another device of the vehicle system for receiving one or more vehicle-control actions.

4 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action causes activation of a vehicle self-driving action.

5 . The non-transitory, computer-readable storage medium of claim 1 , wherein the hand gesture is a first hand gesture and the vehicle-control action is a first vehicle-control action and the instructions, when executed by the one or more processors of the wrist-wearable device, further cause the wrist-wearable device to:

detect, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a fourth hand gesture performed by the user; and

in accordance with a determination that the fourth hand gesture is associated with a vehicle-monitoring action:

receive from, the vehicle system, current vehicle information, and

cause presentation of the current vehicle information at the wrist-wearable device or at a device in communication with the wrist-wearable device.

6 . The non-transitory, computer-readable storage medium of claim 5 , wherein the vehicle-monitoring action includes playback of image data captured by an imaging device of the vehicle system.

7 . The non-transitory, computer-readable storage medium of claim 5 , wherein the vehicle-monitoring action includes playback of audio data captured by a microphone device of the vehicle system, presentation of a vehicle battery level, presentation of a vehicle-charging status, and/or presentation of a vehicle's status.

8 . The non-transitory, computer-readable storage medium of claim 5 , wherein the vehicle-monitoring action includes presentation of one or more of a vehicle battery level, a vehicle-charging status, and a vehicle's status.

9 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action is a media-player control action.

10 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action is an engine-ignition control action or an engine-shutoff control action.

11 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action is a vehicle-security control action.

12 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action is a vehicle-thermal control action.

13 . The non-transitory, computer-readable storage medium of claim 1 , wherein the vehicle-control action is a vehicle-window control action.

14 . A method comprising:

while a wrist-wearable device is communicatively coupled with a vehicle system and a head-wearable device:

detecting, via one or more biopotential sensors of the wrist-wearable device while it is worn by a user, a hand gesture performed by the user;

identifying a field of view of the user based on data from the head-wearable device;

in accordance with a determination that the hand gesture is associated with a vehicle-control action:

identifying a device of the vehicle system associated with the vehicle-control action, and

providing instructions to the vehicle system to cause performance of the vehicle-control action at the device of the vehicle system; and

in accordance with a determination that another device of the vehicle system is within the field of view of the user, provide instructions to the vehicle system to cause performance of the vehicle-control action at the other device of the vehicle system within the field of view of the user.

15 . The method of claim 14 , wherein the hand gesture is a first hand gesture and the vehicle-control action is a first vehicle-control action, and the method further comprises:

detecting, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a second hand gesture; and

in accordance with a determination that the second hand gesture is associated with a second vehicle-control action, distinct from the first vehicle-control action:

determining a different device of the vehicle system associated with the second vehicle-control action, and

providing instructions to the vehicle system to cause performance of the second vehicle-control action at the different device of the vehicle system.

16 . The method of claim 15 , further comprising:

detecting, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a third hand gesture; and

in accordance with a determination that the third hand gesture is associated with a device-cycle control action that causes the vehicle system to alternate between devices of the vehicle system, providing instructions to the vehicle system to cause selection of another device of the vehicle system for receiving one or more vehicle-control actions.

17 . A wrist-wearable device for controlling a vehicle system using one or more hand gestures, the wrist-wearable device configured to perform or cause performance of:

while a wrist-wearable device is communicatively coupled with a vehicle system and a head-wearable device:

detecting, via one or more biopotential sensors of the wrist-wearable device while it is worn by a user, a hand gesture performed by the user;

identifying a field of view of the user based on data from the head-wearable device;

in accordance with a determination that the hand gesture is associated with a vehicle-control action:

identifying a device of the vehicle system associated with the vehicle-control action, and

providing instructions to the vehicle system to cause performance of the vehicle-control action at the device of the vehicle system; and

in accordance with a determination that another device of the vehicle system is within the field of view of the user, provide instructions to the vehicle system to cause performance of the vehicle-control action at the other device of the vehicle system within the field of view of the user.

18 . The wrist-wearable device of claim 17 , wherein the hand gesture is a first hand gesture and the vehicle-control action is a first vehicle-control action, and the wrist-wearable device is further configured to perform or cause performance of:

detecting, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a second hand gesture; and

in accordance with a determination that the second hand gesture is associated with a second vehicle-control action, distinct from the first vehicle-control action:

determining a different device of the vehicle system associated with the second vehicle-control action, and

providing instructions to the vehicle system to cause performance of the second vehicle-control action at the different device of the vehicle system.

19 . The wrist-wearable device of claim 18 , wherein the wrist-wearable device is further configured to perform or cause performance of:

detecting, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a third hand gesture; and

in accordance with a determination that the third hand gesture is associated with a device-cycle control action that causes the vehicle system to alternate between devices of the vehicle system, providing instructions to the vehicle system to cause selection of another device of the vehicle system for receiving one or more vehicle-control actions.

20 . The wrist-wearable device of claim 17 , wherein the hand gesture is a first hand gesture and the vehicle-control action is a first vehicle-control action, and the wrist-wearable device is further configured to perform or cause performance of:

detecting, via the one or more biopotential sensors of the wrist-wearable device while it is worn by the user, a fourth hand gesture performed by the user; and

in accordance with a determination that the fourth hand gesture is associated with a vehicle-monitoring action:

receiving from, the vehicle system, current vehicle information, and

causing presentation of the current vehicle information at the wrist-wearable device or at a device in communication with the wrist-wearable device.