IP Library › Granted Patent US 12704902

Granted Patent B2

US 12704902 · App. 18/644,326 · Granted Aug 11, 2026

Methods and systems for controlling vehicle functions with gaze

Inventors: Sean Brian Braley (Orillia, CA); Roeland Petrus Hubertus Vertegaal (Perth Road, CA)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

G06F3/013 B60K35/10 B60K35/22 B60K35/235 G06F3/167 B60K2360/148 B60K2360/149 B60K2360/164 B60K2360/167 B60K2360/18

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12704902
App. No.: 18/644,326
Granted: Aug 11, 2026
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A system for gaze-based control of vehicle functions and related methods includes eye contact sensors for monitoring one or both eyes of a user, manual input devices, and circuits configured for tracking a gaze of a user located within a vehicle, identifying a gaze condition from the eye contact sensors in response to determining the gaze of the user is directed to an object of interest in the vehicle for at least a predetermined dwell-time, receiving a manual input from the manual input devices, activating a command interaction associated with the object of interest in response to identifying the gaze condition and receiving the manual input, receiving, while the command interaction is active, a command associated with the object of interest from the user, and initiating a vehicle function based on the received command.

Claims (106)

1 . A method of gaze-based control of vehicle functions, the method comprising:

tracking, by a sensor associated with the vehicle, a gaze of a user located within the vehicle;

identifying a first gaze condition in response to determining that the gaze of the user is directed to a first object of interest in the vehicle for at least a first predetermined dwell-time;

receiving a first manual input from the user, the first manual input associated with the vehicle;

identifying a display device gaze condition in response to determining that the gaze of the user is directed to a display device in the vehicle for at least a device predetermined dwell-time;

activating a first command interaction in the display device associated with the first object of interest in response to identifying both the first gaze condition and the display device gaze condition and receiving the first manual input;

receiving, while the first command interaction is active, a first command associated with the first object of interest from the user;

initiating a first vehicle function based on the received first command;

deactivating the first command interaction in response to determining an absence of the first command prior to expiration of a time-out period;

identifying a pinning gaze condition in response to determining that the gaze of the user is directed to the first object of interest for at least a pinning dwell-time;

receiving a pinning manual input from the user, the pinning manual input associated with the vehicle; and

adding a pinned representation of a display element associated with the first object of interest in the display device in response to identifying the pinning gaze condition and receiving the pinning manual input.

2 . The method of claim 1 , wherein activating the first command interaction further comprises providing the user a visual or an audible indication that the first command interaction is activated.

3 . The method of claim 1 wherein the first command is a first voice command.

4 . The method of claim 1 further comprising:

identifying a second gaze condition in response to determining that the gaze of the user is directed to a second object of interest in the vehicle for at least a second predetermined dwell-time;

receiving a second manual input from the user, the second manual input associated with the vehicle;

activating a second command interaction associated with the second object of interest in response to identifying the second gaze condition and receiving the second manual input;

receiving, while the second command interaction is active, a second command associated with the second object of interest from the user; and

initiating a second vehicle function based on a combination of the received first and second commands.

5 . The method of claim 1 further comprising:

identifying a cancellation gaze condition in response to determining that the gaze of the user is directed to the first object of interest for at least a cancellation predetermined dwell-time;

receiving a cancellation manual input from the user, the cancellation manual input associated with the vehicle; and

deactivating the first command interaction in response to identifying the cancellation gaze condition and receiving the cancellation manual input.

6 . The method of claim 1 , wherein the first object of interest is at least one of a gauge cluster, a dash display, a heads up display, and an infotainment console.

7 . The method of claim 1 further comprising:

identifying a pinning removal gaze condition in response to determining that the gaze of the user is directed to the pinned representation for at least a removal dwell-time;

receiving a pinning removal manual input from the user, the pinning removal manual input associated with the vehicle; and

removing the pinned representation in the display device in response to identifying the pinning removal gaze condition and receiving the pinning removal manual input.

8 . The method of claim 1 further comprising:

identifying a pinned access gaze condition in response to determining that the gaze of the user is directed to the pinned representation for at least a pinned access dwell-time;

receiving a pinned access manual input from the user, the pinned access manual input associated with the vehicle;

activating a pinned command interaction in the display device in response to identifying the pinned access gaze condition and receiving the pinned access manual input;

receiving, while the pinned command interaction is active, a pinned command associated with the pinned representation from the user; and

initiating a pinned vehicle function based on the received pinned command.

9 . The method of claim 1 further comprising:

receiving a notification in the display device;

identifying a notification gaze condition in response to determining that the gaze of the user is directed to the display device for at least a notification predetermined dwell-time;

receiving a notification manual input from the user, the notification manual input associated with the vehicle; and

displaying contents of the notification in the display device in response to identifying the notification gaze condition and receiving the notification manual input.

10 . A system for gaze-based control of vehicle functions comprising:

one or more eye contact sensors for monitoring one or both eyes of a user;

one or more manual input devices; and

one or more processors configured to perform operations including:

tracking a gaze of a user located within a vehicle;

identifying a first gaze condition from the one or more eye contact sensors in response to determining that the gaze of the user is directed to a first object of interest in the vehicle for at least a first predetermined dwell-time;

receiving a first manual input from the one or more manual input devices;

identifying a display device gaze condition in response to determining that the gaze of the user is directed to a display device in the vehicle for at least a device predetermined dwell-time;

receiving, while the first command interaction is active, a first command associated with the first object of interest from the user; and

initiating a first vehicle function based on the received first command;

deactivating the first command interaction in response to determining an absence of the first command prior to expiration of a time-out period;

identifying a pinning gaze condition in response to determining that the gaze of the user is directed to the first object of interest for at least a pinning dwell-time;

receiving a pinning manual input from the user, the pinning manual input associated with the vehicle; and

11 . The system of claim 10 , wherein the one or more processors are configured to perform further operations including:

identifying a second gaze condition from the one or more eye contact sensors in response to determining that the gaze of the user is directed to a second object of interest in the vehicle for the predetermined dwell-time;

receiving a second manual input from the one or more manual input devices;

activating a second command interaction associated with the second object of interest in response to identifying the second gaze condition and receiving the second manual input;

receiving, while the second command interaction is active, a second command associated with the second object of interest from the user; and

initiating a second vehicle function based on a combination of the received first and second commands.

12 . The system of claim 10 , wherein the one or more processors are configured to perform further operations including:

identifying a cancellation gaze condition from the one or more eye contact sensors in response to determining that the gaze of the user is directed to the first object of interest for at least a cancellation predetermined dwell-time;

receiving a cancellation manual input from the one or more manual input devices; and

deactivating the first command interaction in response to identifying the cancellation gaze condition and receiving the cancellation manual input.

13 . The system of claim 10 wherein said identifying the first gaze condition in response to said determining that the gaze of the user is directed to the first object of interest in the vehicle for at least the first predetermined dwell-time comprises:

identifying a display device gaze condition in response to said determining that the gaze of the user is directed to a display device in the vehicle for at least a device predetermined dwell-time;

wherein said activating the first command interaction associated with the first object of interest in response to identifying the first gaze condition and receiving the first manual input comprises:

activating the first command interaction in the display device associated with the first object of interest in response to identifying both the first gaze condition and the display device gaze condition and receiving the first manual input.

14 . One or more non-transitory, computer-readable storage media comprising computer-executable instructions, wherein the instructions, when executed, cause one or more processors to perform actions comprising: