IP Library › Granted Patent US 11,144,052
Granted Patent B2
US 11,144,052 · App. 16/212,785 · Granted Oct 12, 2021

Readiness and identification by gaze and/or gesture pattern detection

Inventor: Simon A. I. Stent (Cambridge, MA)
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
G05D1/0061B60W40/08B60W50/082B60W60/0053B60W60/0059G05D1/0088G06F3/013G06K9/00832B60W2420/42B60W2420/52B60W2540/00B60W2540/225B60W2540/229B60W2556/50B60W2900/00G05D1/0231G05D1/0278G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,144,052
App. No.
16/212,785
Granted
Oct 12, 2021
Kind
B2
Abstract

A vehicle control handoff system includes a controller comprising a processor and a non-transitory computer readable memory, one or more environment sensors and an imaging device communicatively coupled to the controller, and a machine-readable instruction set stored in the non-transitory computer readable memory of the controller. The machine-readable instruction set causes the system to: receive image data from at least one imaging device, receive one or more signals corresponding to an environment of a vehicle from the one or more environment sensors, define a gaze pattern comprising a first gaze direction corresponding to a first location within the environment of the vehicle, determine a first gaze based on the image data of the driver, determine whether the first gaze corresponds to at least one gaze direction of the gaze pattern, and transfer control of a vehicle operation from control by the controller to the driver in response to determining that the first gaze corresponds to the gaze pattern.

Claims (47)

1. A vehicle control handoff system comprising:

a controller comprising a processor;

one or more environment sensors communicatively coupled to the controller; and

at least one imaging device communicatively coupled to the controller, wherein the controller is configured to:

receive image data of a driver from the at least one imaging device;

receive one or more signals corresponding to an environment of a vehicle from the one or more environment sensors;

define a gaze pattern comprising a first gaze direction corresponding to a first location within the environment of the vehicle and a second gaze direction corresponding to a second location within the environment;

determine a first gaze and a first pointing gesture based on the image data of the driver;

determine whether the first gaze corresponds to the first gaze direction of the gaze pattern;

determine whether the first pointing gesture corresponds to the first gaze direction of the gaze pattern;

determine a second gaze and a second pointing gesture based on the image data of the driver;

determine whether the second gaze corresponds to the second gaze direction;

determine whether the second pointing gesture corresponds to the second gaze direction of the gaze pattern; and

transfer control of a vehicle operation from control by the controller to the driver in response to determining that the first gaze or the first pointing gesture corresponds to the first gaze direction of the gaze pattern and that the second gaze or the second pointing gesture corresponds to the second gaze direction of the gaze pattern.

2. The system of claim 1 , wherein the controller is further configured to:

determine an elapsed time from the first gaze of the driver corresponding to the first location, wherein when the elapsed time is greater than a predefined amount of time, the gaze pattern is redefined.

3. The system of claim 1 , wherein the gaze pattern includes a gaze upon a stoplight located in the environment of the vehicle.

4. The system of claim 1 , wherein the gaze pattern is not predefined by the driver.

5. The system of claim 1 , wherein controller is further configured to:

determine whether the first gaze of the driver is held for a predefined amount of time.

6. The system of claim 1 , wherein the transfer of control includes a transfer from an autonomous driving mode to a human driving mode.

7. The system of claim 1 , wherein the transfer of control includes a transfer from an autonomous driving mode to a human driving mode when the vehicle is stopped.

8. The system of claim 1 , wherein the gaze pattern is order independent.

9. The system of claim 1 , wherein the controller is further configured to:

update the gaze pattern in response to an elapsed amount of time or a change in the environment of the vehicle.

10. A method of vehicle control handoff, the method comprising,

receiving image data of a driver from at least one imaging device;

receiving one or more signals corresponding to an environment of a vehicle from one or more environment sensors;

defining a gaze pattern comprising a first gaze direction corresponding to a first location within the environment of the vehicle and a second gaze direction corresponding to a second location within the environment based on the one or more signals;

determining a first gaze and a first pointing gesture based on the image data of the driver;

determining whether the first gaze corresponds to the first gaze direction of the gaze pattern;

determining whether the first pointing gesture corresponds to the first gaze direction of the gaze pattern;

determining a second gaze and a second pointing gesture based on the image data of the driver;

determining whether the second gaze corresponds to the second gaze direction;

determining whether the second pointing gesture corresponds to the second gaze direction of the gaze pattern; and

transferring control of a vehicle operation from control by a controller to the driver in response to determining that the first gaze or the first pointing gesture corresponds to the first gaze direction of the gaze pattern and that the second gaze or the second pointing gesture corresponds to the second gaze direction of the gaze pattern.

11. The method of claim 10 , further comprising:

determining an elapsed time from the first gaze of the driver corresponding to the first location, wherein when the elapsed time is greater than a predefined amount of time, the gaze pattern is redefined.

12. The method of claim 10 , wherein the gaze pattern includes a gaze upon a stoplight located in the environment of the vehicle.

13. The method of claim 10 , wherein the gaze pattern is not predefined by the driver.

14. The method of claim 10 , further comprising:

determining whether the first gaze of the driver is held for a predefined amount of time.

15. The method of claim 10 , wherein the transfer of control includes a transfer from an autonomous driving mode to a human driving mode.

16. The method of claim 10 , wherein the transfer of control includes a transfer from an automated driving mode to a human driving mode when the vehicle is stopped.

17. The method of claim 10 , wherein the gaze pattern is order independent.

18. The method of claim 10 , further comprising:

updating the gaze pattern in response to an elapsed amount of time or a change in the environment of the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2021
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 058369/0420 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2018
From: STENT, SIMON A.I.
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 047702/0847 →
Continuity (1)
Related Publication 20200183383A1 · Jun 11, 2020