IP Library › Granted Patent US 11,663,836
Granted Patent B2
US 11,663,836 · App. 17/321,613 · Granted May 30, 2023

Eye gaze tracking calibration

Inventors: Ke Liu (Ypsilanti, MI); Ron M. Hecht (Raanana, IL); Omer Tsimhoni (Bloomfield Hills, MI); Michael Baltaxe (Raanana, IL)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
G06V20/597G06T7/55G06T7/74G06V20/56G06V40/19G06T2207/20081G06T2207/30252G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,663,836
App. No.
17/321,613
Granted
May 30, 2023
Kind
B2
Abstract

A system and method for error estimation in an eye gaze tracking system in a vehicle may include an operator monitoring system providing measured eye gaze information corresponding to an object outside the vehicle and an external object monitoring system providing theoretical eye gaze information and an error in the measured eye gaze information based upon the measured eye gaze information and the theoretical eye gaze information.

Claims (43)

1. An apparatus for error estimation in an eye gaze tracking system in a vehicle, comprising:

an operator monitoring system providing measured eye gaze information corresponding to an object outside the vehicle; and

an external object monitoring system providing theoretical eye gaze information and an error in the measured eye gaze information based upon the measured eye gaze information and the theoretical eye gaze information;

wherein the theoretical eye gaze information is determined by the external object monitoring system, the external object monitoring system;

determining a first direction of the object at an earlier first time and a second direction of the object at a later second time;

determining a relative displacement of the object between the first time and the second time;

determining a first depth of the object at the first time and a second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object; and

determining the theoretical eye gaze information at a selected one of the first time and the second time based on the corresponding one of the first depth of the object and the second depth of the object and the corresponding one of the first direction of the object and the second direction of the object.

2. The apparatus of claim 1 , wherein the measured eye gaze information comprises at least one of horizontal viewing angle information and vertical viewing angle information.

3. The apparatus of claim 1 , wherein the external object monitoring system comprises at least one forward looking camera.

4. A method for estimating an error in an eye gaze tracking system in a vehicle, comprising:

capturing, with a forward looking camera, a first exterior image at an earlier first time and a second exterior image at a later second time;

detecting within each of the first and second exterior images an object common to both the first and second exterior images whose image position has changed between the first exterior image and the second exterior image;

determining for each of the first and second exterior images respective first and second directions of the object;

determining a relative displacement of the object between the first time and the second time;

determining a first depth of the object at the first time and a second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object;

determining a theoretical eye gaze of an operator of the vehicle looking at the object at a selected one of the first time and the second time based on the corresponding one of the first depth of the object and the second depth of the object and the corresponding one of the first direction of the object and the second direction of the object;

receiving from the eye gaze tracking system a measured eye gaze of the operator of the vehicle looking at the object at the selected one of the first time and the second time; and

determining an error in the measured eye gaze of the operator based upon the measured eye gaze and the theoretical eye gaze.

5. The method of claim 4 , wherein determining the theoretical eye gaze of the operator of the vehicle looking at the object at one of the first time and the second time is further based on a separation between the operator's eyes and the forward looking camera.

6. The method of claim 4 , wherein the object is static and determining the relative displacement of the object between the first time and the second time is based upon displacement of the vehicle between the first time and the second time.

7. The method of claim 4 , wherein the object is dynamic and determining the relative displacement of the object between the first time and the second time is based upon displacement of the vehicle between the first time and the second time and displacement of the object between the first time and the second time.

8. The method of claim 4 , wherein determining the first depth of the object at the first time and the second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object comprises representing the relative displacement of the object as a vector and solving an injective function comprising the vector, the first direction of the object, and the second direction of the object.

9. The method of claim 4 , wherein the measured eye gaze of the operator comprises at least one of a horizontal viewing angle and a vertical viewing angle.

10. The method of claim 4 , wherein the first direction of the object and the second direction of the object each comprise at least one of a respective horizontal object angle and a vertical object angle.

11. The method of claim 4 , further comprising recalibrating the eye gaze tracking system based upon the determined error in the measured eye gaze of the operator.

12. The method of claim 4 , wherein determining the error comprises at least one of a comparison of the measured eye gaze with the theoretical eye gaze, a statistical model, and a machine learning model.

13. The method of claim 11 , wherein the recalibration occurs each vehicle cycle.

14. An apparatus for error estimation in an eye gaze tracking system in a vehicle, comprising:

an operator monitoring system providing measured eye gaze information corresponding to an object outside the vehicle; and

an external object monitoring system;

determining a first direction of the object at an earlier first time and a second direction of the object at a later second time,

determining a relative displacement of the object between the first time and the second time,

determining a first depth of the object at the first time and a second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object,

determining theoretical eye gaze information at a selected one of the first time and the second time based on the corresponding one of the first depth of the object and the second depth of the object and the corresponding one of the first direction of the object and the second direction of the object, and

determining error in the measured eye gaze based upon the measured eye gaze information and the determined theoretical eye gaze information;

wherein the determined error in the measured eye gaze provides an error estimation in the eye gaze tracking system.

15. The apparatus of claim 14 , wherein the measured eye gaze information comprises at least one of horizontal viewing angle information and vertical viewing angle information.

16. The apparatus of claim 14 , wherein the external object monitoring system comprises at least one forward looking camera.

17. The method of claim 1 , wherein the object is static and determining the relative displacement of the object between the first time and the second time is based upon displacement of the vehicle between the first time and the second time.

18. The method of claim 1 , wherein the object is dynamic and determining the relative displacement of the object between the first time and the second time is based upon displacement of the vehicle between the first time and the second time and displacement of the object between the first time and the second time.

19. The method of claim 1 , wherein determining the first depth of the object at the first time and the second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object comprises representing the relative displacement of the object as a vector and solving an injective function comprising the vector, the first direction of the object, and the second direction of the object.

20. The method of claim 14 , wherein determining the first depth of the object at the first time and the second depth of the object at the second time based on the first direction of the object, the second direction of the object, and the relative displacement of the object comprises representing the relative displacement of the object as a vector and solving an injective function comprising the vector, the first direction of the object, and the second direction of the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2021
From: LIU, KE; HECHT, RON M.; TSIMHONI, OMER; BALTAXE, MICHAEL
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 056260/0365 →
Continuity (1)
Related Publication 20220366178A1 · Nov 17, 2022