IP Library Granted Patent US 10,528,832
Granted Patent B2
US 10,528,832 · App. 15/954,855 · Granted Jan 7, 2020

Methods and systems for processing driver attention data

Inventors: Upali P. Mudalige (Oakland Township, MI); Donald K. Grimm (Utica, MI); Wende Zhang (Birmingham, MI)
Assignee: GM GLOBAL TECHNOLOGY OPERATIONS LLC
G06K9/00845G06F3/013G06K9/00805G06T2207/30201G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,528,832
App. No.
15/954,855
Granted
Jan 7, 2020
Kind
B2
Abstract

Methods and systems are provided for processing attention data. In one embodiment, a method includes: receiving, by a processor, object data associated with at least one object of an exterior environment of the vehicle; receiving upcoming behavior data determined from a planned route of the vehicle; receiving gaze data sensed from an occupant of the vehicle; processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle; and selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score.

Claims (38)

1. A method of processing attention data, comprising:

receiving, by a processor, object data associated with at least one object of an exterior environment of a vehicle;

receiving, by the processor, upcoming behavior data determined from a planned route of the vehicle;

receiving, by the processor, gaze data sensed from an occupant of the vehicle;

processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle,

wherein the processing comprises:

computing an object location from the object data, computing a gaze location from the gaze data, computing an upcoming behavior location from the upcoming behavior data, and computing the attention score based on the gaze location, the object location, and the upcoming behavior location, and wherein the computing the attention score comprises computing a matching level between the object location and the gaze location;

selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score.

2. The method of claim 1 , wherein the computing the attention score further comprises applying a weighting factor to the matching level.

3. The method of claim 2 , wherein the weighting factor is dynamically determined based on environmental conditions.

4. The method of claim 2 , wherein the weighting factor is dynamically determined based on a driving scenario.

5. The method of claim 1 , wherein the computing the matching level is based on a duration of matching.

6. The method of claim 1 , wherein the computing the matching level is based on a breadth of matching.

7. The method of claim 1 , further comprising determining a distraction level based on a matching between the gaze location and a distraction region, and wherein the attention score is further determined based on the distraction level.

8. The method of claim 7 , further comprising applying a weighting factor to the distraction level and wherein the attention score is further based on the weighted distraction level.

9. The method of claim 8 , wherein the weighting factor is dynamically determined based on environmental conditions.

10. The method of claim 8 , wherein the weighting factor is dynamically determined based on a driving scenario.

11. The method of claim 1 , further comprising receiving gaze data from occupants of other vehicles at a same location; determining a general gaze direction based on the gaze data from the occupants of other vehicles; and wherein the processing the object data, the upcoming behavior data, and the gaze data to determine the attention score further comprises processing the object data, the upcoming behavior data, the gaze data from the occupant, and the general gaze direction to determine the attention score associated with an attention of the occupant of the vehicle.

12. A method of processing attention data, comprising:

receiving, by a processor, object data associated with at least one object of an exterior environment of a vehicle;

receiving, by the processor, upcoming behavior data determined from a planned route of the vehicle;

receiving, by the processor, gaze data sensed from an occupant of the vehicle;

processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle,

wherein the processing comprises:

computing an object location from the object data, computing a gaze location from the gaze data, computing an upcoming behavior location from the upcoming behavior data, and computing the attention score based on the gaze location, the object location, and the upcoming behavior location;

selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score, wherein the computing the attention score comprises computing a matching level between the upcoming behavior location and the gaze location.

13. The method of claim 12 , wherein the computing the attention score further comprises applying a weighting factor to the matching level.

14. The method of claim 12 , wherein the computing the matching level is based on a duration of matching.

15. The method of claim 12 , wherein the computing the matching level is based on a breadth of matching.

16. A system for processing attention data, comprising:

at least one sensor device that senses an exterior environment of a vehicle and provides sensor data; and

a non-transitory computer module that, by a processor,

processes the sensor data for object data associated with at least one object of an exterior environment of a vehicle and to determine an object location;

processes upcoming behavior data determined from a planned route of the vehicle to determine an upcoming behavior location;

processes gaze data sensed from an occupant of the vehicle to determine a gaze location;

processes the object location, the upcoming behavior location, and the gaze location to determine an attention score associated with an attention of the occupant of the vehicle, wherein the attention score is computed based on a matching level between the object location and the gaze location and a matching level between the upcoming behavior location and the gaze location; and

generates signals to at least one of notify the occupant and control the vehicle based on the attention score.

17. The system of claim 16 , wherein the non-transitory computer module computes the attention score based on a weighting factor applied to the matching level between the object location and the gaze location and a weighting factor applied to the matching level between the upcoming behavior location and the gaze location.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 17, 2018
From: MUDALIGE, UPALI P.; GRIMM, DONALD K.; ZHANG, WENDE
To: GM GLOBAL TECHNOLOGY OPERATIONS LLC
Reel/Frame 045560/0516 →
Continuity (1)
Related Publication 20190318180A1 · Oct 17, 2019
Cited By (1)
US 12,384,396