IP Library Granted Patent US 11,017,249
Granted Patent B2
US 11,017,249 · App. 15/882,581 · Granted May 25, 2021

Primary preview region and gaze based driver distraction detection

Inventors: Hai Yu (San Jose, CA); Fatih Porikli (San Jose, CA); Yuzhu Wu (Sunnyvale, CA)
Assignee: Futurewei Technologies, Inc.
G06K9/00845A61B5/0077A61B5/18A61B5/746B60W10/18B60W10/20B60W30/09B60W50/16G06F3/013G06K9/66H04N5/33B60W2050/143B60W2540/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,017,249
App. No.
15/882,581
Granted
May 25, 2021
Kind
B2
Abstract

A computer-implemented method of detecting distracted driving comprises: determining, by one or more processors, a primary preview region (PPR) in a representation of an environment; determining, by the one or more processors, a gaze point for a driver based on a sequence of images of the driver; determining, by the one or more processors, that the gaze point is outside of the PPR; based on the determined gaze point being outside of the PPR, decreasing, by the one or more processors, an attention level for the PPR; based on the attention level for the PPR, generating, by the one or more processors, an alert.

Claims (68)

1. A computer-implemented method of detecting distracted driving comprising:

determining, by one or more processors, a primary preview region (PPR) in a representation of an environment;

determining, by the one or more processors, a first gaze point for a driver based on a first sequence of images of the driver;

determining, by the one or more processors, that the first gaze point is inside of the PPR;

based on the first gaze point being inside of the PPR and a reaction coefficient of the driver, increasing, by the one or more processors, an attention level for the PPR using a first function;

determining, by the one or more processors, a second gaze point based on a second sequence of images of the driver;

determining, by the one or more processors, that the second gaze point is outside of the PPR;

based on the second gaze point being outside of the PPR and the reaction coefficient of the driver, decreasing, by the one or more processors, the attention level for the PPR using a second function different from the first function; and

based on the decreased attention level for the PPR, generating, by the one or more processors, an alert.

2. The method of claim 1 , wherein:

the PPR is a first PPR and is one of a plurality of PPRs, each PPR of the plurality of PPRs having a corresponding attention level;

the generating of the alert is further based on the attention level for each PPR of the plurality of PPRs; and

the method further comprises:

estimating a future path using vehicle and road information;

determining that the first PPR is not along the future path; and

based on the determination that the first PPR is not along the future path, removing the first PPR from the plurality of PPRs.

3. The method of claim 2 , further comprising:

determining a priority score for each PPR of the plurality of PPRs; and

wherein the attention level for each PPR of the plurality of PPRs is based on the priority score for the PPR.

4. The method of claim 1 , further comprising:

identifying an object depicted in the representation of the environment by analyzing the representation with a trained machine-learning algorithm.

5. The method of claim 1 , wherein the determining of the PPR comprises:

determining a primary preview point (PPP); and

determining the PPR based on the PPP and a predetermined radius.

6. The method of claim 1 , wherein the determining of the PPR in the representation of the environment comprises identifying a lane of a road.

7. The method of claim 1 , wherein the representation of the environment is generated by a laser scanner.

8. The method of claim 1 , wherein the generating of the alert comprises generating an audio alert.

9. The method of claim 1 , wherein the generating of the alert comprises generating a haptic alert.

10. The method of claim 1 , wherein the generating of the alert comprises activating brakes of a vehicle.

11. The method of claim 1 , wherein the generating of the alert comprises altering a direction of a vehicle.

12. The method of claim 1 , wherein the determining of the attention level for the PPR is based on a skill level of the driver.

13. The method of claim 1 , wherein the generating of the alert is further based on a predetermined threshold.

14. A system for detecting distracted driving, comprising:

a memory storage comprising instructions; and

one or more processors in communication with the memory, wherein the one or more processors execute the instructions to perform:

determining a primary preview region (PPR) in a representation of an environment;

determining a first gaze point for a driver based on a first sequence of images of the driver;

determining that the first gaze point is inside of the PPR;

based on the first gaze point being inside of the PPR and a reaction coefficient of the driver, increasing an attention level for the PPR using a first function;

determining a second gaze point based on a second sequence of images of the driver;

determining that the second gaze point is outside of the PPR;

based on the second gaze point being outside of the PPR and the reaction coefficient of the driver, decreasing the attention level for the PPR using a second function different from the first function; and

based on the decreased attention level for the PPR, generating an alert.

15. A non-transitory computer-readable medium that stores instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

determining a first primary preview region (PPR) in a representation of an environment;

determining a first gaze point for a driver based on a first sequence of images of the driver;

determining that the first gaze point is inside of the PPR;

based on the first gaze point being inside of the PPR and a reaction coefficient of the driver, increasing an attention level for the PPR using a first function;

determining a second gaze point based on a second sequence of images of the driver;

determining that the second gaze point is outside of the PPR;

based on the second gaze point being outside of the PPR and the reaction coefficient of the driver, decreasing the attention level for the PPR using a second function different from the first function; and

based on the decreased attention level for the PPR , generating an alert.

16. The system of claim 14 , wherein:

the PPR is a first PPR and is one of a plurality of PPRs, each PPR of the plurality of PPRs having a corresponding attention level;

the generating of the alert is further based on the attention level for each PPR of the plurality of PPRs; and

the one or more processors further execute the instructions to perform:

estimating a future path using vehicle and road information;

determining that the first PPR is not along the future path; and

based on the determination that the first PPR is not along the future path, removing the first PPR from the plurality of PPRs.

17. The computer-implemented method of claim 1 , wherein:

the first function is a logistic decay function; and

the second function is a logistic growing function.

18. The computer-implemented method of claim 17 , wherein:

a scaling factor of the logistic decay function is different than a scaling factor of the logistic growing function.

19. The computer-implemented method of claim 1 , further comprising:

determining a second PPR in the representation of the environment;

determining that the first gaze point is inside of the second PPR; and

based on the first gaze point being inside of the second PPR, increasing a second attention level for the second PPR using the first function.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2018
From: YU, HAI; PORIKLI, FATIH; WU, YUZHU
To: FUTUREWEI TECHNOLOGIES, INC.
Reel/Frame 044757/0878 →
Continuity (1)
Related Publication 20190236386A1 · Aug 1, 2019
Cited By (4)
US 12,346,498 US 12,450,924 US 12,545,281 US 12,709,278