IP Library › Granted Patent US 10,336,252
Granted Patent B2
US 10,336,252 · App. 15/400,059 · Granted Jul 2, 2019

Long term driving danger prediction system

Inventor: Eric Cosatto (Red Bank, NJ)
Assignee: NEC Corporation
B60Q9/008B60Q5/006B60Q9/00B60W10/04B60W10/18B60W10/30B60W30/0953B60W30/0956B60W40/04B60W40/09B60W50/14B60W40/06B60W50/16B60W2040/0827B60W2050/143B60W2050/146B60W2420/42B60W2420/52B60W2540/22B60W2540/26B60W2540/30B60W2550/12B60W2550/20B60W2550/402B60W2710/18B60W2710/30B60W2720/106
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,336,252
App. No.
15/400,059
Granted
Jul 2, 2019
Kind
B2
Abstract

Systems and methods are disclosed to assist a driver with a dangerous condition by creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices; adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information; creating a codebook of dangerous driving situations, each represented as graphs; performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook; determining a distance metric between each selected codebook graphs and the matching subgraph of the current driving situation; from codebook graphs with a low distance, determining potential dangers; and generating an alert if one or more of the codebook dangers are imminent.

Claims (41)

1. A guidance system for a vehicle, comprising:

a processor executing a code for:

receiving an input from sensors, the input indicating surrounding traffic participants and information regarding a driving situation;

receiving a local map locating the vehicle;

creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices;

adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information;

creating or obtaining a codebook of dangerous driving situations, each represented as graphs;

performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook;

calculating a distance between each selected codebook graph and the matching subgraph of the current driving situation;

determining potential dangers from codebook graphs with a low distance; and

generating an alert if one or more of the codebook dangers are imminent.

2. The system of claim 1 , comprising predicting a dangerous driving situation a few seconds before occurrence.

3. The system of claim 1 , comprising an input modality obtained from radar.

4. The system of claim 1 , comprising an input modality obtained from a video camera with a structure from motion 3D generator.

5. The system of claim 1 , comprising recognizing object types using computer vision on a video image as an input modality.

6. The system of claim 1 , comprising using GPS coordinates from a vehicle sensor combined with a GPS-enabled local map.

7. The system of claim 1 , comprising receiving as an input modality one or more of:

a driver status including sleepiness, aggressiveness and impairment;

a weather status including road surface, temperature, daylight, rain, snow;

a traffic situation proximal to the vehicle including roadwork, delays, accident; and

vehicle physical characteristics including length, width, weight, braking capabilities.

8. The system of claim 1 , wherein the codebook is populated by known dangerous driving situations.

9. The system of claim 1 , wherein the codebook is refined by using a database of recorded driving and unsupervised learning techniques to discover attribute ranges of traffic participant relations.

10. The system of claim 1 , wherein a distance function is defined between subgraphs, wherein said distance function between subgraphs is learned by supervised machine learning using subgraph attributes as features.

11. The system of claim 10 , wherein one classifier per each codebook graph is trained using labeled data to predict a distance between two matching subgraphs.

12. The system of claim 1 , wherein a distance function is defined between subgraphs, wherein said distance function between subgraphs is learned using supervised machine learning using statistics of the attributes and a single classifier is trained for the codebook.

13. The system of claim 1 , wherein a distance function is defined between subgraphs, wherein said distance function is crafted for each codebook graph based on a particular set of attributes.

14. The system of claim 1 , where a distance function is defined between subgraphs, wherein said distance function is a generic distance based on measured ranges of attributes.

15. The system of claim 1 , comprising sounding an audible alert, a 3D-localized audible alert to attract a driver's attention to a location of a potential danger, or a voice alert telling the driver about a danger.

16. The system of claim 1 , comprising displaying one or more of: a warning message on the dashboard, a warning message on the windshield, a warning hologram.

17. The system of claim 1 , comprising illuminating or projecting a highlight on the windshield, side windows or car interior to locate traffic participants involved in a dangerous situation.

18. The system of claim 1 , vibrating the steering wheel, brakes, accelerator or seat to warn of danger.

19. The system of claim 1 , comprising acting on the brakes or an ability of a driver to accelerate.

20. A method executed on a vehicle guidance system with sensors to capture surrounding traffic participants and information regarding a driving situation to assist a driver with a dangerous condition, comprising:

creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices;

adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information;

creating or obtaining a codebook of dangerous driving situations, each represented as graphs;

performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook;

calculating a distance between each selected codebook graph and the matching subgraph of the current driving situation;

determining potential dangers from codebook graphs with a low distance; and;

generating an alert if one or more of the codebook dangers are imminent.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2019
From: NEC LABORATORIES AMERICA, INC.
To: NEC CORPORATION
Reel/Frame 049157/0916 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 27, 2017
From: COSATTO, ERIC
To: NEC LABORATORIES AMERICA, INC.
Reel/Frame 041385/0041 →
Continuity (3)
Division 15088494 · Apr 1, 2016
Provisional Application 62146565 · Apr 13, 2015
Related Publication 20170158129A1 · Jun 8, 2017