Long term driving danger prediction system
Systems and methods are disclosed to assist a driver with a dangerous condition by creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices; adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information; creating a codebook of dangerous driving situations, each represented as graphs; performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook; determining a distance metric between each selected codebook graphs and the matching subgraph of the current driving situation; from codebook graphs with a low distance, determining potential dangers; and generating an alert if one or more of the codebook dangers are imminent.
1. A guidance system for a vehicle, comprising:
sensors to capture 3D representations of surrounding traffic participants and information regarding a driving situation;
a local map locating the vehicle; and
a processor coupled to the sensors and the local map, the processor executing code for:
creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices;
adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information;
creating a codebook of dangerous driving situations, each represented as graphs;
performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook;
defining a distance function between subgraphs and calculating the distance between each selected codebook graphs and the matching subgraph of the current driving situation;
from codebook graphs with a low distance, determining potential dangers;
generating an alert if one or more of the codebook dangers are imminent.
2. The system of claim 1 , comprising predicting a dangerous driving situation a few seconds before occurrence.
3. The system of claim 1 , comprising an input modality obtained from radar.
4. The system of claim 1 , comprising an input modality obtained from a video camera with a structure from motion 3D generator.
5. The system of claim 1 , comprising recognizing object types using computer vision on a video image as an input modality.
6. The system of claim 1 , comprising using GPS coordinates from a vehicle sensor combined with a GPS-enabled local map.
7. The system of claim 1 , comprising receiving as an input modality one or more of:
a driver status including sleepiness, aggressiveness and impairment;
a weather status including road surface, temperature, daylight, rain, snow;
a traffic situation proximal to the vehicle including roadwork, delays, accident; and
vehicle physical characteristics including length, width, weight, braking capabilities.
8. The system of claim 1 , wherein the codebook is populated by known dangerous driving situations.
9. The system of claim 1 , wherein the codebook is refined by using a database of recorded driving and unsupervised learning techniques to discover attribute ranges of traffic participant relations.
10. The system of claim 1 , wherein the distance function between subgraphs is learned by supervised machine learning using subgraph attributes as features.
11. The system of claim 10 , wherein one classifier per each codebook graph is trained using labeled data to predict a distance between two matching subgraphs.
12. The system of claim 1 , wherein the distance function between subgraphs is learned using supervised machine learning using statistics of the attributes and a single classifier is trained for the codebook.
13. The system of claim 1 , wherein the distance function is crafted for each codebook graph based on a particular set of attributes.
14. The system of claim 1 , where the distance function is a generic distance based on measured ranges of attributes.
15. The system of claim 1 , comprising sounding an audible alert, a 3D-localized audible alert to attract a driver's attention to a location of a potential danger, or a voice alert telling the driver about a danger.
16. The system of claim 1 , comprising displaying one or more of: a warning message on the dashboard, a warning message on the windshield, a warning hologram.
17. The system of claim 1 , comprising illuminating or projecting a highlight on the windshield, side windows or car interior to locate traffic participants involved in a dangerous situation.
18. The system of claim 1 , vibrating the steering wheel, brakes, accelerator or seat to warn of danger.
19. The system of claim 1 , comprising acting on the brakes or an ability of a driver to accelerate.
20. A method executed on a vehicle guidance system with sensors to capture 3D representations of surrounding traffic particpants and information regarding a driving situation to assist a driver with a dangerous condition, comprising:
creating a graph representation where traffic participants and static elements are the vertices and the edges are relations between pairs of vertices;
adding attributes to the vertices and edges of the graph based on information obtained on the driving vehicle, the traffic participants and additional information;
creating a codebook of dangerous driving situations, each represented as graphs;
performing subgraph matching between the graphs in the codebook and the graph representing a current driving situation to select a set of matching graphs from the codebook;
determining a distance function between each selected codebook graphs and the matching subgraph of the current driving situation;
from codebook graphs with a low distance, determining potential dangers;
generating an alert if one or more of the codebook dangers are imminent.