Detecting dangerous driving situations by parsing a scene graph of radar detections

1. A computer-implemented method executed on a processor for training a neural network to recognize driving scenes from sensor data received from vehicle radar, the method comprising:

extracting substructures from the sensor data received from the vehicle radar to define a graph having a plurality of nodes and a plurality of edges, each of the substructures being a path from an initial node through one or more detected nodes up to a maximum predetermined depth;

constructing, by a construction module, a neural network for each extracted path, an initial point of each constructed neural network being a NULL bias vector learned during training;

concatenating the outputs of each of the constructed neural networks for each of the plurality of edges into a single vector describing a driving scene of a vehicle; and

classifying, by a classifying module, the single vector into a set of one or more driving scene situations involving the vehicle represented as a vector of numbers, each number corresponding to a probability of a particular driving scene situation being present in the driving scene.

2. The method of claim 1 , wherein the plurality of nodes are vehicles and/or persons, and the plurality of edges are relationships between vehicles and/or persons.

3. The method of claim 1 , wherein the graph is a set of detections that form observations of the driving scene.

4. The method of claim 1 , wherein each constructed neural network includes known information and differential information.

5. The method of claim 1 , wherein the outputs of the neural networks corresponding to each extracted substructure are concatenated by a component-wise operator to result in a single output vector.

6. The method of claim 5 , wherein the component-wise operator is a maximum operator.

7. The method of claim 1 , wherein the classification is performed by a neural network.

8. The method of claim 1 , wherein the sensor data are obtained from a radar detection unit providing relative positions and velocities of traffic participants.

9. The method of claim 1 , wherein the neural network for each node is duplicated from a single neural network and connected in a manner where a neural network output of a given node feeds into a neural network input of a next node, in combination with a set of known information at the node.

10. The method of claim 9 , wherein the neural network input at each node further receives a set of differential information comparing the information at the node to that at a previous node.

11. The method of claim 1 , wherein the driving scenes are continuously compared with predetermined driving scenes identified as leading to driving scene situations.

12. The method of claim 11 , wherein a notification to an initial node of the plurality of nodes is transmitted upon occurrence of a match between a current driving scene and a predetermined driving scene.

13. A system for training a neural network to recognize driving scenes from sensor data received from vehicle radar, the system comprising:

a memory; and

a processor in communication with the memory, wherein the processor is configured to:

extract substructures from the sensor data received from the vehicle radar to define a graph having a plurality of nodes and a plurality of edges, each of the substructures being a path from an initial node through one or more detected nodes up to a maximum predetermined depth;

construct, by a construction module, a neural network for each extracted path, an initial point of each constructed neural network being a NULL bias vector learned during training;

concatenate the outputs of each of the constructed neural networks for each of the plurality of edges into a single vector describing a driving scene of a vehicle; and

classify, by a classifying module, the single vector into a set of one or more driving scene situations involving the vehicle represented as a vector of numbers, each number corresponding to a probability of a particular driving scene situation being present in the driving scene.

14. The system of claim 13 ,

wherein the plurality of nodes are vehicles and/or persons, and the plurality of edges are relationships between vehicles and/or persons; and

wherein the graph is a set of detections that form observations of the driving scene.

15. The system of claim 13 , wherein each constructed neural network includes known information and differential information.

16. The system of claim 13 , wherein the outputs of the neural networks corresponding to each extracted substructure are concatenated by a component-wise operator to result in a single output vector.

17. The system of claim 16 , wherein the component-wise operator is a maximum operator.

18. The system of claim 13 , wherein the classification is performed by a neural network.

19. The system of claim 13 , wherein the sensor data are obtained from a radar detection unit providing relative positions and velocities of traffic participants.

20. The system of claim 13 ,

wherein the neural network for each node is duplicated from a single neural network and connected in a manner where a neural network output of a given node feeds into a neural network input of a next node, in combination with a set of known information at the node; and

wherein the neural network input at each node further receives a set of differential information comparing the information at the node to that at a previous node.