Automotive sensor fusion of radar, LIDAR, camera systems with improved safety by use of machine learning

1 . A vehicle safety sensor system comprising:

a processor;

a first radar sensor configuration to scan a first wide area;

a first auxiliary sensor configuration to scan a second wide area, wherein the first radar sensor and the first auxiliary sensor are configured to scan the first wide area and second wide area for detecting a moving object with respect to the vehicle, in conjunction with a sensor that measures travelling speed of the vehicle;

a second radar sensor configuration to scan a first narrow area containing the detected moving object, wherein the first narrow area is smaller than the first wide area; and

a second auxiliary sensor configuration to scan a second narrow area containing the detected moving object, wherein the second narrow area is smaller than the second wide area, wherein the second auxiliary sensor configuration comprises at least one sensor operating at a higher resolution than the sensors of the first auxiliary sensor configuration,

wherein data from the first radar sensor configuration and data from the first auxiliary sensor configuration is fused together to provide a first set of fused data;

wherein the system is configured to operate in a first mode when the processor does not detect an object from the first set of fused data,

wherein the system is configured to operate in a second mode when an object has been detected in the first mode,

wherein data from the second radar sensor configuration and data from second auxiliary sensor configuration is fused together to provide a second set of fused data,

wherein the processor is configured to monitor the detected objects based on the second set of fused data,

wherein the processor is further configured to determine a risk of collision with the detected moving object based on the second set of fused data, and

wherein determining the risk of collision comprises selecting a classification for the moving object from a plurality of classifications.

2 . The system of claim 1 , wherein:

the first auxiliary sensor configuration comprises an array of a plurality of optical sensors and the optical sensors detect visible spectrum electromagnetic radiation; and

the second auxiliary sensor configuration comprises an array of a plurality of optical sensors and the optical sensors detect visible spectrum electromagnetic radiation.

3 . The system of claim 1 comprising a LIDAR sensor.

4 . The system of claim 1 , wherein the system is coupled to a global satellite navigation system antenna (GNSS).

5 . The system of claim 4 , wherein at least one sensor is coupled to a GNSS antenna and data provided by the at least one sensor comprises a timestamp derived from a received GNSS signal.

6 . The system of claim 4 , wherein the processor is coupled to a GNSS antenna and positional data in a GNSS signal received by the GNSS antenna is used to control at least one sensor.

7 . The system of claim 1 , wherein the processor is coupled to one or more environmental sensors, wherein environmental data provided by the one or more environmental sensors is used to determine a confidence weighting for at least one sensor, wherein the confidence weighting is indicative of the sensor's accuracy in a detected environmental condition.

8 . The system of claim 7 , wherein the environmental sensor is one or more of:

a light sensor or a precipitation sensor, and preferably wherein the processor is further coupled to one or more of: a compass, a wheel odometer, or a gyroscope.

9 . The system of claim 1 , wherein:

the second auxiliary sensor configuration comprises at least one sensor operating at a higher resolution than the sensors of the first auxiliary sensor configuration; and

the second auxiliary sensor configuration is configured to process a narrow area containing the detected moving object, wherein the narrow area scanned by the second auxiliary sensor configuration is smaller than the area scanned by the first auxiliary sensor configuration.

10 . The system of claim 1 , wherein the first radar configuration is a radar operating in a first mode and the second radar configuration is the radar operating in a second mode.

11 . The system of claim 1 , wherein the first auxiliary sensor configuration is an array of optical sensors operating in a first mode and the second auxiliary sensor configuration is the array of optical sensors operating in a second mode.

12 . The system of claim 1 , wherein selecting a classification comprises extracting the classification from a look up table using one or more of: a calculated size, shape, or colour of the moving object.

13 . The system of claim 12 , wherein selecting a classification comprises using a neural network.

14 . The system of claim 12 , wherein selecting a classification comprises using a random decision forest.

15 . A method for improving safety of a vehicle comprising:

in a first mode:

scanning a first wide area with a first radar sensor configuration;

scanning a second wide area with a first auxiliary sensor configuration, wherein the first radar sensor and the first auxiliary sensor are configured to scan the first wide area and second wide area for detecting a moving object with respect to the vehicle, in conjunction with a sensor that measures travelling speed of the vehicle;

fusing data from the first radar sensor configuration with the data from the first auxiliary sensor configuration to provide a first set of fused data;

determining if an object is present based on the first set of data; and

if it is determined than an object is present in the first mode, switching to a second mode; and

in the second mode:

processing a first narrow area with a second radar sensor configuration, wherein the first narrow area is smaller than the first wide area;

processing a second narrow area with a second auxiliary sensor configuration, wherein the second narrow area is smaller than the second wide area, wherein the second auxiliary sensor configuration comprises at least one sensor operating at a higher resolution than the sensors of the first auxiliary sensor configuration;

fusing the data from the second radar sensor configuration and the data from second auxiliary sensor configuration to provide a second set of fused data; and

monitoring the detected moving object from the second set of fused data;

wherein monitoring comprises:

determining a risk classification, where the risk classification is indicative of the risk of collision with the detected moving object.

16 . The method of claim 15 , wherein monitoring further comprises:

if the risk classification meets a predetermined criterion, switching to a third mode wherein a vehicle safety sensor system controls the vehicle to avoid the detected moving object.

17 . A computer readable storage medium comprising instructions, which when executed by a processor coupled to a radar and an auxiliary sensor configuration, causes the processor to:

in a first mode:

scan a first wide area with a first radar sensor configuration;

scan a second wide area with a first auxiliary sensor configuration, wherein the first radar sensor and the first auxiliary sensor are configured to scan the first wide area and second wide area for detecting a moving object with respect to a vehicle, in conjunction with a sensor that measures travelling speed of the vehicle;

fuse data from the first radar sensor configuration with the data from the first auxiliary sensor configuration to provide a first set of fused data;

determine if an object is present based on the first set of data; and

if it is determined than an object is present in the first mode, switching to a second mode; and

in the second mode:

process a first narrow area with a second radar sensor configuration, wherein the first narrow area is smaller than the first wide area;

process a second narrow area with a second auxiliary sensor configuration, wherein the second narrow area is smaller than the second wide area, wherein the second auxiliary sensor configuration comprises at least one sensor operating at a higher resolution than the sensors of the first auxiliary sensor configuration;

fuse the data from the second radar sensor configuration and the data from second auxiliary sensor configuration to provide a second set of fused data; and

monitor the detected moving object from the second set of fused data;

wherein monitoring the detected moving object comprises:

determine a risk classification, where the risk classification is indicative of the risk of collision with the detected moving object.

18 . The computer readable storage medium of claim 17 , wherein the instructions further cause the processor to monitor the detected moving object further by switching to a third mode wherein a vehicle safety sensor system controls the vehicle to avoid the detected moving object if the risk classification meets a predetermined criterion.