Machine-learned explainable object detection system and method
A method of detecting a way-side object includes receiving data frames from sensors mounted on a vehicle. Position data corresponding to the position of the vehicle is received. Object-of-interest data is retrieved from a database. The sensor data frames and the object-of-interest data are processed to determine a region-of-interest in the sensor data frame. A portion of the sensor data frames corresponding to the region of interest is processed using machine-learned object detection to identify a first object-of-interest. The portion of the sensor data frames corresponding to the region of interest is processed using computer vision methods to detect features of the expected object-of-interest and identifying the detected object-of-interest as explained when the features of the expected object-of-interest are detected. Explained object-of-interest data corresponding to the explained detected object-of-interest is output to a navigation system of the vehicle.
1. A method of detecting a way-side object comprising;
receiving sensor data frames corresponding to a first time from sensors mounted on a vehicle,
receiving position data corresponding to a first position of the vehicle at the first time;
retrieving object-of-interest data from a database, wherein the object-of-interest data corresponds to an expected object-of-interest detectable by the sensors at the first position of the vehicle;
processing the sensor data frames and the object-of-interest data to determine a region-of-interest in at least one frame of the sensor data frames;
processing a portion of the sensor data frames corresponding to the region of interest using machine-learned object detection to identify a first object-of-interest;
processing the portion of the sensor data frames corresponding to the region of interest using computer vision methods to detect features of the expected object-of-interest and identifying the detected object-of-interest as explained when the features of the expected object-of-interest are detected; and
outputting explained object-of-interest data corresponding to the explained detected object-of-interest to a navigation system of the vehicle.
2. The method of claim 1 , further comprising receiving and processing sensor data frames corresponding to a series of times to generate a series of explained object-of-interest data.
3. The method of claim 2 , wherein the series of explained object-of-interest is compared to validate the explained object-of-interest.
4. The method of claim 1 , wherein the machine-learned object detection is a deep convolutional neural network.
5. The method of claim 3 , further comprising comparing the validated explained object-of-interest with expected object-of-interest data to identify detection failures.
6. An explained machine learning object detection system, comprising:
an input module connected to sensors and a data server, wherein the input module receives data frames from the sensors corresponding to a first time and wherein the input module receives data corresponding to the sensors position and objects-of-interest detectable from the sensors position;
a region-of-interest module connected to the input module, wherein the region-of-interest module determines, using object-of-interest data, regions-of-interest of the data frames;
a machine-learned object detection module connected to the region-of-interest module, wherein the machine-learned object detection module uses the regions-of-interest of the data frames to detect objects in the regions of interest of the data frames; and
an explainable detection module connected to the machine-learned detection module, wherein the explainable detection module uses the regions of interest of the data frames where objects have been detected to identify objects of interest using computer vision processing and outputting explained detected object data.
7. The system of claim 6 , wherein the sensors are at least one of cameras, RADAR or LIDAR.
8. The system of claim 6 , further comprising a tracking module, connected to the explainable detection module and receiving explained detected object data, storing the explained detected object data in memory and validating an explained detected object.
9. The system of claim 8 , further comprising a tracking supervision module, connected to the tracking module and receiving validated explained detected object data, comparing the validated explained detected object data with expected object-of-interest data to identify detection failures.
10. The system of claim 6 , further comprising a detection supervision module connected to the machine-learned module and receiving object detections from the machine-learned module and comparing the object detection with expected object-of-interest data to identify detection failures.
11. A method of detecting a way-side object comprising;
receiving a sensor data frames corresponding to a first time from sensors mounted on a vehicle,
receiving position data corresponding to the first position of the vehicle at the first time;
retrieving object-of-interest data from a database, wherein the object-of-interest data corresponds to an expected object-of-interest detectable by the sensors at the first position of the vehicle;
processing the sensor data frames and the object-of-interest data to determine a region-of-interest in the sensor data frame;
processing a portion of the sensor data frames corresponding to the region of interest using machine-learned object detection to identify a first object-of-interest;
processing the portion of the sensor data frames corresponding to the region of interest using computer vision methods to detect features of the expected object-of-interest and identifying the detected object-of-interest as explained when the features of the expected object-of-interest are detected;
processing sensor data frames corresponding to a series of times to generate a series of explained object-of-interest data;
validating an object detection using the series of explained object-of-interest data; and
outputting explained object-of-interest data corresponding to the validated object detection to a navigation system of the vehicle.
12. The method of claim 11 , further comprising comparing the validated object detections with expected object-of-interest data to identify object detection failures.
13. The method of claim 12 , further comprising calibrating the sensors based on the object detection failures.
14. The method of claim 12 , further comprising calibrating the machine-learned object detection based on the object detection failures.
15. The method of claim 12 , wherein the expected object-of-interest data corresponds to a calibration object.
16. The method of claim 13 wherein the calibrating is performed by modifying one or more of an aperture, a shutter speed or a focal length.
17. The method of claim 14 wherein the calibrating is performed by modifying neural-network detection probability thresholds.
18. A method of detecting a way-side object comprising;
receiving sensor data frames corresponding to a first time from sensors mounted on a vehicle;
retrieving object-of-interest data from a database, wherein the object-of-interest data corresponds to an expected object-of-interest detectable by the sensors at a first position of the vehicle;
processing the sensor data frames and the object-of-interest data to determine a region-of-interest in at least one frame of the sensor data frames;
processing a portion of the sensor data frames corresponding to the region of interest using machine-learned object detection to identify a first object-of-interest;
processing the portion of the sensor data frames corresponding to the region of interest using computer vision methods to detect features of the expected object-of-interest and identifying the detected object-of-interest as explained when the features of the expected object-of-interest are detected; and
outputting explained object-of-interest data corresponding to the explained detected object-of-interest to a navigation system of the vehicle.
19. The method of detecting a way-side object of claim 18 , wherein the region of interest determined for the at least one frame of the sensor data frames is the entirety of at least one frame.
20. The method of detecting a way-side object of claim 18 , further comprising comparing the expected object-of-interest and detected object-of-interest to determine a failure rate and wherein the failure rate is used to identify a safety level.