Fault isolation for perception systems in autonomous/active safety vehicles

1. A method comprising:

obtaining, via one or more sensors, sensor data pertaining including one or more images of one or more detected objects in proximity to a vehicle;

processing the one or more images via a neural network model of a perception system of the vehicle, generating perception results for the one or more images;

analyzing, via a processor, a plurality of potential faults in the perception system, based on the perception results, using a respective different technique for each of the plurality of potential faults, for the neural network model;

determining a fault of the plurality of potential faults, via the processor, based on the analyzing; and

taking a vehicle control action with respect to the vehicle, based on the fault, via instructions provided by the processor.

2. The method of claim 1 , wherein the step of taking the vehicle control action comprise relinquishing control of one or more automated functions for the vehicle.

3. The method of claim 1 , wherein:

the analyzing of the plurality of faults comprises:

obtaining a ground truth for the one or more images;

generating one or more bounding boxes with respect to the perception results and the ground truth for the one or more images, comprising a respective bounding box for each of the one or more detected objects in the one or more images;

determining a bounding box correlation for each of the one or more bounding boxes; and

determining a respective class correlation between the perception results and the ground truth for each of the detected objects; and

the determining of the fault comprises determining whether a false positive fault, a false negative fault, or both for the one or more images are present, based on the bounding box correlations and the respective class correlations.

4. The method of claim 3 , wherein:

the analyzing further comprises determining an amount of time taken by the perception system in processing the one or more images; and

the determining of the fault further comprises determining whether a processing time fault is present, based on the amount of time.

5. The method of claim 3 , wherein:

the analyzing further comprises determining one or more differences between one or more variables between two consecutive frames of the images; and

the determining of the fault further comprises determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images.

6. The method of claim 3 , wherein:

the analyzing further comprises determining one or more differences in intensity between different pixels within a particular one of the images; and

the determining of the fault further comprises determining whether a spatial variance fault is present, based on the one or more differences in intensity.

7. The method of claim 3 , wherein:

the analyzing further comprises:

determining an amount of time taken by the perception system in processing the one or more images;

determining one or more differences between one or more variables between two consecutive frames of the images; and

determining one or more differences in intensity between different pixels within a particular one of the images; and

the determining of the fault further comprises:

determining whether a processing time fault is present, based on the amount of time;

determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images; and

determining whether a spatial variance fault is present, based on the one or more differences in intensity.

8. A system comprising:

one or more sensors configured to generate sensor data pertaining including one or more images of one or more detected objects in proximity to a vehicle; and

a processor coupled to the one or more sensors and configured to at least facilitate:

processing the one or more images via a neural network model of a perception system of the vehicle, generating perception results for the one or more images;

analyzing a plurality of potential faults in the perception system, based on the perception results, using a respective different technique for each of the plurality of potential faults, for the neural network model;

determining a fault of the plurality of potential faults based on the analyzing; and

providing instructions for taking a vehicle control action with respect to the vehicle, based on the fault.

9. The system of claim 8 , wherein the processor is configured to at least facilitate:

obtaining a ground truth for the one or more images;

generating one or more bounding boxes with respect to the perception results and the ground truth for the one or more images, comprising a respective bounding box for each of the one or more detected objects in the one or more images;

determining a bounding box correlation for each of the one or more bounding boxes;

determining a respective class correlation between the perception results and the ground truth for each of the detected objects; and

determining whether a false positive fault, a false negative fault, or both for the one or more images are present, based on the bounding box correlations and the respective class correlations.

10. The system of claim 9 , wherein the processor is configured to at least facilitate:

determining an amount of time taken by the perception system in processing the one or more images; and

determining whether a processing time fault is present, based on the amount of time.

11. The system of claim 9 , wherein the processor is configured to at least facilitate:

determining one or more differences between one or more variables between two consecutive frames of the images; and

determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images.

12. The system of claim 9 , wherein the processor is configured to at least facilitate:

determining one or more differences in intensity between different pixels within a particular one of the images; and

determining whether a spatial variance fault is present, based on the one or more differences in intensity.

13. The system of claim 9 , wherein the processor is further configured to at least facilitate:

determining an amount of time taken by the perception system in processing the one or more images;

determining one or more differences between one or more variables between two consecutive frames of the images;

determining one or more differences in intensity between different pixels within a particular one of the images;

determining whether a processing time fault is present, based on the amount of time;

determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images; and

determining whether a spatial variance fault is present, based on the one or more differences in intensity.

14. A vehicle having autonomous functionality, the vehicle comprising:

one or more sensors configured to generate sensor data pertaining including one or more images of one or more detected objects in proximity to the vehicle;

a processor coupled to the one or more sensors and configured to at least facilitate:

processing the one or more images via a neural network model of a perception system of the vehicle, generating perception results for the one or more images;

analyzing a plurality of potential faults in the perception system, based on the perception results, using a respective different technique for each of the plurality of potential faults, for the neural network model;

determining a fault of the plurality of potential faults based on the analyzing; and

providing instructions for taking a vehicle control action with respect to the vehicle, based on the fault; and

a vehicle actuator system coupled to the processor and configured to implement the vehicle control action in accordance with the instructions provided by the processor.

15. The vehicle of claim 14 , wherein:

the processor is configured to at least facilitate providing instructions for relinquishing control of one or more automated functions for the vehicle; and

the vehicle actuator system is configured to at least facilitate implementing the relinquishing of control of the one or more automated functions of the vehicle in accordance with the instructions provided by the processor.

16. The vehicle of claim 14 , wherein the processor is further configured to at least facilitate:

obtaining a ground truth for the one or more images;

generating one or more bounding boxes with respect to the perception results and the ground truth for the one or more images, comprising a respective bounding box for each of the one or more detected objects in the one or more images;

determining a bounding box correlation for each of the one or more bounding boxes;

determining a respective class correlation between the perception results and the ground truth for each of the detected objects; and

determining whether a false positive fault, a false negative fault, or both for the one or more images are present, based on the bounding box correlations and the respective class correlations.

17. The vehicle of claim 16 , wherein the processor is further configured to at least facilitate:

determining an amount of time taken by the perception system in processing the one or more images; and

determining whether a processing time fault is present, based on the amount of time.

18. The vehicle of claim 16 , wherein the processor is configured to at least facilitate:

determining one or more differences between one or more variables between two consecutive frames of the images; and

determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images.

19. The vehicle of claim 16 , wherein the processor is configured to at least facilitate:

determining one or more differences in intensity between different pixels within a particular one of the images; and

determining whether a spatial variance fault is present, based on the one or more differences in intensity.

20. The vehicle of claim 16 , wherein the processor is further configured to at least facilitate:

determining an amount of time taken by the perception system in processing the one or more images;

determining one or more differences between one or more variables between two consecutive frames of the images;

determining one or more differences in intensity between different pixels within a particular one of the images;

determining whether a processing time fault is present, based on the amount of time;

determining whether a temporal change fault is present, based on the one or more differences between the one or more variables between the two consecutive frames of the images; and

determining whether a spatial variance fault is present, based on the one or more differences in intensity.