Functional safety in autonomous driving
Autonomous driving of a vehicle in which computerized perception by the vehicle, including of its environment and of itself (e.g., its egomotion), is used to autonomously drive the vehicle and, additionally, can also be used to provide feedback to enhance performance, safety, and/or other attributes of autonomous driving of the vehicle (e.g., when certain conditions affecting the vehicle are determined to exist by detecting patterns in or otherwise analyzing what is perceived by the vehicle), such as by adjusting autonomous driving of the vehicle, conveying messages regarding the vehicle, and/or performing other actions concerning the vehicle.
1. A system for autonomous driving of a vehicle, the system comprising:
an interface configured to receive data from sensors of the vehicle that include a camera and a lidar sensor; and
a processing entity comprising at least one processor and configured to:
provide perception information regarding perception of an environment of the vehicle and a state of the vehicle based on the data from the sensors, the perception information comprising a 3D model of the environment of the vehicle and information about a position of the vehicle;
generate control signals for autonomously driving the vehicle based on the 3D model of the environment of the vehicle and the information about the position of the vehicle; and
process the perception information, other than for generating the control signals for autonomously driving the vehicle based on the 3D model of the environment of the vehicle and the information about the position of the vehicle, to determine whether a predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle exists and, if so, perform an action concerning the vehicle based on the predefined condition, wherein, to determine that the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle exists, the processing entity is configured to detect a pattern in the perception information indicative of the predefined condition.
2. The system of claim 1 , wherein the processing entity comprises an artificial neural network trained to detect the pattern in the perception information indicative of the predefined condition.
3. The system of claim 1 , wherein the pattern in the perception information indicative of the predefined condition is in the 3D model of the environment of the vehicle.
4. The system of claim 1 , wherein the pattern in the perception information indicative of the predefined condition is in the information about the position of the vehicle.
5. The system of claim 1 , wherein the pattern in the perception information indicative of the predefined condition is in both the 3D model of the environment of the vehicle and the information about the position of the vehicle.
6. The system of claim 1 , wherein the pattern in the perception information indicative of the predefined condition is in neither of the 3D model of the environment of the vehicle nor the information about the position of the vehicle.
7. The system of claim 1 , wherein the pattern in the perception information indicative of the predefined condition arises from a combination of different ones of the sensors and is undetectable from any of the different ones of the sensors individually.
8. The system of claim 1 , wherein, to determine that the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle exists, the processing entity is configured to monitor temporal variation of the perception information.
9. The system of claim 1 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle is external to the vehicle and results from the environment of the vehicle.
10. The system of claim 9 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a road on which the vehicle travels.
11. The system of claim 10 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a slipperiness of the road.
12. The system of claim 10 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a roughness of the road.
13. The system of claim 10 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a shape of the road.
14. The system of claim 10 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a sinuosity of the road.
15. The system of claim 9 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to weather in the environment of the vehicle.
16. The system of claim 15 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to precipitation in the environment of the vehicle.
17. The system of claim 15 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to wind in the environment of the vehicle.
18. The system of claim 9 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to illumination in the environment of the vehicle.
19. The system of claim 1 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle is intrinsic to the vehicle and results from a component of the vehicle.
20. The system of claim 19 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to malfunction of the component of the vehicle.
21. The system of claim 19 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to deterioration of the component of the vehicle.
22. The system of claim 21 , wherein: the component of the vehicle is a tire of a wheel of the vehicle; and the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to wear of the tire.
23. The system of claim 21 , wherein: the component of the vehicle is a tire of a wheel of the vehicle; and the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to deflation of the tire.
24. The system of claim 19 , wherein the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to vibration of the component of the vehicle.
25. The system of claim 19 , wherein: the component of the vehicle is a steering system of the vehicle; and the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to a steering anomaly of the steering system.
26. The system of claim 19 , wherein: the component of the vehicle is a headlight of the vehicle; and the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle relates to the headlight of the vehicle not working properly.
27. The system of claim 1 , wherein the action concerning the vehicle comprises an adjustment of autonomous driving of the vehicle based on the predefined condition.
28. The system of claim 1 , wherein the action concerning the vehicle comprises generation of a signal directed to a component of the vehicle based on the predefined condition.
29. The system of claim 28 , wherein the component of the vehicle is a powertrain of the vehicle.
30. The system of claim 28 , wherein the component of the vehicle is a steering system of the vehicle.
31. The system of claim 1 , wherein the action concerning the vehicle comprises conveyance of a message regarding the vehicle.
32. The system of claim 31 , wherein the message regarding the vehicle is indicative of malfunction of a component of the vehicle.
33. A system for autonomous driving of a vehicle, the system comprising:
an interface configured to receive data from sensors of the vehicle that include a camera and a lidar sensor; and
a processing entity comprising at least one processor and configured to:
provide perception information regarding perception of an environment of the vehicle and a state of the vehicle based on the data from the sensors, the perception information comprising a 3D model of the environment of the vehicle and information about a position of the vehicle;
generate control signals for autonomously driving the vehicle based on the 3D model of the environment of the vehicle and the information about the position of the vehicle;
process the perception information to detect a pattern in the perception information indicative of a predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle; and
perform an action concerning the vehicle based on the predefined condition.
34. Non-transitory computer-readable media comprising instructions executable by a processing apparatus for autonomous driving of a vehicle, wherein the instructions, when executed by the processing apparatus, cause the processing apparatus to:
receive data from sensors of the vehicle that include a camera and a lidar sensor;
provide perception information regarding perception of an environment of the vehicle and a state of the vehicle based on the data from the sensors, the perception information comprising a 3D model of the environment of the vehicle and information about a position of the vehicle;
generate control signals for autonomously driving the vehicle based on the 3D model of the environment of the vehicle and the information about the position of the vehicle; and
process the perception information, other than for generating the control signals for autonomously driving the vehicle based on the 3D model of the environment of the vehicle and the information about the position of the vehicle, to determine whether a predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle exists and, if so, perform an action concerning the vehicle based on the predefined condition, wherein, to determine that the predefined condition affecting the vehicle and independent of any other vehicles, pedestrians and other objects of interest detected in the environment of the vehicle exists, the processing apparatus is caused to detect a pattern in the perception information indicative of the predefined condition.