Devices and methods for assisting operation of vehicles based on situational assessment fusing expoential risks (safer)
An apparatus includes: a first sensor configured to provide a first input; a second sensor configured to provide a second input; and a processing unit configured to receive the first input from the first sensor, and a second input from the second sensor; wherein the processing unit is configured to determine a first probability of a first predicted event, and a second probability of a second predicted event, wherein the first predicted event and the second predicted event are associated with an operation of the vehicle; and wherein the processing unit is configured to calculate a risk score based on the first probability of the first predicted event, and based on the second probability of the second predicted event.
1. An apparatus comprising:
a first sensor configured to provide a first input;
a second sensor configured to provide a second input; and
a processing unit configured to receive the first input from the first sensor, and a second input from the second sensor;
wherein the processing unit is configured to determine a first probability of a first predicted event, and a second probability of a second predicted event, wherein the first predicted event and the second predicted event are associated with an operation of a vehicle;
wherein the processing unit is configured to calculate a risk score based on the first probability of the first predicted event, and based on the second probability of the second predicted event;
wherein the first predicted event is a collision event, and the second predicted event is a non-risky event; and
wherein the processing unit is configured to calculate the risk score based on the first probability of the collision event, and based on the second probability of the non-risky event.
2. The apparatus of claim 1 , wherein the second sensor comprises one or more sensing units configured to sense one or more characteristics of the vehicle.
3. The apparatus of claim 1 , wherein the processing unit is configured to calculate the risk score by applying a first weight to the first probability to obtain a first weighted probability, applying a second weight to the second probability to obtain a second weighted probability, and adding the first weighted probability and the second weighted probability.
4. The apparatus of claim 1 , wherein the processing unit is configured to determine a third probability of a third predicted event; and
wherein the processing unit is configured to calculate the risk score based on the first probability of the first predicted event, based on the second probability of the second predicted event, and based on the third probability of the third predicted event.
5. An apparatus comprising:
a first sensor configured to provide a first input;
a second sensor configured to provide a second input; and
a processing unit configured to receive the first input from the first sensor, and a second input from the second sensor;
wherein the processing unit is configured to determine a first probability of a first predicted event, a second probability of a second predicted event, and a third probability of a third predicted event, wherein the first predicted event and the second predicted event are associated with an operation of a vehicle;
wherein the processing unit is configured to calculate a risk score based on the first probability of the first predicted event, based on the second probability of the second predicted event, and based on the third probability of the third predicted event; and
wherein the first predicted event is a collision event, the second predicted event is a near-collision event, and the third predicted event is a non-risky event.
6. The apparatus of claim 4 , wherein the processing unit is configured to calculate the risk score by:
applying a first weight to the first probability to obtain a first weighted probability,
applying a second weight to the second probability to obtain a second weighted probability,
applying a third weight to the third probability to obtain a third weighted probability, and
adding the first weighted probability, the second weighted probability, and the third weighted probability.
7. The apparatus of claim 1 , wherein the first input and the second input comprise data obtained in past T seconds, and wherein the processing unit is configured to process the data obtained in the past T seconds to determine the first probability of the first predicted event and the second probability of the second predicted event, wherein T is at least 3 seconds.
8. The apparatus of claim 1 , wherein the first predicted event is for a future time that is at least 1 second from a current time.
9. The apparatus of claim 1 , wherein the processing unit is configured to calculate the risk score for a first time point, and wherein the processing unit is also configured to calculate an additional risk score for a second time point, and to determine a difference between the risk score and the additional risk score, wherein the difference indicates whether a risky situation is escalating or subsiding.
10. The apparatus of claim 1 , wherein the processing unit is configured to generate a control signal based on the risk score.
11. The apparatus of claim 10 , wherein the processing unit is configured to generate the control signal if the risk score meets a criterion.
12. The apparatus of claim 11 , wherein the processing unit is configured to generate the control signal for operating a device if the risk score meets the criterion.
13. The apparatus of claim 12 , wherein the device comprises:
a speaker for generating an alarm;
a display or a light-emitting device for providing a visual signal;
a haptic feedback device;
a collision avoidance system; or
a vehicle control for the vehicle.
14. The apparatus of claim 1 , wherein the processing unit is configured to determine the first probability of the first predicted event and the second probability of the second predicted event based on the first input and the second input.
15. An apparatus comprising:
a first sensor configured to provide a first input;
a second sensor configured to provide a second input; and
a processing unit configured to receive the first input from the first sensor, and a second input from the second sensor;
wherein the processing unit is configured to determine a first probability of a first predicted event, and a second probability of a second predicted event, wherein the first predicted event and the second predicted event are associated with an operation of a vehicle;
wherein the processing unit is configured to calculate a risk score based on the first probability of the first predicted event, and based on the second probability of the second predicted event; and
wherein the processing unit comprises a first-stage processing system and a second-stage processing system, wherein the first-stage processing system is configured to obtain the first input and to process the first input to provide first output, wherein the second-stage processing system is configured to obtain the first output and to process the first output to provide second output, wherein the first output has fewer dimensions or less complexity compared to the first input, and wherein the second output has fewer dimensions or less complexity compared to the first output.
16. The apparatus of claim 1 , wherein the processing unit comprises a neural network model.
17. The apparatus of claim 16 , wherein the neural network model is configured to receive a first time series of information indicating a first risk factor, and a second time series of information indicating a second risk factor.
18. The apparatus of claim 17 , wherein the neural network model is configured to receive the first time series and the second time series in parallel, and/or to process the first time series and the second time series in parallel.
19. The apparatus of claim 17 , wherein the processing unit is configured to package the first time series and the second time series into a data structure for feeding to the neural network model.
20. The apparatus of claim 19 , wherein the data structure comprises a two-dimensional matrix of data.
21. The apparatus of claim 17 , wherein the first time series indicates a condition outside the vehicle for different respective time points, and the second time series indicates a state of a driver and/or a state of the vehicle for the different respective time points.
22. The apparatus of claim 17 , wherein the first time series indicates a first characteristic of the vehicle for different respective time points, and the second time series indicates a second characteristic of the vehicle for the different respective time points.
23. The apparatus of claim 17 , wherein the first time series and second time series comprise one or a combination of two or more of: distance to lead vehicle, distance to intersection stop line, speed of the vehicle, time-to-collision, time-to-intersection-violation, estimated braking distance, information regarding road condition, information regarding special zone, information regarding environment, information regarding traffic condition, time of day, information regarding visibility condition, information regarding identified object, object position, object moving direction of, object speed, bounding box(es), operating parameter(s) of the vehicle, information regarding state(s) of a driver, information regarding driver history, time spent driving consecutively, proximity to meal times, information regarding accident history, and audio information.
24. The apparatus of claim 1 , wherein the first sensor comprises a camera, a Lidar, a radar, or any combination of the foregoing, configured to sense an environment outside a vehicle.
25. The apparatus of claim 1 , wherein the second sensor comprises a camera configured to view a driver of the vehicle.
26. The apparatus of claim 15 , wherein the first input comprises first images, and the second input comprises second images, and wherein the first-stage processing system is configured to receive the first images and the second images, process the first images to obtain a first time series of information, and process the second images to obtain a second time series of information.