Intelligent detection of emergency vehicles
An autonomous vehicle detects an approaching emergency vehicle by performing a plurality of types of analysis of signals output by sensors in the vehicle. Each type of analysis generates a different prediction that indicates a likelihood of a proximity of an emergency vehicle to the autonomous vehicle. The predictions are processed and fused to generate a determination that indicates whether an emergency vehicle is proximate to the autonomous vehicle. In response to the determination indicating the emergency vehicle is proximate, the autonomous vehicle performs an action.
1. A method comprising:
processing a signal received from a sensor associated with an autonomous vehicle, the processing outputting a prediction indicating a likelihood that an emergency vehicle is present in an environment proximate to the autonomous vehicle;
receiving, at the autonomous vehicle, a vehicle-to-vehicle communication message from another vehicle that contains a notification of an approaching emergency vehicle;
generating a determination, based on the output prediction and the vehicle-to-vehicle communication message, that indicates whether an emergency vehicle is proximate to the autonomous vehicle;
in response to the determination indicating the emergency vehicle is proximate, causing the autonomous vehicle to perform an action;
wherein processing the signal received from the sensor associated with the autonomous vehicle comprises:
determining if behaviors of other vehicles in the environment of the autonomous vehicle indicate that the other vehicles are reacting to an approaching emergency vehicle;
in response to detecting behaviors that indicate the other vehicles are reacting to the approaching emergency vehicle, outputting the prediction indicating the likelihood that the emergency vehicle is present in the environment.
2. The method of claim 1 , further comprising:
receiving information from a remote computing device that indicates a path of an emergency vehicle;
wherein the determination indicating whether the emergency vehicle is proximate to the autonomous vehicle is further generated based on the information received from the remote computing device.
3. The method of claim 1 , wherein processing the signal received from the sensor associated with the autonomous vehicle further comprises:
determining that an emergency vehicle is likely approaching if traffic behind the autonomous vehicle all starts moving to a side of a road.
4. The method of claim 1 , further comprising:
training different machine learning models to recognize behaviors that apply to different types of roads or that comply with applicable traffic laws in different jurisdictions.
5. The method of claim 1 , further comprising:
comparing extracted lighting features with expected features of different types of emergency vehicles; and
determining a type of the emergency vehicle that is proximate to the autonomous vehicle according to the comparing.
6. The method of claim 1 , further comprising:
determining, in response to extracted lighting features match other types of lights, no emergency vehicle has been detected;
wherein the other types of lights comprises at least one of lights on maintenance vehicle, lights on construction vehicles, or traffic control lights.
7. The method of claim 1 , wherein processing the signal received from the sensor associated with the autonomous vehicle further comprises:
determining, in response to features of a sound signal received from a microphone in the autonomous vehicle matching expected siren features, whether a strength of the sound signal is increasing;
outputting, in response to the strength being increasing, an indication that the emergency vehicle has been detected.
8. The method of claim 7 , further comprising:
in response to the strength being constant or decreasing, determining that the emergency vehicle is not approaching a location of the autonomous vehicle and ignoring the sound signal at this time.
9. A non-transitory computer readable storage medium storing executable instructions, the instructions when executed by one or more processors causing the one or more processors to:
obtain computer vision data captured by sensors associated with one or more vehicles during operation of the one or more vehicles;
train a model using the obtained computer vision data, the model when trained configured to receive real-time computer vision data indicative of an environment of an autonomous vehicle and output a classification of a behavior of traffic in the environment as either indicative of a presence of an emergency vehicle or not indicative of a presence of an emergency vehicle;
receiving classifications output by the trained model during operation of a target vehicle and predictions indicating a likelihood that an emergency vehicle is proximate to the target vehicle, the predictions generated based on processing of a signal received from a sensor in the target vehicle; and
retraining the model based on a difference between the classifications and the predictions;
wherein the model when trained is further configured to:
determine if behaviors of other vehicles in the environment of the autonomous vehicle indicate that the other vehicles are reacting to an approaching emergency vehicle;
in response to detecting behaviors that indicate the other vehicles are reacting to the approaching emergency vehicle, output the prediction indicating the likelihood that the emergency vehicle is present in the environment proximate to the autonomous vehicle.
10. The non-transitory computer readable storage medium of claim 9 , wherein the instructions further cause the one or more processors to:
train a second model using the obtained computer vision data, the second model when trained configured to receive the real-time computer vision data indicative of the environment of the target vehicle and output a classification of vehicle lights detected in the environment as either indicative of a presence of an emergency vehicle or not indicative of a presence of an emergency vehicle.
11. The non-transitory computer readable storage medium of claim 10 , wherein the second model when trained further outputs a classification of a type of the emergency vehicle.
12. The non-transitory computer readable storage medium of claim 9 , wherein the instructions further cause the one or more processors to:
train a third model using the obtained computer vision data, the third model when trained configured to receive the real-time computer vision data indicative of the environment of the target vehicle and output a classification of a direction an emergency vehicle is traveling relative to the target vehicle.
13. An autonomous vehicle, comprising:
a plurality of sensors configured to measure parameters of an environment around the autonomous vehicle;
one or more processors; and
a non-transitory computer readable storage medium storing executable instructions, the instructions when executed by the one or more processors causing the one or more processors to:
perform a plurality of types of analysis of signals output by the plurality of sensors, each type of analysis generating a prediction that indicates a likelihood of a proximity of an emergency vehicle to the autonomous vehicle based on one or more of the signals;
process the generated predictions to generate a determination that indicates whether an emergency vehicle is proximate to the autonomous vehicle; and
in response to the determination indicating the emergency vehicle is proximate, cause the autonomous vehicle to perform an action;
wherein the instructions when executed by the one or more processors further causing the one or more processors to:
determining if behaviors of other vehicles in the environment of the autonomous vehicle indicate that the other vehicles are reacting to an approaching emergency vehicle;
in response to detecting behaviors that indicate the other vehicles are reacting to the approaching emergency vehicle, outputting the prediction indicating the likelihood that the emergency vehicle is present in the environment.
14. The autonomous vehicle of claim 13 , wherein the instructions further cause the one or more processors to:
receive information from a remote computing device that indicates a path of an emergency vehicle;
wherein processing the generated predictions further comprises using the received information to generate the determination indicating whether the emergency vehicle is proximate to the autonomous vehicle.
15. The autonomous vehicle of claim 13 , wherein the plurality of types of analysis comprise a siren sound detection analysis, and wherein performing the siren sound detection analysis comprises:
extracting one or more features from a sound signal received from a microphone in the autonomous vehicle;
comparing the extracted features to expected features of an emergency vehicle siren; and
in response to identifying a match between the extracted features and the expected features of the emergency vehicle siren, outputting a prediction that the emergency vehicle is proximate to the autonomous vehicle.
16. The autonomous vehicle of claim 13 , wherein the plurality of types of analysis comprise an emergency vehicle light detection analysis, and wherein performing the emergency vehicle light detection analysis comprises:
extracting lighting features from a perception data signal received from a computer vision system in the autonomous vehicle;
comparing the extracted lighting features to expected features of emergency vehicle lights; and
in response to identifying a match between the extracted lighting features and the expected features of emergency vehicle lights, outputting a prediction that the emergency vehicle is proximate to the autonomous vehicle.
17. The autonomous vehicle of claim 13 , wherein the plurality of types of analysis comprise a traffic behavior analysis, and wherein performing the traffic behavior analysis comprises:
providing perception data received from a computer vision system in the autonomous vehicle to a machine learning model, the machine learning model configured to output a classification of traffic behaviors captured in the perception data as indicative of or not indicative of an approaching emergency vehicle.
18. The autonomous vehicle of claim 13 , wherein the plurality of types of analysis comprise processing a vehicle-to-vehicle message received from another vehicle to determine if the vehicle-to-vehicle message contains an indication that an emergency vehicle is approaching the autonomous vehicle.
19. The autonomous vehicle of claim 13 , wherein processing the generated predictions to generate the determination comprises:
determining the emergency vehicle is proximate to the autonomous vehicle if a specified number of the types of analysis generated a prediction indicating an emergency vehicle is likely to be proximate to the autonomous vehicle.
20. The autonomous vehicle of claim 13 , wherein processing the generated predictions to generate the determination comprises:
determining the emergency vehicle is proximate to the autonomous vehicle if:
a first type of analysis generated a first prediction indicating a likely proximity of the emergency vehicle; and
a second type of analysis generated a second prediction indicating a likely proximity of the emergency vehicle.
21. The autonomous vehicle of claim 13 , wherein processing the generated predictions to generate the determination comprises:
applying a weighting function to the predictions generated by each of the plurality of types of analysis.