Forward collision warning
Techniques are presented for the detection and management of collision warning (CW) events. A training dataset comprising videos of vehicle collisions and non-collisions, sensor readings, environmental conditions, and more is utilized to train a CW classification model for detecting potential collision events in vehicles. A backend CW classification model, with greater computational resources, employs a more complex neural network to review CW events received by the Behavioral Monitoring System (BMS) based on video data, achieving higher precision and reducing false positives. The CW model is installed in vehicles for real-time detection, while the backend model is deployed at the BMS. The BMS validates detected CW events, filters out false positives, and streamlines the review process for fleet administrators and customers. Additional BMS filtering operations include assessing non-proximity-related CW events and camera impairments, with the filtered CW events presented for review in the safety inbox.
1. A computer-implemented method comprising:
training a first collision warning (CW) classifier with training data comprising images of a road ahead taken from vehicles, speed of the vehicles, and labels indicating occurrence of a collision; and
periodically estimating a probability of a collision at a first vehicle, wherein estimating the probability of the collision comprises:
processing, by a frame feature extractor, a new image frame of an image of the road ahead of the first vehicle to obtain a new feature vector of the new image frame;
accessing a previous state comprising a plurality of previous feature vectors;
creating a current state by discarding, from the previous state, the previous feature vector of an oldest image frame and adding the new image frame to generate a plurality of current feature vectors;
providing as input the current state with the plurality of current feature vectors to the first CW classifier that outputs the probability of collision; and
generating an alert in the first vehicle based on the probability of collision.
2. The method as recited in claim 1 , further comprising:
sending, in response to the alert in the first vehicle, information about a CW event to a server, the information about the CW event comprising a video of the CW event captured by an outward camera.
3. The method as recited in claim 1 , further comprising:
utilizing a second CW classifier at a server to validate a CW event based on a video of the CW event received from the first vehicle; and
discarding the CW event at the server based on the validation.
4. The method as recited in claim 1 , further comprising:
analyzing, at a server, a video of a CW event to determine a position of a lead vehicle; and
performing first filtering of the CW event at the server based on the lead vehicle causing a detection of the CW event.
5. The method as recited in claim 1 , further comprising:
analyzing, at a server, a video of a CW event to determine occurrence of camera impairment while capturing the video; and
performing second filtering of the CW event at the server based on detection of the camera impairment.
6. The method as recited in claim 1 , wherein the processing of the new image frame by the frame feature extractor further comprises:
creating a pixel vector comprising pixel values of the new image frame; and
providing the pixel vector as input to the frame feature extractor that outputs the new feature vector.
7. The method as recited in claim 1 , wherein the training data comprises a first plurality of images associated with collisions and a second plurality of images associated with lack of collisions.
8. The method as recited in claim 1 , further comprising:
providing, by a server, a user interface (UI) to review CW events in a safety inbox configured to store driver behavior events.
9. The method as recited in claim 1 , wherein the image of the road ahead of the first vehicle is captured by an outward camera in a cam device installed on a windshield of the first vehicle.
10. The method as recited in claim 1 , wherein the new feature vector comprises 512 real numbers.
11. A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
training a first collision warning (CW) classifier with training data comprising images of a road ahead taken from vehicles, speed of the vehicles, and labels indicating occurrence of a collision; and
periodically estimating a probability of a collision at a first vehicle, wherein estimating the probability of the collision comprises:
processing, by a frame feature extractor, a new image frame of an image of the road ahead of the first vehicle to obtain a new feature vector of the new image frame;
accessing a previous state comprising a plurality of previous feature vectors:
creating a current state by discarding, from the previous state, the previous feature vector of an oldest image frame and adding the new image frame to generate a plurality of current feature vectors;
providing as input the current state with the plurality of current feature vectors to the first CW classifier that outputs the probability of collision; and
generating an alert in the first vehicle based on the probability of collision.
12. The system as recited in claim 11 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
sending, in response to the alert in the first vehicle, information about a CW event to a server, the information about the CW event comprising a video of the CW event captured by an outward camera.
13. The system as recited in claim 11 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
utilizing a second CW classifier at a server to validate a CW event based on a video of the CW event received from the first vehicle; and
discarding the CW event at the server based on the validation.
14. The system as recited in claim 11 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
analyzing, at a server, a video of a CW event to determine a position of a lead vehicle; and
performing first filtering of the CW event at the server based on the lead vehicle causing a detection of the CW event.
15. The system as recited in claim 11 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
analyzing, at a server, a video of a CW event to determine occurrence of camera impairment while capturing the video; and
performing second filtering of the CW event at the server based on detection of the camera impairment.
16. A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
training a first collision warning (CW) classifier with training data comprising images of a road ahead taken from vehicles, speed of the vehicles, and labels indicating occurrence of a collision; and
periodically estimating a probability of a collision at a first vehicle, wherein estimating the probability of the collision comprises:
processing, by a frame feature extractor, a new image frame of an image of the road ahead of the first vehicle to obtain a new feature vector of the new image frame;
accessing a previous state comprising a plurality of previous feature vectors;
creating a current state by discarding, from the previous state, the previous feature vector of an oldest image frame and adding the new image frame to generate a plurality of current feature vectors;
providing as input the current state with the plurality of current feature vectors to the first CW classifier that outputs the probability of collision; and
generating an alert in the first vehicle based on the probability of collision.
17. The non-transitory machine-readable storage medium as recited in claim 16 , wherein the machine further performs operations comprising:
sending, in response to the alert in the first vehicle, information about a CW event to a server, the information about the CW event comprising a video of the CW event captured by an outward camera.
18. The non-transitory machine-readable storage medium as recited in claim 16 , wherein the machine further performs operations comprising:
utilizing a second CW classifier at a server to validate a CW event based on a video of the CW event received from the first vehicle; and
discarding the CW event at the server based on the validation.
19. The non-transitory machine-readable storage medium as recited in claim 16 , wherein the machine further performs operations comprising:
analyzing, at a server, a video of a CW event to determine a position of a lead vehicle; and
performing first filtering of the CW event at the server based on the lead vehicle causing a detection of the CW event.
20. The non-transitory machine-readable storage medium as recited in claim 16 , wherein the machine further performs operations comprising:
analyzing, at a server, a video of a CW event to determine occurrence of camera impairment while capturing the video; and
performing second filtering of the CW event at the server based on detection of the camera impairment.