Dash cam with artificial intelligence safety event detection
A vehicle dash cam may be configured to execute one or more neural networks (and/or other artificial intelligence), such as based on input from one or more of the cameras and/or other sensors associated with the dash cam, to intelligently detect safety events in real-time. Detection of a safety event may trigger an in-cab alert to make the driver aware of the safety risk. The dash cam may include logic for determining which asset data to transmit to a backend server in response to detection of a safety event, as well as which asset data to transmit to the backend server in response to analysis of sensor data that did not trigger a safety event. The asset data transmitted to the backend server may be further analyzed to determine if further alerts should be provided to the driver and/or to a safety manager.
1. A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising:
accessing images of an area in front of a vehicle acquired by an outward-facing camera associated with the vehicle;
evaluating at least some of the accessed images with reference to each of a plurality of safety event criteria associated with a corresponding plurality of safety event types;
triggering a first safety event in response to determining that features of the at last some of the accessed images match a first safety event criteria associated with a first safety event type;
accessing an asset data structure including associations between safety event types and respective asset data, wherein a first asset data associated with the first safety event type is different than a second asset data associated with a second safety event type; and
initiating access to additional sensor data including asset data from one or more sensors associated with the vehicle, wherein the additional sensor data is identified based at least on a type of the triggered first safety event.
2. The computerized method of claim 1 , further comprising:
detecting, based at least on application of an event detection model to at least portions of the accessed images, features within the accessed images.
3. The computerized method of claim 2 , wherein said detecting features includes providing a confidence level associated with each detected feature.
4. The computerized method of claim 3 , wherein the features include one or more of: identification of an object in front of the vehicle, an eye pose of a driver, a head pose of the driver, identification of objects in an inward-facing video, identification of other objects in the outward-facing video, metadata associated with identified objects, accelerometer data, global positioning system (GPS) data, ECU data, and vehicle speed data.
5. The computerized method of claim 2 , wherein the features of the one or more accessed images include a distance to a vehicle or object in front of the vehicle.
6. The computerized method of claim 1 , wherein the first safety event indicates one or more of: a forward collision warning, a tailgating warning, a distracted driver warning, or a policy violation.
7. The computerized method of claim 1 , wherein the additional sensor data includes metadata associated with the first safety event type.
8. The computerized method of claim 7 , wherein the metadata for a forward collision warning and tailgating warning safety event types includes:
location of a lead vehicle in front of the vehicle; and
lead vehicle metadata associated with the lead vehicle.
9. The computerized method of claim 4 , wherein a default head pose is customized for the driver.
10. The computerized method of claim 1 , further comprising:
accessing, in real-time as images are acquired by an inward-facing camera, inward facing images of the driver area;
detecting, based at least on application of a second event detection model comprising one or more neural networks to at least portions of the accessed inward facing images, second features within the one or more inward facing images;
triggering a second safety event in response to determining that the second detected features match second safety event criteria associated with the second safety event;
determining second metadata based on the sensor data that is associated with the detected second safety event; and
transmitting the second metadata to the event analysis system server.
11. The computerized method of claim 10 , wherein the second features of the one or more accessed images include a head pose of the driver.
12. The computerized method of claim 11 , wherein the head pose of the driver indicates how far away from straight ahead the driver's head is positioned.
13. The computerized method of claim 1 , wherein the additional sensor data includes metadata indicating the first safety event type, wherein the first safety event type is one or more of a forward collision warning, a tailgating warning, a distracted driver warning, or a policy violation;
wherein the method further comprises:
accessing settings for determining portions of video data from one or more of the inward-facing camera and outward-facing to transmit to the event management system; and
generating and transmitting to an event management system the determined portions of video data associated with the detected safety event.
14. The computerized method of claim 13 , wherein generating the determined portions of video data associated with the detected safety event comprises trimming a portion of the video data and downsampling the trimmed portion of the video data.
15. The computerized method of claim 1 , wherein the computing system comprise one or more of a vehicle device attached to the vehicle or an event management system.
16. A computing system comprising:
one or more hardware computer processor;
one or more non-transitory computer readable medium having software instructions stored thereon, the software instructions executable by the hardware computer processor to cause the computing system to perform operations comprising:
accessing images of an area in front of a vehicle acquired by an outward-facing camera associated with the vehicle;
evaluating at least some of the accessed images with reference to each of a plurality of safety event criteria associated with a corresponding plurality of safety event types;
triggering a first safety event in response to determining that features of the at last some of the accessed images match a first safety event criteria associated with a first safety event type;
accessing an asset data structure including associations between safety event types and respective asset data, wherein a first asset data associated with the first safety event type is different than a second asset data associated with a second safety event type; and
initiating access to additional sensor data including asset data from one or more sensors associated with the vehicle, wherein the additional sensor data is identified based at least on a type of the triggered first safety event.
17. The computing system of claim 16 , wherein the operations further comprise:
detecting, based at least on application of an event detection model to at least portions of the accessed images, features within the accessed images.
18. The computing system of claim 16 , wherein said detecting features includes providing a confidence level associated with each detected feature.
19. The computing system of claim 16 , wherein the computing system comprises one or more of a vehicle device attached to the vehicle or an event management system.
20. The computing system of claim 16 , wherein the additional sensor data includes metadata associated with the first safety event type.