Object-model based event detection system
Example embodiments described herein therefore relate to an object-model based event detection system that comprises a plurality of sensor devices, to perform operations that include: generating sensor data at the plurality of sensor devices; accessing the sensor data generated by the plurality of sensor devices; detecting an event, or precursor to an event, based on the sensor data, wherein the detected event corresponds to an event category; accessing an object model associated with the event type in response to detecting the event, wherein the object model defines a procedure to be applied by the event detection system to the sensor data; and streaming at least a portion of a plurality of data streams generated by the plurality of sensor devices to a server system based on the procedure, wherein the server system may perform further analysis or visualization based on the portion of the plurality of data streams.
1. A system comprising:
a plurality of sensor devices that generate sensor data that comprises a plurality of data streams;
a memory; and
at least one hardware processor to perform operations comprising:
accessing a first data stream from among the plurality of data streams, the first data stream comprising a video stream generated by a first sensor device;
detecting an event precursor based on the first data stream, the event precursor including a feature within a frame of the video stream;
accessing an object model associated with the event precursor in response to the detecting the event precursor, the object model defining a data flow that corresponds with an event associated with the event precursor, the data flow including an identification of a second data stream generated by a second sensor device from among the plurality of sensor devices, the second data stream comprising inertial data;
selecting the second sensor device from among the plurality of sensor devices based on the data flow that includes the identification of the second data stream;
causing the system to access the second data stream from the second sensor device based on the identification of the portion of the plurality of data streams by the object model associated with the event precursor;
detecting the event associated with the event precursor based on at least the first data stream and the second data stream, the event corresponding to an event type;
performing a validation of the event based on the video data of the first data stream, and the inertial data of the second data stream;
identifying a portion of the plurality of data streams based on the data flow that corresponds with the event precursor in response to the validation of the event; and
causing the system to stream the portion of the plurality of data streams that corresponds with the event to a server system based on the data flow defined by the object model.
2. The system of claim 1 , wherein the plurality of data streams comprise one or more of:
an audio stream that comprises audio data;
an accelerometer stream that comprises the inertial data; and
a global positioning system stream that comprises location data.
3. The system of claim 2 , wherein the second data stream includes the accelerometer stream, and the detecting the event further comprises:
accessing inertial data from the accelerometer stream based on a timestamp associated with the frame based on the detecting the precursor to the event based on the feature within the frame of the video stream; and
detecting the event based on the inertial data and the frame of the video stream.
4. The system of claim 1 , wherein the first data stream is a first video stream, and:
the plurality of data streams include the first video stream and a second video stream, and the detecting the event includes detecting the event based on the first video stream, and the second data stream; and
the streaming at least the portion of the plurality of data streams to the server system based on the data-flow defined by the object model includes streaming the second video stream to the server system.
5. The system of claim 1 , wherein the streaming at least the portion of the plurality of data streams to the server system based on the data-flow defined by the object model includes:
selecting the portion of the plurality of data streams to the server system based on the data-flow defined by the object model.
6. The system of claim 1 , wherein:
the accessing of the sensor data includes accessing the sensor data generated by the at least one sensor device at a first data rate;
the object model associated with the event type defines a second data rate; and
the streaming of the portion of the plurality of data streams to the server system includes streaming the portion of the plurality of data streams at the second data rate based on the detecting the event.
7. The system of claim 1 , wherein the sensor data includes video data, and the detecting the event includes:
extracting a set of features from the video data;
applying the set of features from the video data to a machine learned model; and
detecting the event based on the machine learned model.
8. The system of claim 1 , wherein the sensor data includes location data and speed data, and the detecting the event based on the sensor data further comprises:
determining a location based on the location data;
accessing a repository that includes a speed limit associated with the location; and
performing a comparison of the speed data and the speed limit associated with the location.
9. The system of claim 8 , wherein the at least one hardware processor performs operations further comprising:
retrieving a threshold value associated with the speed limit;
determining that the speed data transgresses the threshold value; and
presenting a notification at a client device based on determining that the speed data transgresses the threshold value, the notification including a display of the speed data and an identifier associated with the sensor device.
10. A method comprising:
accessing a first data stream from among a plurality of data streams, the first data stream comprising a video stream generated by a first sensor device;
detecting an event precursor based on the first data stream, the event precursor including a feature within a frame of the video stream;
accessing an object model associated with the event precursor in response to the detecting the event precursor, the object model defining a data flow that corresponds with an event associated with the event precursor, the data flow including an identification of a second data stream generated by a second sensor device from among the plurality of sensor devices, the second data stream comprising inertial data;
selecting the second sensor device from among the plurality of sensor devices based on the data flow that includes the identification of the second data stream;
causing the system to access the second data stream from the second sensor device based on the identification of the portion of the plurality of data streams by the object model associated with the event precursor;
detecting the event associated with the event precursor based on at least the first data stream and the second data stream, the event corresponding to an event type;
performing a validation of the event based on the video data of the first data stream, and the inertial data of the second data stream;
identifying a portion of the plurality of data streams based on the data flow that corresponds with the event precursor in response to the validation of the event; and
causing the system to stream the portion of the plurality of data streams that corresponds with the event to a server system based on the data flow defined by the object model.
11. The method of claim 10 , wherein the plurality of data streams comprise one or more of:
an audio stream that comprises audio data;
an accelerometer stream that comprises the inertial data; and
a global positioning system stream that comprises location data.
12. The method of claim 11 , wherein the second data stream includes the accelerometer stream, and the detecting the event further comprises:
accessing inertial data from the accelerometer stream based on a timestamp associated with the frame based on the detecting the precursor to the event based on the feature within the frame of the video stream; and
detecting the event based on the inertial data and the frame of the video stream.
13. The method of claim 11 , wherein the first data stream is a first video stream, and:
the plurality of data streams include the first video stream and a second video stream, and
the detecting the event includes detecting the event based on the first video stream and the second data stream; and
the streaming at least the portion of the plurality of data streams to the server system based on the data-flow defined by the object model includes streaming the second video stream to the server system.
14. The method of claim 10 , wherein the streaming at least the portion of the plurality of data streams to the server system based on the data-flow defined by the object model includes:
selecting the portion of the plurality of data streams to the server system based on the data-flow defined by the object model.
15. The method of claim 10 , wherein:
the accessing of the sensor data includes accessing the sensor data generated by the at least one sensor device at a first data rate;
the object model associated with the event type defines a second data rate; and
the streaming of the portion of the plurality of data streams to the server system includes streaming the portion of the plurality of data streams at the second data rate based on the detecting the event.
16. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
accessing a first data stream from among a plurality of data streams, the first data stream comprising a video stream generated by a first sensor device;
detecting an event precursor based on the first data stream, the event precursor including a feature within a frame of the video stream;
accessing an object model associated with the event precursor in response to the detecting the event precursor, the object model defining a data flow that corresponds with an event associated with the event precursor, the data flow including an identification of a second data stream generated by a second sensor device from among the plurality of sensor devices, the second data stream comprising inertial data;
selecting the second sensor device from among the plurality of sensor devices based on the data flow that includes the identification of the second data stream;
causing the system to access the second data stream from the second sensor device based on the identification of the portion of the plurality of data streams by the object model associated with the event precursor;
detecting the event associated with the event precursor based on at least the first data stream and the second data stream, the event corresponding to an event type;
performing a validation of the event based on the video data of the first data stream, and the inertial data of the second data stream;
identifying a portion of the plurality of data streams based on the data flow that corresponds with the event precursor in response to the validation of the event; and
causing the system to stream the portion of the plurality of data streams that corresponds with the event to a server system based on the data flow defined by the object model.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the plurality of data streams comprise one or more of:
an audio stream that comprises audio data;
an accelerometer stream that comprises the inertial data; and
a global positioning system stream that comprises location data.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the second data stream includes the accelerometer stream, and the detecting the event further comprises:
accessing inertial data from the accelerometer stream based on a timestamp associated with the frame based on the detecting the precursor to the event based on the feature within the frame of the video stream; and
detecting the event based on the inertial data and the frame of the video stream.