SYSTEMS AND METHODS OF DATA STRUCTURING
Presented herein are system and methods for handling of sensor data in home security and automation applications to present as a single event multiple portions of the sensor data, and more particularly image data (e.g., clips), captured from a plurality of sensor devices, such as a plurality of image capture devices.
1 . An apparatus comprising:
one or more sensor devices to capture sensor data corresponding to an environment; and
one or more processors configured to execute instructions to perform operations to cause the apparatus to:
detect, using the one or more sensor devices, one or more events within an environment;
determine, using the sensor data captured by the one or more sensor devices, an attribute of the one or more events;
correlate, using the attributes, first sensor data of a first instance of the one or more events and second sensor data of a second instance of the one or more events; and
generate a single data structure of combined sensor data including the first sensor data and the second sensor data, the single data structure providing a unified representation based on the attribute of the one or more events.
2 . The apparatus of claim 1 , wherein the correlation of the first sensor data and the second sensor data comprises:
predicting, based on the attribute of the one or more events, that the one or more events is likely to involve another of the one or more sensor devices; and
confirming, according to the attribute, accuracy of the prediction by confirming correlation of sensor data of the other of the one or more sensor devices to the one or more events.
3 . The apparatus of claim 2 , wherein the attribute comprises an entity attribute detectable by the one or more sensor devices, the entity attribute comprising one or more of an entity size, an entity height, an entity appearance, an entity clothing style, an entity clothing color, and an entity intent, and
wherein the confirming accuracy of the prediction is based on the one or more of the entity size, the entity height, the entity appearance, the entity clothing style, the entity clothing color, the entity intent.
4 . The apparatus of claim 1 , wherein correlating the first sensor data and the second sensor data comprises:
identifying an entity in the first sensor data, according to an entity attribute of the entity; and
identifying the entity in the second sensor data, according to the entity attribute of the entity.
5 . The apparatus of claim 1 , wherein correlating the first sensor data and the second sensor data comprises:
correlating a timestamp of the first sensor data and a timestamp of the second sensor data.
6 . The apparatus of claim 1 , wherein the first sensor data and the second sensor data are contemporaneous, and
wherein the first sensor data and the second sensor data are arranged together, contemporaneously, in the single data structure.
7 . The apparatus of claim 1 , wherein the first sensor data and the second sensor data appear sequentially in the single data structure.
8 . The apparatus of claim 1 , wherein correlating the first sensor data and the second sensor data comprises applying a data analytics model that maps which data of the one or more sensor devices includes portions of data of other of the one or more sensor devices.
9 . The apparatus of claim 1 , wherein the one or more sensor devices include one or more microphones to capture audio data for the environment and one or more depth sensors to capture depth data for the environment, and
wherein correlating the first sensor data and the second sensor data is according to at least one of the audio data and the depth data.
10 . The apparatus of claim 1 , wherein the one or more processors are further configured to execute instructions to perform operations to cause the apparatus to:
generate a synopsis associated with the single data structure, the synopsis including additional data captured contemporaneous to the first sensor data and the second sensor data, the additional data including data other than the sensor data from the first sensor device and the sensor data from the second sensor device.
11 . A method comprising:
detecting, using sensor data from one or more sensor devices, one or more events within an environment;
determining, using the sensor data captured by the one or more sensor devices, an attribute of the one or more events;
correlating, using the attributes, first sensor data of a first instance of the one or more events and second sensor data of a second instance of the one or more events; and
generating a single data structure of combined sensor data including the first sensor data and the second sensor data, the single data structure providing a unified representation based on the attribute of the one or more events.
12 . The method of claim 11 , wherein the correlation of the first sensor data and the second sensor data comprises:
predicting, based on the attribute of the one or more events, that the one or more events is likely to involve another of the one or more sensor devices; and
confirming, according to the attribute, accuracy of the prediction by confirming correlation of sensor data of the other of the one or more sensor devices to the one or more events.
13 . The method of claim 12 , wherein the attribute comprises an entity attribute detectable by the one or more sensor devices, the entity attribute comprising one or more of an entity size, an entity height, an entity appearance, an entity clothing style, an entity clothing color, and an entity intent, and
wherein the confirming accuracy of the prediction is based on the one or more of the entity size, the entity height, the entity appearance, the entity clothing style, the entity clothing color, the entity intent.
14 . The method of claim 11 , wherein correlating the first sensor data and the second sensor data comprises:
identifying an entity in the first sensor data, according to an entity attribute of the entity; and
identifying the entity in the second sensor data, according to the entity attribute of the entity.
15 . The method of claim 11 , wherein correlating the first sensor data and the second sensor data comprises:
correlating a timestamp of the first sensor data and a timestamp of the second sensor data.
16 . The method of claim 11 , wherein the first sensor data and the second sensor data are contemporaneous, and
wherein the first sensor data and the second sensor data are arranged together, contemporaneously, in the single data structure.
17 . The method of claim 11 , wherein the first sensor data and the second sensor data appear sequentially in the single data structure.
18 . The method of claim 11 , wherein correlating the first sensor data and the second sensor data comprises applying a data analytics model that maps which data of the one or more sensor devices includes portions of data of other of the one or more sensor devices.
19 . The method of claim 11 , wherein the one or more sensor devices include one or more microphones to capture audio data for the environment and one or more depth sensors to capture depth data for the environment, and
wherein correlating the first sensor data and the second sensor data is according to at least one of the audio data and the depth data.
20 . The method of claim 11 , further comprising:
generating a synopsis associated with the single data structure, the synopsis including additional data captured contemporaneous to the first sensor data and the second sensor data, the additional data including data other than the sensor data from the first sensor device and the sensor data from the second sensor device.