Universal time series data encoding and compression
This application is directed to compressing sensor data. A computer system obtains the sensor data from a plurality of sensor devices disposed in a physical environment during a time duration, and each sensor device corresponds to a temporal sequence of respective sensor samples. For each of the plurality of sensor devices, the temporal sequence of respective sensor samples is processed to generate an ordered sequence of respective sensor data features defining a respective parametric representation of the temporal sequence of respective sensor samples, independently of a sensor type of the respective sensor device. The computer system detects one or more signature events within the time duration based on the respective parametric representations of the plurality of sensor devices, and generates one or more information items characterizing the one or more signature events detected in the sensor data.
1 . A method for compressing sensor data, comprising:
at a computer system having one or more processors and memory:
obtaining the sensor data from a plurality of sensor devices disposed in a physical environment during a time duration, each sensor device corresponding to a temporal sequence of respective sensor samples;
for each of the plurality of sensor devices, processing the temporal sequence of respective sensor samples to generate an ordered sequence of respective sensor data features defining a respective parametric representation of the temporal sequence of respective sensor samples, independently of a sensor type of the respective sensor device, wherein the respective parametric representation includes an N-th degree polynomial representation having the highest power of a sampling time equal to N, where N is a positive integer, and the ordered sequence of respective sensor data features includes N+1 data features each of which corresponds to a distinct coefficient of the N-th degree polynomial representation;
detecting one or more signature events within the time duration based on the respective parametric representations of the plurality of sensor devices; and
generating one or more information items characterizing the one or more signature events detected in the sensor data.
2 . The method of claim 1 , wherein processing the temporal sequence of respective sensor samples further comprises:
receiving the temporal sequence of respective sensor samples at an input of a sensor data encoder model; and
generating, by the sensor data encoder model, the ordered sequence of respective sensor data features based on at least the temporal sequence of respective sensor samples.
3 . The method of claim 2 , wherein processing the temporal sequence of respective sensor samples further comprises:
receiving temporal data at the input of the sensor data encoder model jointly with the temporal sequence of respective sensor samples, wherein the temporal data include one or more of: a sequence of time stamps, a temporal length of the time duration, and a sampling rate of the temporal sequence of respective sensor samples.
4 . The method of claim 2 , wherein the sensor data encoder model is applied, independently of the sensor type of each of the respective sensor device.
5 . The method of claim 1 , further comprising:
applying a large behavior model to process the one or more information items and generate a multimodal output associated with the sensor data, the multimodal output describing the one or more signature events associated with the sensor data in one of a plurality of predefined output modalities, wherein the large behavior model includes a large language model (LLM).
6 . The method of claim 1 , wherein the sensor data includes a temporal sequence of sensor data, and obtaining the sensor data further comprises:
obtaining a stream of context data measured continuously by the plurality of sensor devices, the stream of context data including the temporal sequence of respective sensor samples that are grouped for each sensor device based on a temporal window, the temporal window configured to move with a time axis; and
associating each sensor data item of the temporal sequence of sensor data with a respective timestamp and a subset of respective sensor samples that are grouped based on the temporal window.
7 . The method of claim 1 , further comprising:
storing, in the memory, the one or more information items associated with the one or more signature events, the one or more information items including a timestamp and a location of each of the one or more signature events.
8 . The method of claim 1 , wherein a universal event projection model is applied to process the respective parametric representations of the plurality of sensor devices and generate the one or more information items characterizing the one or more signature events.
9 . The method of claim 8 , wherein each of the respective parametric representations is associated with a sensor tag indicating a type of a respective sensor device, and the respective parametric representations are inputted into the universal event projection model jointly with the sensor tag of each of the plurality of sensor devices in a predefined data format.
10 . The method of claim 8 , wherein the respective parametric representations of the plurality of sensor devices are inputted into the universal event projection model in a predefined order determined based on respective types of the plurality of sensor devices.
11 . The method of claim 1 , wherein, for each of a subset of the plurality of sensor devices, an individual projection model is applied to process the respective parametric representation and generate a subset of the one or more information items.
12 . The method of claim 1 , wherein for a temporal window corresponding to a subset of sensor data, detecting the one or more signature events further comprises:
using machine learning to process the subset of sensor data within the respective temporal window and detect one or more signature events.
13 . A computer system, comprising:
one or more processors; and
memory having instructions stored thereon, which when executed by the one or more processors cause the processors to perform:
obtaining sensor data from a plurality of sensor devices disposed in a physical environment during a time duration, each sensor device corresponding to a temporal sequence of respective sensor samples;
for each of the plurality of sensor devices, processing the temporal sequence of respective sensor samples to generate an ordered sequence of respective sensor data features defining a respective parametric representation of the temporal sequence of respective sensor samples, independently of a sensor type of the respective sensor device, wherein the respective parametric representation includes an N-th degree polynomial representation having the highest power of a sampling time equal to N, where N is a positive integer, and the ordered sequence of respective sensor data features includes N+1 data features each of which corresponds to a distinct coefficient of the N-th degree polynomial representation;
detecting one or more signature events within the time duration based on the respective parametric representations of the plurality of sensor devices; and
generating one or more information items characterizing the one or more signature events detected in the sensor data.
14 . The computer system of claim 13 , wherein the plurality of sensor devices include one or more of: a presence sensor, a proximity sensor, a microphone, a motion sensor, a gyroscope, an accelerometer, a Radar, a Lidar scanner, a camera, a temperature sensor, a heartbeat sensor, and a respiration sensor.
15 . The computer system of claim 13 , further comprising instructions for:
storing the ordered sequence of respective sensor data features or the one or more information items in a database, in place of the sensor data obtained from the plurality of sensor devices.
16 . The computer system of claim 15 , further comprising instructions for, after obtaining the sensor data:
processing the sensor data to generate one or more sets of intermediate items successively and iteratively, until generating the one or more information items.
17 . The computer system of claim 16 , further comprising instructions for:
processing the sensor data to generate a first set of intermediate items at a first time;
storing the first set of intermediate items in the database;
processing the first set of intermediate items to generate one or more second sets of intermediate items successively at one or more successive second times following the first time;
successively storing the one or more second sets of intermediate items in the database, and deleting the first set of intermediate items from the database; and
processing a most recent intermediate set of the one or more second sets of intermediate items to generate the one or more information items at a third time following the one or more successive second times.
18 . A non-transitory computer-readable storage medium, having instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform:
obtaining sensor data from a plurality of sensor devices disposed in a physical environment during a time duration, each sensor device corresponding to a temporal sequence of respective sensor samples;
for each of the plurality of sensor devices, processing the temporal sequence of respective sensor samples to generate an ordered sequence of respective sensor data features defining a respective parametric representation of the temporal sequence of respective sensor samples, independently of a sensor type of the respective sensor device, wherein the respective parametric representation includes an N-th degree polynomial representation having the highest power of a sampling time equal to N, where N is a positive integer, and the ordered sequence of respective sensor data features includes N+1 data features each of which corresponds to a distinct coefficient of the N-th degree polynomial representation;
detecting one or more signature events within the time duration based on the respective parametric representations of the plurality of sensor devices; and
generating one or more information items characterizing the one or more signature events detected in the sensor data.
19 . The non-transitory computer-readable storage medium of claim 18 , further comprising instructions for:
applying a large behavior model to process the one or more information items and generate a multimodal output associated with the sensor data, the multimodal output describing the one or more signature events associated with the sensor data in one of a plurality of predefined output modalities, wherein the large behavior model includes a large language model (LLM).
20 . The non-transitory computer-readable storage medium of claim 19 , wherein the multimodal output includes one or more of: description, timestamp, numeral information, statistic summary, warning message, and recommended action associated with one or more signature events, and the plurality of predefined output modalities include one or more of: textual statements, software code, an image or video, an information dashboard having a predefined format, a user interface, and a heatmap.