Multivariate data compression system and method thereof
A smart sensing architecture ( 100 ) includes smart meters ( 102 ) and processing units ( 104 ). The smart meters ( 102 ) generate and transmit multidimensional data streams to the processing units ( 104 ). A processing unit ( 104 ) determines an optimum batch size for a multidimensional data stream and generates a multidimensional batch of data. The processing unit ( 104 ) reduces dimensionality of the multidimensional batch of data using principal component analysis to generate a low-dimensional batch of data and performs compressive sampling on the low-dimensional batch of data to generate a compressed batch of data, thereby saving bandwidth of transmission.
1. A method of compressing multivariate data, the method comprising:
receiving a plurality of multidimensional data streams generated by a plurality of smart meters, wherein the plurality of smart meters comprises a processor, a memory, and a communication unit;
determining an optimum batch size for the multidimensional data streams;
generating a multidimensional batch of data based on aforesaid batch size;
reducing dimensionality of the multidimensional batch of data using principal component analysis to generate a low-dimensional batch of data; and
performing temporal compression on the low-dimensional batch of data to generate a compressed batch of data,
wherein the optimum batch size is determined such that a normalized root mean square error of output data reconstructed based on the compressed batch of data is less than a predefined error limit.
2. The method as claimed in claim 1 , further comprising:
identifying the principal components in the multidimensional batch of data;
determining a first set of principal components comprising more variance than a predefined threshold variance; and
generating the low-dimensional batch of data including the first set of principal components.
3. The method as claimed in claim 1 , wherein the optimum batch size is determined based on a sparsity of the multidimensional data streams.
4. A smart sensing architecture, comprising:
a plurality of smart meters for generating and transmitting a plurality of multidimensional data streams; and
a processing unit connected to the plurality of smart sensing devices, said processing unit configured to:
receive the plurality of multidimensional data streams generated by the plurality of smart meters, wherein the plurality of smart meters comprises a processor, a memory, and a communication unit;
determine an optimum batch size for the multidimensional data streams,
generate a multidimensional batch of data based on aforesaid batch size,
reduce dimensionality of the multidimensional batch of data using principal component analysis to generate a low-dimensional batch of data, and
perform temporal compression on the low dimensional batch of data to generate a compressed batch of data,
wherein the optimum batch size is determined such that a normalized root mean square error of output data reconstructed based on the compressed batch of data is less than a predefined error limit.
5. The smart sensing architecture as claimed in claim 4 , wherein the processing unit is further configured to:
identify a number of principal components in the multidimensional batch of data,
determine a first set of principal components comprising more variance than a predefined threshold variance, and
generate the low-dimensional batch of data including the first set of principal components.
6. The smart sensing architecture as claimed in claim 4 , wherein the optimum batch size is determined based on a sparsity of the multidimensional data streams.