Quality of lossy compressed sensor data with machine learning
The quality of decompressed data that was compressed in a lossy manner to generate lossy compressed data, is improved. Lossy compressed data may be generated at a device and transmitted to a gateway. The gateway includes a reconstruction engine that is trained to reduce the compression loss by generating a learned residue. The residue is added to the lossy decompressed data to generated corrected decompressed data that has a smaller loss than the lossy decompressed data.
1. A method comprising:
receiving, at a reconstruction engine over a network, a sequence of lossy compression data that has been generated at a device, wherein the sequence of lossy compression data was generated by compressing a sequence of data with a lossy compression operation;
decompressing the lossy compression data to generate decompressed data;
providing the lossy compression data, the decompressed data, and at least one past sequence as input to a reconstruction model associated with the reconstruction engine to generate a residue that is configured to reduce a distance between the decompressed data and the sequence of data; and
generating corrected decompressed data by combining the decompressed data with the residue.
2. The method of claim 1 , wherein the lossy compression data comprises a compressed signal vector that has been quantized and binarized, the compressed signal vector comprising an array of values associated with the compression performed at the device.
3. The method of claim 2 , further comprising generating a reconstructed compressed signal vector by reversing the binarization and quantization performed on the compressed signal vector.
4. The method of claim 3 , wherein the decompressed data input to the reconstruction model further comprises the reconstructed compressed signal vector and the at least one past sequence comprises at least one past sequence of lossy compression data.
5. The method of claim 1 , further comprising adding the residue to the decompressed data.
6. The method of claim 1 , wherein the reconstruction model comprises a transformer-based neural network model or an MLP (Multilayer Perceptron) model.
7. The method of claim 1 , wherein the sequence comprises a plurality of samples generated at the device.
8. The method of claim 1 , further comprising training the reconstruction model using historical sequences of raw data from the device.
9. The method of claim 8 , wherein a portion of the historical sequences are used as training data and a second portion of the historical sequences are used as testing data.
10. The method of claim 1 , further comprising outputting a decompressed signal.
11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
receiving, at a reconstruction engine over a network, a sequence of lossy compression data that has been generated at a device, wherein the sequence of lossy compression data was generated by compressing a sequence of data with a lossy compression operation;
decompressing the lossy compression data to generate decompressed data;
providing the lossy compression data, the decompressed data, and at least one past sequence as input to a reconstruction model associated with the reconstruction engine to generate a residue that is configured to reduce a distance between the decompressed data and the sequence of data; and
generating corrected decompressed data by combining the decompressed data with the residue.
12. The non-transitory storage medium of claim 11 , wherein the lossy compression data comprises a compressed signal vector that has been quantized and binarized, the compressed signal vector comprising an array of values associated with the compression performed at the device.
13. The non-transitory storage medium of claim 12 , further comprising generating a reconstructed compressed signal vector by reversing the binarization and quantization performed on the compressed signal vector.
14. The non-transitory storage medium of claim 13 , wherein the decompressed data input to the reconstruction model further comprises the reconstructed compressed signal vector and the at least one past sequence comprises at least one past sequence of lossy compression data.
15. The non-transitory storage medium of claim 11 , further comprising adding the residue to the decompressed data.
16. The non-transitory storage medium of claim 11 , wherein the reconstruction model comprises a transformer-based neural network model or an MLP (Multilayer Perceptron) model.
17. The non-transitory storage medium of claim 11 , wherein the sequence comprises a plurality of samples generated at the device.
18. The non-transitory storage medium of claim 11 , further comprising training the reconstruction model using historical sequences of raw data from the device.
19. The non-transitory storage medium of claim 18 , wherein a portion of the historical sequences are used as training data and a second portion of the historical sequences are used as testing data.
20. The non-transitory storage medium of claim 1 , further comprising outputting a decompressed signal.