Systems and methods for feature transformation, correction and regeneration for robust sensing, transmission, computer vision, recognition and classification
View Patent ↗Embodiments of a deep learning enabled generative sensing and feature regeneration framework which integrates low-end sensors/low quality data with computational intelligence to attain a high recognition accuracy on par with that attained with high-end sensors/high quality data or to optimize a performance measure for a desired task are disclosed.
1. A method for generative sensing comprising:
conducting a training phase on a plurality of datasets in which the plurality of datasets comprises high quality data of a type X and low quality data of a type Y by a processor, wherein the high quality data of type X and low quality data of type Y can be of the same type or of different types; and
applying a neural network φ to the high quality data of a type X and to the low quality data of a type Y to obtain a first set of feature maps associated with the high quality data of a type X (φ high ), and a second set of feature maps associated with low quality data of a type Y (φ low );
using a distance measure to quantify feature differences between co-located features in φ high and φ low to obtain feature difference maps, Δφ; and
locating feature differences in φ low that are significantly different from co-located features in φ high based on a change in classification accuracy.
2. The method of claim 1 , wherein the neural network is a deep neural network.
3. The method of claim 1 , wherein the high quality data and the low quality data are acquired using sensors.
4. The method of claim 1 , wherein the high-quality data is acquired using a high end sensor and a low quality data is acquired using a low end sensor.
5. The method of claim 1 , wherein one or more of the high quality and low quality data are transmitted using a transmission medium.
6. The method of claim 5 wherein the transmission medium is one or more of a communication channel or a transmission link.
7. The method of claim 1 , wherein locating feature differences comprises measuring the classification accuracy drop when a feature in φ high is replaced by its co-located φ low feature while keeping all other feature in φ high unchanged.
8. The method of claim 1 , wherein locating feature differences comprises maximizing a drop in classification accuracy by replacing clusters of feature in φ high with clusters of features in φ low .
9. The method of claim 1 , further comprising applying feature correction to the high quality data or the low quality data.
10. The method of claim 9 , wherein the feature correction is performed using one or more transformations applied to features in φ low .
11. A method for generative sensing comprising:
conducting a training phase on a plurality of datasets in which the plurality of datasets comprises high quality data of a type X and low quality data of a type Y by a processor, wherein the high quality data of type X and low quality data of type Y can be of the same type or of different types;
applying a neural network φ to the high quality data of a type X and to the low quality data of a type Y to obtain a first set of feature maps associated with the high quality data of a type X (φ high ), and a second set of feature maps associated with low quality data of a type Y (φ low ); and
applying feature correction to the high quality data or the low quality data,
wherein the feature correction is performed using one or more transformations applied to features in φ low , and
wherein the one or more transformations are learned.
12. The method of claim 11 , wherein the one or more transformations take each the form of a multi-layer network with learnable parameters.
13. A method for generative sensing comprising:
conducting a training phase on a plurality of datasets in which the plurality of datasets comprises high quality data of a type X and low quality data of a type Y by a processor, wherein the high quality data of type X and low quality data of type Y can be of the same type or of different types; and
applying a neural network φ to the high quality data of a type X and to the low quality data of a type Y to obtain a first set of feature maps associated with the high quality data of a type X (φ high ), and a second set of feature maps associated with low quality data of a type Y (φ low ); and
applying feature correction to the high quality data or the low quality data,
wherein the feature correction is applied through a selective feature correction by correcting selectively features in φ low .
14. The method of claim 13 , wherein the selective feature correction is performed by correcting features in φ low corresponding to a significant drop in classification accuracy while leaving the other features in φ low unchanged.