Class-specific neural network for video compressed sensing
A class-specific neural network for video compressed sensing and methods for training and testing the class-specific neural network are provided. The class-specific neural network includes a Gaussian-mixture model (GMM) and a plurality of encoders, where the GMM classifies video frame blocks with a plurality of clusters and assigns the video frame blocks to the plurality of clusters. Further, the plurality of encoders receive the video frame blocks and generate a plurality of compressed-sensed frame block vectors, where the plurality of encoders correspond to the plurality of clusters.
1. A method for video compressed sensing by a class-specific neural network, comprising:
classifying, by a Gaussian-mixture model (GMM), video frame blocks with a plurality of clusters and assigning the video frame blocks to the plurality of clusters;
receiving, by a plurality of encoders, the video frame blocks;
generating, by the plurality of encoders, a plurality of compressed-sensed frame block vectors, wherein the plurality of encoders respectively correspond to the plurality of clusters,
wherein each encoder comprises a flatten layer, a discrete cosine transform (DCT) transform layer, and a trainable compressed sensing layer;
predicting, by a logistic regression classifier, class labels for the plurality of compressed-sensed frame block vectors without recording clustering information, wherein the logistic regression classifier predicts the class labels by respectively maximizing probabilities of the plurality of compressed-sensed frame block vectors;
sending, by the logistic regression classifier, the compressed-sensed frame block vectors to a plurality of decoders, and
wherein the plurality of compressed-sensed frame block vectors are assigned to the plurality of decoders based on the class labels for the plurality of compressed-sensed frame block vectors, wherein the class labels are saved in a hashmap data structure.
2. The method of claim 1 , wherein the GMM predicts labels for the video frame blocks.
3. The method of claim 1 , wherein each decoder comprises an expansion layer, a trainable reconstruction layer, an inverse DCT transform layer, and a reshape layer that converts the plurality of compressed-sensed frame block vectors to a plurality of predicted block matrices.
4. A method for training a class-specific neural network for video compressed sensing, comprising:
training a Gaussian-mixture model (GMM) in the class-specific neural network using a plurality of video frame blocks, wherein the GMM comprises a plurality of clusters;
assigning the plurality of video frame blocks to the plurality of clusters;
training a plurality of encoders that respectively correspond to the plurality of clusters using the plurality of video frame blocks,
wherein each encoder comprises a flatten layer, a discrete cosine transform (DCT) transform layer, and a trainable compressed sensing layer,
generating class labels for a plurality of compressed-sensed frame block vectors by labeling the plurality of compressed-sensed frame block vectors according to the plurality of clusters;
training a logistic regression classifier for the plurality of compressed-sensed frame block vectors based on the class labels without recording clustering information, wherein the logistic regression classifier predicts the labels by respectively maximizing probabilities of the plurality of compressed-sensed frame block vectors; and
sending, by the logistic regression classifier, the compressed-sensed frame block vectors to a plurality of decoders,
wherein the plurality of compressed-sensed frame block vectors are assigned to the plurality of decoders based on the class labels for the plurality of compressed-sensed frame block vectors, wherein the class labels are saved in a hashmap data structure.
5. A method for testing a class-specific neural network for video compressed sensing, comprising:
assigning and sending, by a trained Gaussian-mixture model (GMM) in the class-specific neural network, a plurality of video frame blocks to a plurality of clusters;
generating, by a plurality of encoders in the class-specific neural network, a plurality of compressed-sensed frame block vectors,
wherein each encoder comprises a flatten layer, a discrete cosine transform (DCT) transform layer, and a trainable compressed sensing layer,
classifying, by a trained logistic regression classifier, class labels of the plurality of compressed-sensed frame block vectors without recording clustering information, wherein the trained logistic regression classifier predicts the class labels by respectively maximizing probabilities of the plurality of compressed-sensed frame block vectors; and
assigning, by the trained logistic regression classifier, the plurality of compressed-sensed frame block vectors to corresponding decoders based on the class labels to reconstruct a whole video frame,
wherein the class labels are saved in a hashmap data structure.
6. An apparatus for training a class-specific neural network for video compressed sensing, comprising:
one or more processors; and
a memory configured to store instructions executable by the one or more processors,
wherein the one or more processors, upon execution of the instructions, are configured to:
train a Gaussian-mixture model (GMM) in the class-specific neural network using a plurality of video frame blocks, wherein the GMM comprises a plurality of clusters;
assign the plurality of video frame blocks to the plurality of clusters;
train a plurality of encoders that respectively correspond to the plurality of clusters using the plurality of video frame blocks,
wherein each encoder comprises a flatten layer, a discrete cosine transform (DCT) transform layer, and a trainable compressed sensing layer,
generate class labels for a plurality of compressed-sensed frame block vectors by labeling the plurality of compressed-sensed frame block vectors according to the plurality of clusters;
train a logistic regression classifier for the plurality of compressed-sensed frame block vectors based on the class labels without recording clustering information, wherein the logistic regression classifier predicts the labels by respectively maximizing probabilities of the plurality of compressed-sensed frame block vectors; and
send, by the logistic regression classifier, the compressed-sensed frame block vectors to a plurality of decoders,
wherein the plurality of compressed-sensed frame block vectors are assigned to the plurality of decoders based on the class labels for the plurality of compressed-sensed frame block vectors, wherein the class labels are saved in a hashmap data structure.
7. An apparatus for testing a class-specific neural network for video compressed sensing, comprising:
one or more processors; and
a memory configured to store instructions executable by the one or more processors,
wherein the one or more processors, upon execution of the instructions, are configured to:
assign and send, by a trained Gaussian-mixture model (GMM) in the class-specific neural network, a plurality of vectorized video frame blocks to corresponding clusters; and
generate, by a plurality of encoders in the class-specific neural network, a plurality of compressed-sensed frame block vectors,
wherein each encoder comprises a flatten layer, a discrete cosine transform (DCT) transform layer, and a trainable compressed sensing layer,
classify, by a trained logistic regression classifier, class labels of the plurality of compressed-sensed frame block vectors without recording clustering information, wherein the trained logistic regression classifier predicts the class labels by respectively maximizing probabilities of the plurality of compressed-sensed frame block vectors; and
assign, by the trained logistic regression classifier, the plurality of compressed-sensed frame block vectors to corresponding decoders based on the class labels to reconstruct a whole video frame,
wherein the class labels are saved in a hashmap data structure.