System and method for image super-resolution
A system and corresponding method perform image super-resolution (SR). The system comprises an element-unshuffled downsampler and an image SR module. The image SR module performs image SR on a low-resolution (LR) representation of a high-resolution (HR) original image. The HR original image is at a higher resolution relative to a resolution of the LR representation. The image SR module produces a reconstructed version of the HR original image via the image SR performed. The image SR is based on element-unshuffled downsampling of the LR representation. The element-unshuffled downsampler performs the element-unshuffled downsampling. The image SR module outputs the reconstructed version produced. The system performs the image SR with fewer parameters and less computation cost relative to conventional image SR.
1 . A method for performing image super-resolution (SR), the method comprising:
performing image SR on a low-resolution (LR) representation of a high-resolution (HR) original image, the HR original image at a higher resolution relative to a resolution of the LR representation, the image SR including:
performing element-unshuffled downsampling, the element-unshuffled downsampling including producing a plurality of sub-features from input features, the input features including elements from a transformed version of the LR representation;
performing a max-pooling operation on sub-features of the plurality of sub-features to produce a plurality of pooled sub-features;
convolving, using group convolution, pooled sub-features of the plurality of pooled sub-features, the convolving outputting features;
upsampling the features output from the convolving to produce up-sampled features;
producing enhanced features by adding the up-sampled features to the input features; and
producing a reconstructed version of the HR original image; and
outputting the reconstructed version produced.
2 . The method of claim 1 , wherein the element-unshuffled downsampling includes:
performing an element-unshuffle operation, the element-unshuffle operation including downsampling the input features, the downsampling including reducing a size of the input features by separating the input features into the plurality of sub-features.
3 . The method of claim 2 , wherein the separating includes selecting a subset of elements from an input feature of the input features and creating a sub-feature of the plurality of sub-features by grouping the subset of elements selected.
4 . The method of claim 1 , wherein the image SR further includes producing the reconstructed version based on the enhanced features produced.
5 . The method of claim 1 , wherein the element-unshuffled downsampling includes performing an element-unshuffle operation and wherein the element-unshuffle operation enables the element-unshuffled downsampling that yields higher performance relative to a performance based on downsampling via a different downsampling operation different from the element-unshuffled downsampling.
6 . The method of claim 5 , wherein the higher performance includes higher image quality.
7 . The method of claim 1 , wherein the image SR is performed in a non-recurrent, feed-forward manner.
8 . A system for performing image super-resolution (SR), the system comprising:
an element-unshuffled downsampler; and
an image SR module configured to perform image SR on a low-resolution (LR) representation of a high-resolution (HR) original image, the HR original image at a higher resolution relative to a resolution of the LR representation, the image SR module further configured to produce a reconstructed version of the HR original image via the image SR performed, the image SR based on element-unshuffled downsampling of the LR representation, the element-unshuffled downsampler configured to perform the element-unshuffled downsampling and to produce, via the element-unshuffled downsampling, a plurality of sub-features from input features, the input features including elements from a transformed version of the LR representation, the image SR module further configured to output the reconstructed version produced, wherein, to perform the image SR, the image SR module is further configured to:
perform a max-pooling operation on sub-features of the plurality of sub-features to produce a plurality of pooled sub-features;
convolve, using group convolution, pooled sub-features of the plurality of pooled sub-features, the convolving outputting features;
upsample the features output from the convolving to produce up-sampled features; and
produce enhanced features by adding the up-sampled features to the input features.
9 . The system of claim 8 , wherein, to perform the element-unshuffled downsampling, the element-unshuffled downsampler is further configured to:
perform an element-unshuffle operation, the element-unshuffle operation including downsampling the input features, the downsampling including reducing a size of the input features by separating the input features into the plurality of sub-features.
10 . The system of claim 9 , wherein, to separate the input features into sub-features, the element-unshuffled downsampler is further configured to select a subset of elements from an input feature of the input features and create a sub-feature of the plurality of sub-features by grouping the subset of elements selected.
11 . The system of claim 8 , wherein the image SR module is further configured to produce the reconstructed version based on the enhanced features produced.
12 . The system of claim 8 , wherein, to perform the element-unshuffled downsampling, the element-unshuffled downsampler is further configured to perform an element-unshuffle operation and wherein the element-unshuffle operation enables the element-unshuffled downsampling that yields higher performance relative to a performance based on downsampling via a different downsampling operation different from the element-unshuffled downsampling.
13 . The system of claim 12 , wherein the higher performance includes higher image quality.
14 . The system of claim 8 , wherein the image SR module is further configured to perform the image SR in a non-recurrent, feed-forward manner.
15 . The system of claim 8 , wherein the image SR module is further configured to implement a neural network and employ the neural network to perform the image SR.
16 . A non-transitory computer-readable medium for performing image super-resolution (SR), the non-transitory computer-readable medium having encoded thereon a sequence of instructions which, when loaded and executed by at least one processor, causes the at least one processor to:
perform image SR on a low-resolution (LR) representation of a high-resolution (HR) original image, the HR original image at a higher resolution relative to resolution of the LR representation, the image SR including:
performing element-unshuffled downsampling, the element-unshuffled downsampling including producing a plurality of sub-features from input features, the input features including elements from a transformed version of the LR representation;
performing a max-pooling operation on sub-features of the plurality of sub-features to produce a plurality of pooled sub-features;
convolving, using group convolution, pooled sub-features of the plurality of pooled sub-features, the convolving outputting features;
upsampling the features output from the convolving to produce up-sampled features;
producing enhanced features by adding the up-sampled features to the input features; and
producing a reconstructed version of the HR original image; and
output the reconstructed version produced.