Image processing device and moving image transmission method
Provided is an image processing device which performs scaling and applies super-resolution in accordance with the complexity of the image. On the compression side, a complexity calculation unit calculates the transmission image complexity, which indicates the complexity of the input image, multiplexes and outputs it to an encoded stream, and a control unit configures the cutoff frequency of a low pass filter and the scaling coefficient of an image reduction unit in accordance with the transmission image complexity. On the decompression side, a complexity calculation unit calculates the decoded image complexity indicating the complexity of a decoded and expanded image, a complexity comparison unit compares the transmission image complexity and the decoded image complexity and outputs the difference or ratio, and, on the basis of output, a parameter calculation unit optimizes and configures the scaling coefficient of an image enlargement unit and the folding frequency of a super-resolution processing unit.
1. An image processing device comprising an image compression unit for compressing an input image, and an image decompression unit for decompressing the compressed image for output,
wherein the image compression unit includes:
a low pass filter configured to limit a bandwidth of an input image signal based on a preset cutoff frequency;
an image reduction unit configured to reduce an output signal from the low pass filter according to a preset reduction factor;
an encoding unit configured to encode an output signal from the image reduction unit;
a first complexity calculation unit configured to calculate at least one first parameter indicating image complexity for the input image signal;
a control unit configured to set a reduction factor corresponding to a pre-stored cutoff frequency in the image reduction unit while setting another cutoff frequency for limiting a bandwidth by cutting components above a specific frequency in the low pass filter, based on the first parameter; and
a multiplexer configured to multiplex the first parameter and an encoded stream outputted from the encoding unit for output, and
wherein the image decompression unit includes:
a demultiplexer configured to extract the encoded stream and the first parameter from an input signal; a decoding unit configured to decode the encoded stream;
an image enlargement unit configured to enlarge an output signal from the decoding unit according to a specified enlargement factor;
a super-resolution processing unit configured to perform a super-resolution process on the enlarged image using a specified folding frequency;
a second complexity calculation unit configured to calculate at least one second parameter indicating image complexity for the enlarged image;
a complexity comparison unit configured to calculate a difference or a ratio between the first parameter and the second parameter; and
a parameter calculation unit configured to set another folding frequency for removing aliasing components using the calculated second parameter in the super-resolution processing unit while setting an enlargement factor corresponding to a pre-stored second parameter in the image enlargement unit, based on the difference or the ratio.
2. The image processing device of claim 1 , wherein in the image compression unit, the image reduction unit performs reduction of an entire frame when the first parameter is smaller than a preset first threshold value, and does not perform reduction of the entire frame when the first parameter is equal to or greater than the preset first threshold value, and the encoding unit calculates a parameter indicating a complexity for each of divided regions of the input image regardless of performing the reduction of the entire frame, performs encoding with reducing a region for which the parameter is smaller than a preset second threshold value, and performs encoding without reducing a region for which the parameter is equal to or greater than the preset second threshold value, and
wherein in the image decompression unit, the decoding unit performs decoding each of regions according to whether reduction of said each region and reduction of the entire frame has been performed or not in the image compression unit.
3. A moving image transmission method using an image compression unit for compressing an amount of data of an input moving image and an image decompression unit for restoring a compressed moving image in which the amount of data has been compressed to an original moving image, the method comprising:
in the image compression unit,
a bandwidth limiting step of limiting a bandwidth of an input image signal based on a preset cutoff frequency by a low pass filter of the image compression unit;
a reduction step of reducing an output signal from the low pass filter according to a preset reduction factor;
an encoding step of encoding an output signal from the reducing;
a first calculation step of calculating, for the input image signal, a first parameter indicating image complexity for each of divided regions of the image;
a first control step of setting a reduction factor corresponding to a ore-stored cutoff frequency in the image reduction unit while setting another cutoff frequency for limiting a bandwidth by cutting components above a specific frequency in the low pass filter based on the first parameter; and
a multiplexing step of multiplexing the first parameter and an encoded stream outputted from the encoding step for output,
in the image decompression unit,
a demultiplexing step of extracting the encoded stream and the first parameter from a signal inputted to the image decompression unit;
a decoding step of decoding the encoded stream;
an enlargement step of enlarging an output signal from the decodng step according to a specified enlargement factor;
a super-resolution step of performing a super-resolution process on the enlarged image using a specified folding frequency;
a second calculation step of calculating a second parameter indicating an image complexity for the enlarged image;
a comparison step of calculating a difference or a ratio between the first parameter and the second parameter; and
a setting step of setting another folding frequency for removing aliasing components using the second parameter calculated in the super-resolution step while setting an enlargement factor corresponding to a ore-stored second parameter in the enlargement step based on the difference or the ratio,
wherein, in the reduction step, performing reduction of an entire frame when the first parameter is smaller than a preset first threshold value, and not performing reduction of the entire frame when the first parameter is equal to or greater than the preset first threshold value,
wherein, in the encoding step, performing encoding with reducing a region for which the first parameter is smaller than a preset second threshold value, and performing encoding without reducing a region for which the parameter is equal to or greater than the preset second threshold value, and
wherein, in the decoding step, performing decoding each of regions according to whether reduction of said each region and reduction of the entire frame has been performed or not in the image compression unit.