Method and system of automatic content-dependent image processing algorithm selection
A method, system, and article is directed to automatic content-dependent image processing algorithm selection.
1 . A computer-implemented method of image processing, comprising:
obtaining one or more frames of a video sequence;
dividing a frame of the one or more frames into one or more sections;
selecting between a non-neural network image processing algorithm and a neural network image processing algorithm to apply to a section of the one or more sections, wherein selecting comprises:
determining whether the section is a salient section;
based on the section being a non-salient section, selecting the non-neural network image processing algorithm;
based on the section being a salient section, determining whether a standard deviation of the section's luminance channel meets a criterion;
based on the standard deviation meeting the criterion, selecting the neural network image processing algorithm; and
based on the standard deviation not meeting the criterion, selecting the non-neural network image processing algorithm; and
applying the selected algorithm to the section.
2 . The computer-implemented method of claim 1 , wherein the non-neural network image processing algorithm and the neural network image processing algorithm comprises performing super-resolution to upscale the section.
3 . The computer-implemented method of claim 1 , wherein the criterion is a texture-based threshold.
4 . The computer-implemented method of claim 1 , wherein the criterion includes a texture-based threshold of smoothness of the section.
5 . The computer-implemented method of claim 4 , wherein the threshold is a limit of the standard deviation of the section's luminance channel.
6 . The computer-implemented method of claim 1 , wherein the criterion is only based on the section's luminance channel.
7 . The computer-implemented method of claim 1 , wherein the one or more sections have different sizes on a single frame.
8 . A system for image processing, comprising:
memory to at least store image data of frames of a video sequence; and
processor circuitry being communicatively coupled to the memory and being arranged to operate by:
obtaining one or more frames of a video sequence;
dividing a frame of the one or more frames into one or more sections;
selecting between a non-neural network image processing algorithm and a neural network image processing algorithm to apply to a section of the one or more sections, wherein selecting comprises:
determining whether the section is a salient section;
based on the section being a non-salient section, selecting the non-neural network image processing algorithm;
based on the section being a salient section, determining whether a standard deviation of the section's luminance channel meets a criterion;
based on the standard deviation meeting the criterion, selecting the neural network image processing algorithm; and
based on the standard deviation not meeting the criterion, selecting the non-neural network image processing algorithm; and
applying the selected algorithm to the section.
9 . The system of claim 8 , wherein the non-neural network image processing algorithm and the neural network image processing algorithm comprises performing super-resolution to upscale the section.
10 . The system of claim 8 , wherein the criterion includes a texture-based threshold of smoothness of the section.
11 . The system of claim 10 , wherein the texture-based threshold is a limit of the standard deviation of the section's luminance channel.
12 . The system of claim 10 , wherein the texture-based threshold is adjustable to balance performance in computation time and power capacity versus image quality.
13 . At least one non-transitory article having at least one computer readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to operate by:
obtaining one or more frames of a video sequence;
dividing a frame of the one or more frames into one or more sections;
selecting between a non-neural network image processing algorithm and a neural network image processing algorithm to apply to a section of the one or more sections, wherein selecting comprises:
determining whether the section is a salient section;
based on the section being a non-salient section, selecting the non-neural network image processing algorithm;
based on the section being a salient section, determining whether a standard deviation of the section's luminance channel meets a criterion;
based on the standard deviation meeting the criterion, selecting the neural network image processing algorithm; and
based on the standard deviation not meeting the criterion, selecting the non-neural network image processing algorithm; and
applying the selected algorithm to the section.
14 . The at least one non-transitory article of claim 13 , wherein the non-neural network image processing algorithm and the neural network image processing algorithm comprises performing super-resolution to upscale the section.
15 . The at least one non-transitory article of claim 13 , wherein the criterion includes a texture-based threshold that is a measure of smoothness of the section.
16 . The at least one non-transitory article of claim 15 , wherein the measure is a limit to the standard deviation of the section's luminance channel.
17 . The at least one non-transitory article of claim 13 , wherein the one or more sections have uniform sizes on a single frame.
18 . The at least one non-transitory article of claim 13 , wherein the criterion is a variable threshold that varied to balance performance including computation time and power consumption versus image quality.
19 . The at least one non-transitory article of claim 18 , wherein the section is determined to be the non-salient section based on the section being part of a background of the frame.
20 . The at least one non-transitory article of claim 18 , wherein the section is determined to be the salient section based on the section having a recognized object deemed to be important to a viewer of the video sequence.