Range aware spatial upscaling
Aspects are provided for preserving the edge sharpness and enhancing details while upscaling a rendered image. Some aspects specifically relate to upscaling images using a spatial difference between a pixel to be interpolated and respective neighboring pixels, and an intensity difference between respective neighboring pixels and a mean intensity of a subset of the neighboring pixels. Considering both the spatial and intensity difference during upscaling prevents small details of the input image from being smoothed, since a larger weight may be assigned to pixels with a closer intensity value and that are closer in distance to the pixel to be interpolated than in techniques such as bilinear interpolation where only spatial difference may be considered. The result is an upscaled image that preserves sharp edges and small detail for a more accurate upscaled image in a higher resolution than compared to related upscaling methods that only consider using spatial difference.
1 . A method for graphics processing, comprising:
receiving a first image including a plurality of first pixels at a first resolution;
upscaling the first image to a second image at a second resolution higher than the first resolution, the second image including a plurality of second pixels including a first interpolated pixel and a second interpolated pixel that are derived from the plurality of first pixels, the upscaling comprising:
deriving an intensity value of the first interpolated pixel from a first plurality of neighboring pixels of a location of the first interpolated pixel corresponding to the plurality of first pixels, the intensity value being derived using a first function based at least in part on, for a respective pixel of the first plurality of neighboring pixels, an intensity difference between an intensity value of the respective pixel and an average intensity value of a subset of the first plurality of neighboring pixels, and a spatial difference between the respective pixel and the first interpolated pixel;
calculating a variance of intensity values for a second plurality of neighboring pixels of a location of the second interpolated pixel corresponding to the plurality of first pixels; and
deriving an intensity value of the second interpolated pixel using a second function different than the first function in response to the variance of intensity values being less than a threshold; and
rendering the second image at the second resolution.
2 . The method of claim 1 , the upscaling further comprising:
calculating a variance of intensity values for the subset of the first plurality of neighboring pixels, wherein the intensity value of the first interpolated pixel is derived from the intensity difference and the spatial difference in response to the variance of intensity values for the subset of the first plurality of neighboring pixels being greater than or equal to a threshold.
3 . The method of claim 2 , wherein the intensity value of the first interpolated pixel is derived for a green (G) channel output of the first interpolated pixel.
4 . The method of claim 3 , wherein the intensity value for the G channel output of the first interpolated pixel is derived from the intensity difference and the spatial difference using the first function, and red (R) and blue (B) channel outputs of the second interpolated pixel are calculated using the second function different than the first function.
5 . The method of claim 1 , wherein the second function is bilinear interpolation.
6 . The method of claim 1 , wherein the intensity difference is determined based on a range filter.
7 . The method of claim 1 , wherein the first plurality of neighboring pixels form a 12-pixel window surrounding the location of the first interpolated pixel corresponding to the plurality of first pixels.
8 . The method of claim 1 , wherein the subset of the first plurality of neighboring pixels form a window of two by two pixels adjacent to the location of the first interpolated pixel corresponding to the plurality of first pixels.
9 . The method of claim 1 , further comprising:
sharpening the intensity value of the first interpolated pixel prior to the rendering.
10 . The method of claim 1 , wherein the intensity value of the first interpolated pixel is derived from the intensity difference and the spatial difference using a Lanczos kernel.
11 . An apparatus for graphics processing, comprising:
a processor, memory coupled with the processor; and
instructions stored in the memory and operable, when executed by the processor, to cause the apparatus to:
receive a first image including a plurality of first pixels at a first resolution;
upscale the first image to a second image at a second resolution higher than the first resolution, the second image including a plurality of second pixels including a first interpolated pixel and a second interpolated pixel that are derived from the plurality of first pixels, wherein to upscale the first image, the instructions, when executed by the processor, further cause the apparatus to:
derive an intensity value of the first interpolated pixel from a first plurality of neighboring pixels of a location of the first interpolated pixel corresponding to the plurality of first pixels, the intensity value being derived using a first function based at least in part on, for a respective pixel of the first plurality of neighboring pixels, an intensity difference between an intensity value of the respective pixel and an average intensity value of a subset of the first plurality of neighboring pixels, and a spatial difference between the respective pixel and the first interpolated pixel;
calculate a variance of intensity values for a second plurality of neighboring pixels of a location of the second interpolated pixel corresponding to the plurality of first pixels; and
derive an intensity value of the second interpolated pixel using a second function different than the first function in response to the variance of intensity values being less than a threshold; and
render the second image at the second resolution.
12 . The apparatus of claim 11 , wherein, to upscale, the instructions, when executed by the processor, further cause the apparatus to:
calculate a variance of intensity values for the subset of the first plurality of neighboring pixels, wherein the intensity value of the first interpolated pixel is derived from the intensity difference and the spatial difference in response to the variance of intensity values for the subset of the first plurality of neighboring pixels being greater than or equal to a threshold.
13 . The apparatus of claim 12 , wherein the intensity value of the first interpolated pixel is derived for a green (G) channel output of the first interpolated pixel.
14 . The apparatus of claim 13 , wherein the intensity value for the G channel output of the first interpolated pixel is derived from the intensity difference and the spatial difference using the first function, and red (R) and blue (B) channel outputs of the second interpolated pixel are calculated using the second function different than the first function.
15 . The apparatus of claim 11 , wherein the second function is bilinear interpolation.
16 . The apparatus of claim 11 , wherein the intensity difference is determined based on a range filter.
17 . The apparatus of claim 11 , wherein first plurality of the neighboring pixels form a 12-pixel window surrounding the location of the first interpolated pixel corresponding to the plurality of first pixels.
18 . The apparatus of claim 11 , wherein the subset of the first plurality of neighboring pixels form a window of two by two pixels adjacent to the location of the first interpolated pixel corresponding to the plurality of first pixels.
19 . The apparatus of claim 11 , wherein the instructions, when executed by the processor, further cause the apparatus to:
sharpen the intensity value of the first interpolated pixel prior to the render.
20 . The apparatus of claim 11 , wherein the intensity value of the first interpolated pixel is derived from the intensity difference and the spatial difference using a Lanczos kernel.
21 . An apparatus for graphics processing, comprising:
means for receiving a first image including a plurality of first pixels at a first resolution;
means for upscaling the first image to a second image at a second resolution higher than the first resolution, the second image including a plurality of second pixels including an interpolated pixel that is derived from the plurality of first pixels, the means for upscaling comprising:
means for deriving an intensity value of the interpolated pixel from a plurality of neighboring pixels of a location of the interpolated pixel corresponding to the plurality of first pixels, the intensity value being based at least in part on, for a respective pixel of the plurality of neighboring pixels, an intensity difference between an intensity value of the respective pixel and an average intensity value of a subset of the plurality of neighboring pixels, and a spatial difference between the respective pixel and the interpolated pixel; and
means for rendering the second image at the second resolution.
22 . The apparatus of claim 21 , the means for upscaling further comprising:
means for calculating a variance of intensity values for the subset of the plurality of neighboring pixels, wherein the intensity value of the interpolated pixel is derived from the intensity difference and the spatial difference in response to the variance of intensity values for the subset of the plurality of neighboring pixels being greater than or equal to a threshold.
23 . The apparatus of claim 22 , wherein the intensity value of the interpolated pixel is derived for a green (G) channel output of the interpolated pixel.
24 . The apparatus of claim 23 , wherein the intensity value for the G channel output of the interpolated pixel is derived from the intensity difference and the spatial difference using a first function, and red (R) and blue (B) channel outputs of a different interpolated pixel are calculated using a second function different than the first function.
25 . The apparatus of claim 24 , wherein the second function is bilinear interpolation.
26 . The apparatus of claim 21 , the means for upscaling further comprising:
means for calculating a variance of intensity values for a second plurality of neighboring pixels of a location of a different interpolated pixel corresponding to the plurality of first pixels; and
means for deriving an intensity value of the different interpolated pixel using a second function different than a first function in response to the variance of intensity values being less than a threshold, wherein the intensity value of the interpolated pixel is derived from the intensity difference and the spatial difference using the first function.
27 . The apparatus of claim 21 , wherein the plurality of neighboring pixels form a 12-pixel window surrounding the location of the interpolated pixel corresponding to the plurality of first pixels.
28 . A non-transitory, computer-readable medium storing computer executable code, the code when executed by a processor cause the processor to:
receive a first image including a plurality of first pixels at a first resolution;
upscale the first image to a second image at a second resolution higher than the first resolution, the second image including a plurality of second pixels including a first interpolated pixel and a second interpolated pixel that are derived from the plurality of first pixels, wherein to upscale the first image, the code, when executed by the processor, further causes the processor to:
derive an intensity value of the first interpolated pixel from a first plurality of neighboring pixels of a location of the first interpolated pixel corresponding to the plurality of first pixels, the intensity value being derived using a first function based at least in part on, for a respective pixel of the first plurality of neighboring pixels, an intensity difference between an intensity value of the respective pixel and an average intensity value of a subset of the first plurality of neighboring pixels, and a spatial difference between the respective pixel and the first interpolated pixel;
calculate a variance of intensity values for a second plurality of neighboring pixels of a location of the second interpolated pixel corresponding to the plurality of first pixels; and
derive an intensity value of the second interpolated pixel using a second function different than the first function in response to the variance of intensity values being less than a threshold; and
render the second image at the second resolution.
29 . The non-transitory, computer-readable medium of claim 28 , wherein the intensity value of the first interpolated pixel is derived for a green (G) channel output of the first interpolated pixel.
30 . The non-transitory, computer-readable medium of claim 29 , wherein the intensity value for the G channel output of the first interpolated pixel is derived from the intensity difference and the spatial difference using the first function, and red (R) and blue (B) channel outputs of the second interpolated pixel are calculated using the second function different than the first function.