Systems and Methods of Nonlinear Image Intensity Transformation for Denoising and Low-Precision Image Processing
The techniques described herein provide for transforming images and/or quantizing images using nonlinear techniques. The transformed images can be used for image enhancement (e.g., transformation and/or quantization may be a pre-processing step prior to performing image enhancement). For example, the nonlinear intensity transformation techniques can provide for efficient denoising, better low-precision image processing, and/or the like, compared to performing image processing on the original image.
1 . A computer-implemented method of processing an image, the method comprising:
using at least one processor to perform:
obtaining an input image comprising pixels having pixel intensity values of a first bit depth;
quantizing the input image at least in part by applying a first nonlinear transform to pixel intensity values of the input image to generate a quantized input image comprising pixel intensity values of a second bit depth, wherein the second bit depth is less than the first bit depth; and
providing the quantized input image for image processing.
2 . The method of claim 1 , wherein quantizing the input image comprises:
obtaining a transformed input image from applying the first nonlinear transform to the pixel intensity values of the input image; and
applying a surjective mapping to pixel intensity value of the transformed input image to obtain the quantized input image, wherein the surjective mapping maps pixel intensity value of the first bit depth to pixel intensity value of the second bit depth.
3 . The method of claim 2 , wherein:
the second bit depth comprises a first pixel intensity and a second pixel intensity, wherein the first pixel intensity is less than the second pixel intensity; and
quantizing the input image comprises mapping a fewer number of pixel intensities of the first bit depth to the first pixel intensity than to the second pixel intensity.
4 . The method of claim 1 , further comprising:
obtaining, from the image processing pipeline, an output image comprising pixel intensity values of the second bit depth; and
de-quantizing the output image at least in part by applying a second nonlinear transform to pixel intensity values of the output image to generate a de-quantized output image comprising pixel intensity values of the first bit depth.
5 . The method of claim 4 , wherein the second nonlinear transform comprises an inverse of the first nonlinear transform.
6 . The method of claim 1 , wherein providing the quantized input image to the image processing pipeline comprises providing the quantized input image to a neural processor.
7 . The method of claim 1 , wherein providing the quantized input image to the image processing pipeline comprises providing the quantized input image to a digital signal processor (DSP).
8 . The method of claim 1 , wherein the image processing pipeline comprises one or more processors that are of lower power than the at least one processor.
9 . The method of claim 1 , wherein the first bit depth is 10 bits, 12 bits, 14 bits, or 16 bits.
10 . The method of claim 1 , wherein the second bit depth is 8 bits.
11 . The method of claim 1 , wherein:
the first bit depth is 10 bits, 12 bits, 14 bits, or 16 bits; and
the second bit depth is 8 bits.
12 . The method of claim 1 , wherein:
the image processing pipeline comprises a machine learning model trained using a plurality of quantized images comprising pixel intensity values of the second bit depth; and
providing the quantized input image to the image processing pipeline comprises providing the quantized input image to the machine learning model to obtain an enhanced output image.
13 . An image processing system, the system comprising:
a non-volatile memory containing instructions for an image processing application; and
at least one processor directed by execution of the image processing application to:
obtain an input image comprising pixels having pixel intensity values of a first bit depth;
quantize the input image at least in part by applying a first nonlinear transform to pixel intensity values of the input image to generate a quantized input image comprising pixel intensity values of a second bit depth, wherein the second bit depth is less than the first bit depth; and
provide the quantized input image for image processing.
14 . The method of claim 1 , further comprising a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
obtain an input image comprising pixels having pixel intensity values of a first bit depth;
quantize the input image at least in part by applying a first nonlinear transform to pixel intensity values of the input image to generate a quantized input image comprising pixel intensity values of a second bit depth, wherein the second bit depth is less than the first bit depth; and
provide the quantized input image for image processing.
15 . A computer-implemented method of training a machine learning model for image enhancement, the method comprising:
using at least one processor to perform:
obtaining a plurality of images comprising pixel intensity values of a first bit depth;
quantizing the plurality of images at least in part by applying a nonlinear transform to pixel intensity values of the plurality of images to generate a plurality of quantized images comprising pixel intensity values of a second bit depth, wherein the second bit depth is less than the first bit depth; and
training the machine learning model using the plurality of quantized images.
16 . The method of claim 15 , wherein the plurality of images comprises input images and target output images and training the machine learning model using the plurality of quantized images comprises applying a supervised learning algorithm to quantized input images and quantized target output images.
17 . The method of claim 15 , wherein the machine learning model comprises a neural network.
18 . The method of claim 15 , wherein training the machine learning model using the plurality of quantized images comprises training the machine learning model to denoise an input image.
19 . A computer-implemented method of enhancing an image, the method comprising:
using at least one processor to perform:
obtaining an input image to be enhanced;
applying a nonlinear transform to pixel intensity values of the input image to obtain a transformed input image;
generating, using the transformed input image, an input be provided to a trained machine learning model; and
providing the generated input to the trained machine learning model to obtain an enhanced output image.
20 . The method of claim 19 , wherein:
the input image has a first variance of a noise property across the pixel intensity values of the input image;
the transformed input image has a second variance of the noise property across the pixel intensity values of the input image; and
the second variance is less than the first variance.
21 . The method of 20 , wherein the noise property is noise standard deviation.
22 . The method of claim 19 , wherein the trained machine learning model is trained to denoise the input.
23 . The method of claim 19 , wherein the trained machine learning model comprises a neural network.
24 . The method of claim 19 , wherein the trained machine learning model is generated by applying a supervised training algorithm to training data.
25 . The method of claim 19 , wherein:
the input image comprises pixel intensity values of a first bit depth;
generating the input using the transformed input image comprises:
quantizing the transformed input image to obtain a quantized input image comprising pixel intensity values of a second bit depth, wherein the second bit depth is less than the first bit depth; and
providing the generated input to the trained machine learning model comprises providing the quantized input image as the input to the trained machine learning model.
26 . The method of claim 25 , wherein quantizing the transformed input image comprises applying a surjective mapping to pixel intensity values of the transformed input image, wherein the surjective mapping maps the pixel intensity values of the first bit depth to pixel intensity values of the second bit depth.
27 . The method of claim 26 , wherein:
the second bit depth comprises a first pixel intensity and a second pixel intensity, wherein the first pixel intensity is less than the second pixel intensity; and
quantizing the input image comprises mapping a fewer number of pixel intensities of the first bit depth to the first pixel intensity than to the second pixel intensity.
28 . An image processing system, the system comprising:
a non-volatile memory containing instructions for an image processing application; and
at least one processor directed by execution of the image processing application to:
obtain an input image to be enhanced;
apply a nonlinear transform to pixel intensity values of the input image to obtain a transformed input image;
generate, using the transformed input image, an input be provided to a trained machine learning model; and
provide the generated input to the trained machine learning model to obtain an enhanced output image.
29 . The method of claim 19 , the method further comprising a non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
obtain an input image to be enhanced;
apply a nonlinear transform to pixel intensity values of the input image to obtain a transformed input image;
generate, using the transformed input image, an input be provided to a trained machine learning model; and
provide the generated input to the trained machine learning model to obtain an enhanced output image.