Data pre-processing for low-light images
This disclosure provides methods, devices, and systems for low-light imaging. In some implementations, an image processor may be configured to reduce or remove noise associated with an image based, at least in part, on a neural network. For example, the neural network may be trained to infer a denoised representation of the image. In some aspects, the image processor may scale the brightness level of the image to fall within a normalized range of values associated with the neural network. In some other aspects, a machine learning system may scale the brightness levels of input images to match the brightness levels of ground truth images used to train the neural network. Still further, in some aspects, the machine learning system may scale the brightness levels of the input images and the brightness levels of the ground truth images to fall within the normalized range of values during training.
1. A method of denoising images, comprising:
receiving a plurality of first pixel values representing an input image having a first brightness level and a first amount of noise;
receiving a plurality of second pixel values representing a ground truth image having a second brightness level and a second amount of noise that is less than the first amount;
determining a first range of values associated with the plurality of second pixel values;
determining a first scaling factor associated with a k th quantile of the first range of values;
performing a first scaling operation on the plurality of first pixel values that produces a respective plurality of third pixel values based on the first scaling factor;
performing a second scaling operation on the plurality of second pixel values that produces a respective plurality of fourth pixel values based on the first scaling factor;
training a neural network to reproduce the plurality of fourth pixel values based at least in part on the plurality of third pixel values;
receiving a plurality of fifth pixel values representing a first image;
determining a second range of values associated with the plurality of fifth pixel values;
determining a second scaling factor based at least in part on a normalized range of values associated with the neural network, the second scaling factor representing the k th quantile of the second range of values;
performing a third scaling operation on the plurality of fifth pixel values that produces a respective plurality of sixth pixel values based on the second scaling factor; and
inferring a denoised image from the plurality of sixth pixel values based on the neural network.
2. The method of claim 1 , wherein each of the plurality of fifth pixel values is associated with a respective color of a color filter array.
3. The method of claim 2 , wherein the denoised image comprises a plurality of seventh pixel values each associated with a respective color of the color filter array.
4. The method of claim 1 , further comprising:
determining a black level associated with the first image; and
subtracting the black level from the plurality of fifth pixel values prior to performing the third scaling operation.
5. The method of claim 1 , wherein the first brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of first pixel values and the second brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of second pixel values.
6. The method of claim 1 , further comprising:
determining a third scaling factor representing a ratio of the second brightness level relative to the first brightness level; and
applying the third scaling factor to each of the plurality of first pixel values prior to performing the first scaling operation.
7. The method of claim 6 , wherein the third scaling factor is determined based at least in part on the plurality of first pixel values and the plurality of second pixel values.
8. The method of claim 1 , further comprising:
determining a first black level associated with the input image;
determining a second black level associated with the ground truth image; and
subtracting the first black level from the plurality of first pixel values prior to performing the first scaling operation; and
subtracting the second black level from the plurality of second pixel values prior to performing the second scaling operation.
9. The method of claim 8 , wherein the first black level is determined based at least in part on the plurality of first pixel values, the plurality of second pixel values, and the second black level.
10. An image processor comprising:
a processing system; and
a memory storing instructions that, when executed by the processing system, causes the image processor to:
receive a plurality of first pixel values representing an input image having a first brightness level and a first amount of noise;
receive a plurality of second pixel values representing a ground truth image having a second brightness level and a second amount of noise that is less than the first amount;
determine a range of values associated with the plurality of second pixel values;
determine a first scaling factor associated with a k th quantile of the range of values;
perform a first scaling operation on the plurality of first pixel values that produces a respective plurality of third pixel values based on the first scaling factor;
perform a second scaling operation on the plurality of second pixel values that produces a respective plurality of fourth pixel values based on the first scaling factor; and
train a neural network to reproduce the plurality of fourth pixel values based at least in part on the plurality of third pixel values.
11. The image processor of claim 10 , wherein the first brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of third pixel values and the second brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of fourth pixel values.
12. The image processor of claim 10 , wherein execution of the instructions further causes the image processor to:
determine a second scaling factor representing a ratio of the second brightness level relative to the first brightness level; and
apply the second scaling factor to each of the plurality of first pixel values prior to performing the first scaling operation.
13. The image processor of claim 12 , wherein the second scaling factor is determined based at least in part on the plurality of first pixel values and the plurality of second pixel values.
14. The image processor of claim 10 , wherein execution of the instructions further causes the image processor to:
determine a first black level associated with the input image;
determine a second black level associated with the ground truth image;
subtract the first black level from the plurality of first pixel values prior to performing the first scaling operation; and
subtract the second black level from the plurality of second pixel values prior to performing the second scaling operation.
15. A method of training neural networks, comprising:
receiving a plurality of first pixel values representing an input image having a first brightness level and a first amount of noise;
receiving a plurality of second pixel values representing a ground truth image having a second brightness level and a second amount of noise that is less than the first amount;
determining a range of values associated with the plurality of second pixel values;
determining a first scaling factor associated with a k th quantile of the range of values;
performing a first scaling operation on the plurality of first pixel values that produces a respective plurality of third pixel values based on the first scaling factor;
performing a second scaling operation on the plurality of second pixel values that produces a respective plurality of fourth pixel values based on the first scaling factor; and
training a neural network to reproduce the plurality of fourth pixel values based at least in part on the plurality of third pixel values.
16. The method of claim 15 , wherein the first brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of third pixel values and the second brightness level includes an amount of exposure, gain, or illuminance associated with the plurality of fourth pixel values.
17. The method of claim 15 , further comprising:
determining a second scaling factor representing a ratio of the second brightness level relative to the first brightness level; and
applying the second scaling factor to each of the plurality of first pixel values prior to performing the first scaling operation.
18. The method of claim 17 , wherein the second scaling factor is determined based at least in part on the plurality of first pixel values and the plurality of second pixel values.
19. The method of claim 15 , further comprising:
determining a first black level associated with the input image;
determining a second black level associated with the ground truth image;
subtracting the first black level from the plurality of first pixel values prior to performing the first scaling operation; and
subtracting the second black level from the plurality of second pixel values prior to performing the second scaling operation.
20. The method of claim 19 , wherein the first black level is determined based at least in part on the plurality of first pixel values, the plurality of second pixel values, and the second black level.