Low-light imaging using trained convolutional neural networks

1. An apparatus for imaging in low-light environments, comprising:

a raw sensor data receiver to receive raw sensor data from an imaging sensor;

a black level subtractor to subtract a black level from packed data to generate black-level-subtracted packed data, the black level generated from the raw sensor data;

an amplifier to amplify the black-level-subtracted packed data based on a received amplification ratio to generate amplified black-level-subtracted packed data to be input into a convolutional neural network; and

the convolutional neural network trained to generate an illuminated image based on the amplified black-level-subtracted packed data, wherein the convolutional neural network is trained based on images captured by a sensor similar to the imaging sensor.

2. The apparatus of claim 1 , wherein the convolutional neural network is trained based on a training set comprising pairs of images captured using fast-exposure and long-exposure shutter speeds.

3. The apparatus of claim 1 , wherein the convolutional neural network is trained based on an L1 loss, an L2 loss, or a structural similarity (SSIM) loss.

4. The apparatus of claim 1 , wherein the convolutional neural network is trained using an Adam optimizer, a gradient decent optimizer, a proximal gradient descent optimizer, an RMSProp optimizer, a Momentum optimizer, or an Adadelta optimizer.

5. The apparatus of claim 1 , wherein the convolutional neural network is trained using an amplification ratio based on an exposure difference between an input training image and a corresponding reference image comprising an image captured using a longer shutter speed.

6. The apparatus of claim 1 , wherein the illuminated image comprises a 12-channel image having a spatial resolution comprising half the spatial resolution of the raw sensor data, the apparatus comprising a sub-pixel layer to process the illuminated image to generate the illuminated image comprising a resolution equal to the resolution of the raw sensor data.

7. The apparatus of claim 1 , wherein the convolutional neural network is trained for a specific camera sensor using a training set of images captured using the specific camera sensor.

8. The apparatus of claim 1 , wherein the convolutional neural network comprises a fully convolutional network (FCN).

9. The apparatus of claim 1 , wherein the convolutional neural network comprises a multi-scale context aggregation network.

10. The apparatus of claim 1 , wherein the convolutional neural network comprises a U-net architecture.

11. A method for low-light imaging, comprising:

receiving, via a pipelined processor, raw sensor data from an imaging sensor;

subtracting a black level from packed data to generate black-level-subtracted packed data, the black level generated from the raw sensor data;

amplifying the black-level-subtracted packed data based on a received amplification ratio to generate amplified black-level-subtracted packed data and sending the amplified black-level-subtracted packed data to a trained convolutional neural network; and

generating, via the trained convolutional neural network of the pipelined processor, an illuminated image based on the amplified black-level-subtracted packed data, wherein the trained convolutional neural network is trained based on images captured by a sensor similar to the imaging sensor.

12. The method of claim 11 , comprising packing the raw sensor data into four channels comprising a reduced spatial resolution by a factor of two in each direction to generate packed data and inputting the packed data into the trained convolutional neural network, wherein the raw sensor data comprises a Bayer array.

13. The method of claim 11 , comprising arranging the raw sensor data into 6×6 blocks and pack the raw sensor data into nine channels by exchanging adjacent elements of the 6×6 blocks to generate packed data and inputting the packed data into the trained convolutional neural network, wherein the raw sensor data comprises an X-Trans array.

14. The method of claim 11 , comprising processing the illuminated image to generate the illuminated image comprising a resolution equal to the resolution of the raw sensor data.

15. The method of claim 11 , comprising performing, via the trained convolutional neural network, blind noise suppression on the raw sensor data.

16. The method of claim 11 , comprising performing, via the trained convolutional neural network, a color transformation on the raw sensor data.

17. The method of claim 11 , comprising processing the illuminated image using histogram stretching.

18. The method of claim 11 , comprising processing the illuminated image using high dynamic resolution (HDR) tone mapping.

19. At least one computer readable medium for imaging in low-light environments having instructions stored therein that, in response to being executed on a computing device, cause the computing device to:

receive raw sensor data from an imaging sensor; and

generate a black level from the raw sensor data and subtract the black level from packed data to generate black-level-subtracted packed data;

amplify the black-level-subtracted packed data based on a received amplification ratio to generate amplified black-level-subtracted packed data to be input into a convolutional neural network; and

generate, via the convolutional neural network trained using a training set of images captured using a sensor of a similar type as the imaging sensor, an illuminated image based on the amplified black-level-subtracted packed data.

20. The at least one computer readable medium of claim 19 , comprising instructions to pack the raw sensor data into four channels comprising a reduced spatial resolution by a factor of two in each direction to generate packed data, wherein the raw sensor data comprises a Bayer array.

21. The at least one computer readable medium of claim 19 , comprising instructions to arrange the raw sensor data into 6×6 blocks and pack the raw sensor data into nine channels by exchanging adjacent elements of the 6×6 blocks to generate packed data, wherein the raw sensor data comprises an X-Trans array.