Color reconstruction using homogeneous neural network
Systems and methods for image processing, and specifically for color reconstruction of an output signal from a multispectral imaging sensor (MIS), are described. Embodiments of the present disclosure receive image sensor data from an image sensor of a camera device; apply a non-linear color space mapping to the image sensor data using a neural network to obtain image data, wherein the non-linear color space mapping comprises a non-negative homogeneous function; and store the image data in a memory of the camera device.
1. A method for image processing, comprising:
receiving image sensor data from an image sensor of a camera device;
applying a non-linear color space mapping to the image sensor data using a neural network to obtain image data, wherein the non-linear color space mapping comprises a non-negative homogeneous function; and
storing the image data in a memory of the camera device.
2. The method of claim 1 , wherein:
the image sensor data comprises multispectral image sensor data.
3. The method of claim 1 , further comprising:
retrieving the image data from the memory; and
displaying an image based on the image data.
4. The method of claim 1 , further comprising:
converting the image data from a first color space to a second color space, wherein the image data is stored using a format based on the second color space.
5. The method of claim 1 , further comprising:
applying a color correction to the image sensor data prior to applying the non-linear color space mapping.
6. The method of claim 1 , further comprising:
applying a color correction to the image data prior to storing the image data.
7. The method of claim 1 , wherein:
the non-linear color space mapping converts the image sensor data to a three-dimensional color space.
8. The method of claim 1 , wherein:
the non-linear color space mapping converts the image sensor data to a CIE-XYZ color space.
9. The method of claim 1 , further comprising:
applying a demosaicing process to the image sensor data prior to applying the non-linear color space mapping.
10. A method for image processing, comprising:
receiving training data including a color sample and ground-truth image data in a first color space;
obtaining image sensor data corresponding to the color sample from a sensor of a camera device;
applying a non-linear color space mapping to the image sensor data using a neural network to obtain predicted image data in the first color space; and
updating parameters of the neural network based on the predicted image data and the ground-truth image data.
11. The method of claim 10 , wherein:
the non-linear mapping comprises a non-negative homogeneous function.
12. The method of claim 10 , wherein:
the color sample comprises a plurality of colors selected based on a distribution in the first color space, and wherein the parameters of the neural network are updated based on the plurality of colors.
13. The method of claim 10 , wherein:
the training data includes a plurality of color samples and corresponding ground-truth image data for each of the plurality of color samples, and wherein the parameters of the neural network are updated based on the plurality of color samples and the corresponding ground-truth image data.
14. The method of claim 10 , further comprising:
storing the parameters of the neural network in a memory of a second camera device; and
fine-tuning the parameters of the neural network based on a sensor of the second camera device.
15. An apparatus for image processing, comprising:
an image sensor configured to capture image sensor data;
a neural network configured to apply a non-linear color space mapping to the image sensor data to obtain image data; and
a memory configured to store the image data,
wherein each node of the neural network comprises a non-negative homogeneous activation function.
16. The apparatus of claim 15 , further comprising:
a training component configured to update parameters of the neural network based on a color sample, ground-truth image data corresponding to the color sample, and predicted image data generated by the neural network.
17. The apparatus of claim 15 , further comprising:
a display component configured to display an image based on the image data.
18. The apparatus of claim 15 , further comprising:
a color correction component configured to apply a color correction to the image sensor data prior to the non-linear color space mapping.
19. The apparatus of claim 15 , wherein:
the image sensor is a multispectral image sensor (MIS), and comprises a pixel array and a multispectral filter.