Spectral reconstruction
This disclosure relates to processing colour images to reconstruct hyperspectral images from the colour images. An image processor determines an output hyperspectral image by determining for each point of the output hyperspectral image a combination of multiple hyperspectral prototype components to correspond the output hyperspectral image to the input image. Each of the multiple hyperspectral prototype components comprises multiple component values associated with respective wavelengths and each of the multiple component values is based on multiple points of training image data associated with the wavelength of that component value. Since the component values are based on multiple points of the training image, relationships between points in the training image, such as texture, can be used to reconstruct the hyperspectral image data, which leads to a more robust and more accurate reconstruction.
1. A method for processing an input image, the method comprising:
determining an output hyperspectral image by determining for each point of the output hyperspectral image a combination of multiple hyperspectral prototype components to correspond the output hyperspectral image to the input image;
wherein
each of the multiple hyperspectral prototype components comprise multiple component values associated with respective wavelengths; and
each of the multiple component values is based on multiple points of training image data associated with the wavelength of that component value.
2. The method of claim 1 , further comprising determining for each wavelength of the input colour image an input image value associated with that colour, wherein
the input image value for each wavelength is based on multiple points of the input image associated with that colour,
determining the combination comprises determining combined component values of the combination, and
to correspond the output hyperspectral image to the input colour image comprises to correspond the combined component values to the input image values.
3. The method of claim 1 , wherein each of the multiple hyperspectral prototype components comprise multiple intensity values associated with the respective wavelengths and determining a combination of the multiple hyperspectral prototype components comprises determining for each wavelength a weighted sum of the intensity values associated with that wavelength.
4. The method of claim 1 , wherein the multiple component values are based on convolutional features.
5. The method of claim 4 , wherein the convolutional features are based on filters.
6. The method of claim 5 , wherein the filters comprise one or more of:
Leung-Malik Filter Bank;
Schmidt Filter Bank; and
Maximum Response Filter Bank.
7. The method of claim 1 , further comprising performing the method based on sparse coding.
8. The method of claim 7 , wherein the sparse coding comprises constraint sparse coding.
9. The method of claim 1 , further comprising determining the multiple hyperspectral prototype components based on the training image data.
10. The method of claim 9 , wherein determining the multiple hyperspectral prototype components comprises optimising a cost function based on the multiple hyperspectral prototype components.
11. The method of claim 1 , further comprising determining an illuminant spectrum based on the output hyperspectral image.
12. The method of claim 1 , further comprising processing the output hyperspectral image for one or more of:
Material-specific colour rendition;
Accurate colour reproduction (white balance);
Material reflectance substitution; and
Compressive sensing.
13. A non-transitory computer-readable medium with program code stored thereon that, when executed by a computer causes the computer to perform the method of claim 1 .
14. A computer system for processing an input image, the computer system comprising:
a data port to receive the input image;
a processor to determine an output hyperspectral image by determining for each point of the output hyperspectral image a combination of multiple hyperspectral prototype components to correspond the output hyperspectral image to the input image,
wherein
each of the multiple hyperspectral prototype components comprise multiple component values associated with respective wavelengths; and
each of the multiple component values is based on multiple points of training image data associated with the wavelength of that component value; and
a data store for storing the multiple hyperspectral prototype components and the output hyperspectral image.
15. The computer system of claim 14 further comprising an image sensor to generate the input image.
16. The computer system of claim 15 , wherein the image sensor is a three-colour sensor.
17. The computer system of claim 15 further comprising a micro-mirror device to generate hyperspectral training image data with a low spatial resolution and three-colour input image data with a high spatial resolution, wherein the processor is to determine the multiple hyperspectral prototype components based on the hyperspectral training image data with the low spatial resolution.