Imaging device, electronic device, and information processing method
An optical wavelength band to be detected is flexibly changed. An imaging device includes a coded mask including two or more kinds of band bus filters that are arranged in a two-dimensional grating pattern and that transmit light of different wavelength bands, a light receiving unit that receives modulated light modulated by the coded mask and generates observation signal data, and an image reconstruction processing unit that reconstructs the observation signal data generated by the light receiving unit to generate image data.
1. An imaging device, comprising:
a coded mask including a plurality of kind of band pass filters that are arranged in a two-dimensional grating pattern, wherein
the plurality of kind of band pass filters includes a first kind of band pass filter and a second kind of band pass filter,
a size of the first kind of band pass filter is different from a size of the second kind of band pass filter, and
the plurality of kind of band pass filters is configured to transmit light of different wavelength bands;
a color filter array including a plurality of type of color filters that transmit light of different wavelength bands;
a light receiving unit including a plurality of pixels configured to:
receive modulated light modulated by the coded mask and the color filter array, and
generate observation signal data based on the received modulated light, wherein
the observation signal data comprises a pixel value for each of the plurality of pixels, and
the plurality of pixels includes a plurality of first pixels on which the modulated light of a first wavelength band is incident and a plurality of second pixels on which the modulated light of a second wavelength band different from the first wavelength band is incident; and
an image reconstruction processing unit configured to:
reconstruct the observation signal data generated by the light receiving unit, wherein the observation signal data includes first channel data including the pixel values generated by the plurality of first pixels and second channel data including the pixel values generated by the plurality of second pixels,
restore, based on a binary matrix operation process, the first channel data and the second channel data, and
generate image data based on the restored first channel data and the second channel data.
2. The imaging device according to claim 1 , further comprising an image integration unit configured to restore original image data based on integration of one or a plurality of pieces of the image data generated by the image reconstruction processing unit.
3. The imaging device according to claim 1 , wherein the color filter array is between the light receiving unit and the coded mask.
4. The imaging device according to claim 1 , wherein the image reconstruction processing unit is further configured to:
execute the binary matrix operation process, on the first channel data, based on a first modulation matrix, and
execute the binary matrix operation process, on the second channel data, based on a second modulation matrix, wherein the second modulation matrix is a complement matrix of the first modulation matrix.
5. The imaging device according to claim 1 , further comprising:
a weighting unit configured to weight the pixel value of the first channel data based on a specific weight coefficient; and
a subtraction unit configured to subtract the second channel data from the first channel data weighted by the weighting unit or subtract the first channel data weighted by the weighting unit from the second channel data, wherein
the image reconstruction processing unit is further configured to perform the binary matrix operation process on the first channel data or the second channel data subjected to the subtraction by the subtraction unit.
6. The imaging device according to claim 5 , wherein
the weighting unit is further configured to multiply the pixel value of the first channel data by the specific weight coefficient, and
the subtraction unit is further configured to subtract the second channel data from the first channel data weighted by the weighting unit.
7. The imaging device according to claim 5 , wherein
the weighting unit is further configured to divide the pixel value of the first channel data by the specific weight coefficient, and
the subtraction unit is further configured to subtract the first channel data weighted by the weighting unit from the second channel data.
8. The imaging device according to claim 5 , wherein
the color filter array includes a first color filter configured to transmit light of a third wavelength band and a second color filter configured to transmit light of a fourth wavelength band,
the fourth wavelength band partially overlaps with the third wavelength band, and
the weight coefficient is calculated based on a degree of the overlap between the third wavelength band and the fourth wavelength band.
9. The imaging device according to claim 1 , wherein the image reconstruction processing unit is on a chip same as a chip on which the light receiving unit is mounted.
10. An electronic device, comprising:
a solid-state imaging device, wherein the solid-state imaging device includes:
a coded mask including a plurality of kind of band pass filters that are arranged in a two-dimensional grating pattern, wherein
the plurality of kind of band pass filters includes a first kind of band pass filter and a second kind of band pass filter,
a size of the first kind of band pass filter is different from a size of the second kind of band pass filter, and
the plurality of kind of band pass filters are configured to transmit light of different wavelength bands,
a color filter array including a plurality of type of color filters that transmit light of different wavelength bands, and
a light receiving unit including a plurality of pixels configured to:
receive modulated light modulated by the coded mask and the color filter array, and
generate observation signal data based on the received modulated light, wherein
the observation signal data comprises a pixel value for each of the plurality of pixels, and
the plurality of pixels includes a plurality of first pixels on which the modulated light of a first wavelength band is incident and a plurality of second pixels on which the modulated light of a second wavelength band different from the first wavelength band is incident; and
an information processing device connected to the solid-state imaging device, wherein
the information processing device includes an image reconstruction processing unit configured to:
reconstruct the observation signal data generated by the light receiving unit, wherein the observation signal data includes first channel data including the pixel values generated by the plurality of first pixels and second channel data including the pixel values generated by the plurality of second pixels,
restore, based on a binary matrix operation process, the first channel data and the second channel data, and
generate image data based on the restored first channel data and the second channel data.
11. An information processing method, comprising:
receiving, by a plurality of pixels, modulated light modulated by a color filter array and a coded mask, wherein
the color filter array includes a plurality of type of color filters that transmit light of different wavelength bands,
the coded mask includes a plurality of type of band pass filters that are arranged in a two-dimensional grating pattern,
the plurality of type of band pass filters includes a first type of band pass filter and a second type of band pass filter,
a size of the first type of band pass filter is different from a size of the second type of band pass filter, and
the plurality of type of band pass filters is configured to transmit light of different wavelength bands;
generating observation signal data based on the received modulated light, wherein
the observation signal data comprises a pixel value for each of the plurality of pixels, and
the plurality of pixels includes a plurality of first pixels on which the modulated light of a first wavelength band is incident and a plurality of second pixels on which the modulated light of a second wavelength band different from the first wavelength band is incident;
reconstructing the generated observation signal data, wherein the observation signal data includes first channel data including the pixel values generated by the plurality of first pixels and second channel data including the pixel values generated by the plurality of second pixels;
restoring, based on a binary matrix operation process, the first channel data and the second channel data; and
generating image data based on the restored first channel data and the second channel data.
12. A non-transitory computer-readable medium having stored thereon computer-executable instructions, which when executed by a processor of an imaging device, cause the processor to execute operations, the operations comprising:
receiving, by a plurality of pixels, modulated light modulated by a color filter array and a coded mask, wherein
the color filter array includes a plurality of type of color filters that transmit light of different wavelength bands,
the coded mask includes a plurality of type of band pass filters that are arranged in a two-dimensional grating pattern,
the plurality of type of band pass filters includes a first type of band pass filter and a second type of band pass filter,
a size of the first type of band pass filter is different from a size of the second type of band pass filter, and
the plurality of type of band pass filters is configured to transmit light of different wavelength bands;
generating observation signal data based on the received modulated light, wherein
the observation signal data comprises a pixel value for each of the plurality of pixels, and
the plurality of pixels includes a plurality of first pixels on which the modulated light of a first wavelength band is incident and a plurality of second pixels on which the modulated light of a second wavelength band different from the first wavelength band is incident;
reconstructing the generated observation signal data, wherein the observation signal data includes first channel data including the pixel values generated by the plurality of first pixels and second channel data including the pixel values generated by the plurality of second pixels;
restoring, based on a binary matrix operation process, the first channel data and the second channel data; and
generating image data based on the restored first channel data and the second channel data.