Imaging element, imaging device, and information processing method with image restoration pixels and unidirectional pixel
An imaging element, an imaging device, and an information processing method are disclosed. In one example, an imaging element includes pixel output units configured for independently setting incident angle directivity for incident light incident through both of an imaging lens and a pinhole. The pixel output units include image restoration pixel output units arranged in a matrix, at least some of the pixel output units having the incident angle directivity both in a row direction and in a column direction of the matrix, and a unidirectional pixel output unit having the incident angle directivity only in the row direction of the matrix or only in the column direction of the matrix.
1. An imaging element comprising:
a plurality of pixels each having a configuration capable of independently setting incident angle directivity for incident light incident without passage through an imaging lens and a pinhole, wherein
the plurality of pixels includes
a plurality of image restoration pixels arranged in a matrix, at least some of the image restoration pixels having the incident angle directivity both in a row direction and in a column direction of the matrix, and
a unidirectional pixel having the incident angle directivity only in the row direction of the matrix or only in the column direction of the matrix, wherein the unidirectional pixel is formed in a region other than an effective pixel region that is a region in which the image restoration pixels are arranged.
2. The imaging element according to claim 1 , wherein
the unidirectional pixel is formed in an external region outside the effective pixel region.
3. The imaging element according to claim 2 , wherein
the unidirectional pixel has the incident angle directivity only in the column direction of the matrix and is formed in the external region outside a rectangular effective pixel region in the row direction of the matrix.
4. The imaging element according to claim 2 , wherein
the unidirectional pixel has the incident angle directivity only in the row direction of the matrix and is formed in the external region outside a rectangular effective pixel region in the column direction of the matrix.
5. The imaging element according to claim 2 , wherein
the unidirectional pixel is one of a plurality of unidirectional pixels, and at least one of the image restoration pixels formed in the effective pixel region has a same incident angle directivity as at least some of the unidirectional pixels formed in external regions in the row direction and in the column direction of the matrix.
6. The imaging element according to claim 5 , wherein
at least one of the image restoration pixels in each row of the matrix has a same incident angle directivity in the column direction of the matrix as at least one of the unidirectional pixels provided in a same row, of the external region.
7. The imaging element according to claim 5 , wherein
at least one of the image restoration pixels in each column of the matrix has a same incident angle directivity in the row direction of the matrix as at least one of the unidirectional pixels provided in a same column of the external region.
8. The imaging element according to claim 5 , wherein
at least one of the image restoration pixels in each row of the matrix has the same incident angle directivity in the column direction of the matrix as at least one of the unidirectional pixels provided in another row of the external region.
9. The imaging element according to claim 5 , wherein
at least one of the image restoration pixels in each column of the matrix has a same incident angle directivity in the row direction of the matrix as at least one of the unidirectional pixels provided in another column of the external region.
10. The imaging element according to claim 5 , wherein
all of the image restoration pixels that are formed in the effective pixel region have a same incident angle directivity as at least one of a plurality of unidirectional pixels formed in external regions in the row direction and in the column direction of the matrix.
11. The imaging element according to claim 5 , wherein
some of the image restoration pixels formed in the effective pixel region have the same incident angle directivity as at least one of the unidirectional pixels formed in the external regions in the row direction and in the column direction of the matrix.
12. The imaging element according to claim 11 , wherein
the incident angle directivities in the row direction and in the column direction of the matrix, of other image restoration pixels than the some of the image restoration pixels formed in the effective pixel region, are able to be estimated on a basis of the incident angle directivity of the unidirectional pixel formed in the external region.
13. The imaging element according to claim 2 , wherein
at least one of the unidirectional pixel or one of the image restoration pixels is arranged in an arrangement order based on predetermined regularity of the incident angle directivity.
14. The imaging element according to claim 1 , wherein
a light shielding film is arranged for independently setting the incident angle directivity for each of the pixels.
15. The imaging element according to claim 1 , further comprising:
a processor configured to generate a restoration matrix to be used when generating a restored image from a detection image obtained in the image restoration pixels, using calibration information that is information regarding calibration of the incident angle directivity of the image restoration pixels generated using the unidirectional pixel.
16. The imaging element according to claim 15 , further comprising:
a memory configured to store the generated restoration matrix.
17. An imaging device comprising:
an imaging element including
a plurality of pixels each having a configuration capable of independently setting incident angle directivity for incident light incident without passage through an imaging lens and a pinhole,
the plurality of pixels including
a plurality of image restoration pixels arranged in a matrix, at least some of the pixels having the incident angle directivity both in a row direction and in a column direction of the matrix, and
a unidirectional pixel having the incident angle directivity only in the row direction of the matrix or only in the column direction of the matrix, wherein the unidirectional pixel is formed in a region other than an effective pixel region that is a region in which the image restoration pixels are arranged; and
a processor configured to generate, using an output of the unidirectional pixel, a restoration matrix to be used when generating a restored image from a detection image that is an output of the plurality of image restoration pixels.
18. The imaging device according to claim 17 , wherein
the processor is configured to generate the restoration matrix, using calibration information that is information generated using the output of the unidirectional pixel, and is regarding calibration of the incident angle directivity of the image restoration pixels.
19. An information processing method comprising:
generating, using an output of a unidirectional pixel, a restoration matrix to be used when generating a restored image from a detection image that is an output of a plurality of image restoration pixels, of
an imaging element including
a plurality of pixels each having a configuration capable of independently setting incident angle directivity for incident light incident without passage through an imaging lens and a pinhole,
the plurality of pixels including
the plurality of image restoration pixels arranged in a matrix, at least some of the image restoration pixels having the incident angle directivity both in a row direction and in a column direction of the matrix, and
the unidirectional pixel having the incident angle directivity only in the row direction of the matrix or only in the column direction of the matrix, wherein the unidirectional pixel is formed in a region other than an effective pixel region that is a region in which the image restoration pixels are arranged.
20. The information processing method according to claim 19 , wherein
the restoration matrix is generated using calibration information generated using an output from the unidirectional pixel.
21. The information processing method according to claim 20 , further comprising:
measuring a positional gap of a light-shielding film having a configuration capable of independently setting the incident angle directivity of the imaging element for each of the pixels, and generating the calibration information including information regarding the positional gap of the light-shielding film.
22. The information processing method according to claim 20 , further comprising:
measuring a height gap of an on-chip lens of the imaging element, and generating the calibration information including information regarding the height gap of the on-chip lens.
23. The information processing method according to claim 20 , further comprising:
in the unidirectional pixel, measuring a detection signal value for each of a plurality of incident angles different from one another of the incident light, and generating the calibration information using each measurement result.
24. The information processing method according to claim 20 , further comprising:
in the unidirectional pixel, measuring a detection signal value for a predetermined incident angle of the incident light, and generating the calibration information using a measurement result and a reference value.
25. The information processing method according to claim 20 , wherein the unidirectional pixel is one of a plurality of unidirectional pixels, and further comprising:
measuring a detection signal value of each of the plurality of the unidirectional pixels, and generating the calibration information using a statistical value of measurement results.