Integrated display having multiple capture devices
An integrated imaging apparatus for displaying images while capturing images of a scene, including an electronic display having an array of display pixels which are used to display image content; a plurality of apertures are integrated within the display, wherein each of the apertures includes at least one partially transparent pixel; and a plurality of capture devices each of which captures an image, and includes at least a lens and an image sensor array; wherein each capture device is operatively associated with at least part of a particular aperture of the display; and wherein the partially transparent pixels also provide light for image content display.
1. An integrated imaging apparatus for displaying images while capturing images of a scene, comprising:
a) an electronic display having an array of display pixels which are used to display image content;
b) a plurality of apertures are integrated within the display, wherein each of the apertures includes at least one partially transparent pixel;
c) a plurality of capture devices each of which captures an image, and includes at least a lens and an image sensor array;
wherein each capture device is operatively associated with at least part of a particular aperture of the display; and
wherein the partially transparent pixels also provide light for image content display and wherein each aperture includes a plurality of patterned partially transparent pixels, either individually or in clusters, in a periodic or quasi-periodic manner.
2. The apparatus of claim 1 , which further comprises image processing electronics, which affect the quality of the displayed image content, or the quality of the captured images, or both.
3. The apparatus of claim 2 , wherein the image processing electronics includes means for enhancing the quality of the captured images relative to factors that includes; shadowing, frequency artifacts, spatial non-uniformities, spectral differences, or image flicker, or combinations thereof.
4. The apparatus of claim 2 wherein the image processing electronics form a composite image from captured images obtained from the plurality of image capture devices.
5. The apparatus of claim 1 wherein the partially transparent display pixels are periodically controlled to alternately emit light and emit no light.
6. The apparatus of claim 1 wherein the partially transparent pixels are one or more of a red, green, blue, or white pixels.
7. The apparatus of claim 1 , wherein the partially transparent display pixels are periodically controlled to be partially transparent.
8. The apparatus of claim 1 wherein the image capture devices are selected to span a range of different imaging attributes, which includes different focal lengths, different capture fields of view, different magnifications, different resolutions, or different spectral sensitivities, or combinations thereof.
9. The apparatus of claim 1 wherein the plurality of image capture devices have different spectral sensitivities that range both inside and outside of the visible spectrum.
10. The apparatus of claim 1 wherein there is a plurality of non-adjacent apertures having baffles that are light absorbing or light blocking provided in the intervening spaces between apertures.
11. A method for simultaneously displaying and capturing images from an integrated electronic device comprising:
a) providing an electronic display for displaying a first image and having an array of display pixels and intermingled partially transparent pixels; and
b) operating the partially transparent pixels in a periodic manner, providing both light emission and light transmission states within a frame time;
capturing a scene image with one or more image capturing devices, each or which looks though at least one of the partially transparent pixels; and
c) wherein the output intensity of the display pixels is changed within, the frame time to compensate for the operation of the partially transparent pixels.
12. The method for simultaneously displaying and capturing images according to claim 11 in which both the displayed images and the captured images are subject to image processing to improve image quality.
13. The method for simultaneously displaying and capturing images according to claim 11 wherein the display pixels are used to provide display light for the display image content during at least the portion of a frame time that the partially transparent pixels are operating to emit light.
14. The method for simultaneously displaying and capturing images according to claim 11 wherein the display pixels are used to provide display light for the display image content only during the portion of a frame time that the partially transparent pixels are operating to emit light.
15. The method for simultaneously displaying and capturing images according to claim 11 wherein the one or more image capture devices capture the scene image only during the portion of a frame time that the partially transparent pixels are operating to transmit light.
16. An integrated imaging apparatus for displaying images while capturing images of a scene, comprising:
a) an electronic display having an array of display pixels which are used to display image content;
b) a plurality of image capture devices, each of which includes at least an imaging lens and an image sensor array;
c) wherein each of the image capture devices looks through an aperture in the display, the aperture having at least one partially transparent pixel;
and wherein the partially transparent pixels also provide light for image content display;
d) wherein at least one of the apertures is formed by a plurality of partially transparent pixels;
e) wherein the plurality of partially transparent pixels forming at least one of the apertures is patterned contiguously to form the aperture; and
f) wherein the plurality of partially transparent pixels is patterned, either individually or in clusters in a periodic or quasi-periodic manner.
17. The apparatus of claim 16 wherein the plurality of image capture devices are selected to span a range of different imaging attributes, which includes different focal lengths, different capture fields of view, different magnifications, different resolutions, or different spectral sensitivities, or combinations thereof.
18. The apparatus of claim 16 , which further comprises image processing electronics, which affect the quality of the displayed image content, or the quality of the captured images, or both.
19. The apparatus of claim 18 wherein the image processing electronics form a composite image from captured images obtained from the plurality of image capture devices.
20. The apparatus of claim 16 wherein the partially transparent display pixels are periodically controlled to alternately emit light and emit no light.
21. The apparatus of claim 16 wherein the partially transparent pixels are one or more of a red, green, blue, or white pixel.
22. The apparatus of claim 16 , wherein the display pixels are in operation to provide display light for the display image content during at least the portion of a frame time that the partially transparent pixels provide image content display light.