Compressive sensing based image capture device
Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.
1. A camera comprising:
a set of sensors comprising a plurality of sensors;
an image-compressing mask in front of the set of sensors to occlude light from a scene that is directed to portions of sensors in the set of sensors while allowing light from the scene to other portions of sensors in the set of sensors to be captured in a compressive-imaging operation; and
an image processing circuit to receive compressed-image output from the set of sensors and to perform image-processing operations on the sensor-array output,
wherein the output of the set of sensors is a set of compressed-image coefficients that are associated with a set of basis functions, each basis function representing a particular sampling of areas of a scene captured as a compressed image,
the image-compressing mask comprising a plurality of patterns of occluded and transparent portions, each pattern associated with a sensor in the set of sensors, each pattern physically representing the basis function associated with the sensor, and
the output of each sensor in the set of sensors being a measurement of radiation incident on the sensor through the image compressing mask and interpreted as a compressed-image coefficient that is equivalent to a result of a projection of an image of the captured scene onto the basis function.
2. The camera of claim 1 , wherein the image-compressing mask is printed on the sensors of the set of sensors.
3. The camera of claim 1 , wherein the image-compressing mask is printed on a substrate associated with the set of sensors.
4. The camera of claim 1 , wherein the captured light is at least one of infrared radiation, visible light, and ultraviolet light.
5. The camera of claim 1 further comprising an aperture through which light associated with a scene to be captured passes to arrive at the sensors of the set of sensors without further optical processing.
6. The camera of claim 1 further comprising a lens used to focus radiation associated with a scene to be captured onto the set of sensors.
7. The camera of claim 1 , wherein each sensor in the set of sensors comprises a set of green, red, green, and blue (GRGB) sensor sub-assemblies for capturing data associated with GRGB light respectively.
8. The camera of claim 7 , wherein the image-compressing mask occludes light transmission to identical portions of each GRGB sensor sub-assembly for a particular sensor in the set of sensors while allowing light transmission to identical portions of each GRGB sensor sub-assembly for the particular sensor in the set of sensors.
9. The camera of claim 7 , wherein the image-compressing mask occludes light transmission to different portions of each GRGB sensor sub-assembly for a particular sensor in the set of sensors while allowing light transmission to different portions of each GRGB sensor sub-assembly for the particular sensor in the set of sensors.
10. The camera of claim 1 , wherein, during a video capture operation, data from each sensor is sampled at a set of times that, on average, are separated by an amount of time larger than an amount of time that would separate a set of equally-spaced times for capturing frames at the frame rate of the video resulting from the video capture operation.
11. The camera of claim 1 , wherein the set of sensors comprises 1,000 to 25,000 individual sensors.
12. The camera of claim 11 , wherein the output of the set of sensors is used to reproduce an image equivalent to an image captured by an image capture device with more than 1 million sensors.
13. The camera of claim 1 , wherein the image processing circuit comprises a machine trained network of processing nodes.