Compressive sensing based image processing
Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.
1. A method for analyzing image data captured as a set of compressed-image coefficients, the method comprising:
receiving, from a compressed-image capture device comprising a set of sensors and an image-compressing mask in front of the set of sensors, a set of compressed-image coefficient values associated with a particular compressively-captured image, each compressed-image coefficient value in the set of compressed-image coefficient values associated with a particular sensor in the set of sensors; and
processing, using a plurality of machine-trained nodes, the received set of compressed-image coefficient values to determine content of the particular compressively-captured captured image,
wherein the image-compressing mask comprises a dynamic-pattern mask array, the dynamic-pattern mask array producing a plurality of different masking patterns, each masking pattern differently occluding light transmission to portions of sensors in the set of sensors while allowing light transmission to other portions of sensors in the set of sensors.
2. The method of claim 1 , wherein the plurality of machine-trained nodes is a plurality of machine-trained nodes of the compressed-image capture device.
3. The method of claim 1 , wherein the machine-trained nodes are machine-trained nodes in a machine-trained network.
4. The method of claim 1 , wherein the machine-trained nodes are neurons in a neural network.
5. The method of claim 4 , wherein the machine-trained nodes process received values using linear and non-linear functions.
6. The method of claim 5 , wherein an output of a linear function is used as an input of a non-linear function.
7. The method of claim 6 , wherein the non-linear function is one of a sigmoid function, a tanh function, a rectified linear unit (ReLU) function, or a leaky ReLU function.
8. The method of claim 1 , wherein the set of compressed-image coefficient values comprises a plurality of subsets of compressed-image coefficient values, each subset of compressed-image coefficient values associated with a particular masking pattern in the plurality of different masking patterns.
9. The method of claim 1 , wherein the dynamic-pattern mask array comprises a liquid crystal display (LCD) mask comprising a plurality of regions, wherein the LCD mask produces a plurality of LCD patterns comprising occluding and transparent regions of the LCD mask to produce the plurality of different masking patterns.
10. The method of claim 9 , wherein the LCD mask is placed in a path traveled by light from an input scene to the set of sensors.
11. The method of claim 10 , wherein the set of sensors has an associated set of lenses, wherein each sensor of the set of sensors is associated with a particular lens in the set of lenses, each particular lens focusing light passing through the transparent regions of the LCD mask onto the sensor associated with the lens.
12. The method of claim 1 , wherein the dynamic-pattern mask array comprises at least two fixed-pattern mask arrays, wherein the at least two fixed-pattern mask arrays are moved relative to each other to produce the plurality of different masking patterns.
13. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for analyzing image data captured as a set of compressed-image coefficients, the program comprising sets of instructions for:
receiving, from a compressed-image capture device comprising a set of sensors and an image-compressing mask in front of the set of sensors, a set of compressed-image coefficient values associated with a particular compressively-captured image, each compressed-image coefficient value in the set of compressed-image coefficient values associated with a particular sensor in the set of sensors; and
processing, using a plurality of machine-trained nodes, the received set of compressed-image coefficient values to determine content of the compressively-captured captured image,
wherein the image-compressing mask occludes light transmission to portions of sensors in the set of sensors while allowing light transmission to other portions of sensors in the set of sensors.
14. The non-transitory machine readable medium of claim 13 , wherein the set of compressed-image coefficient values are associated with a set of basis functions, each basis function representing a particular sampling of areas of a scene captured as a compressed image.
15. The non-transitory machine readable medium of claim 14 , wherein each sensor in the set of sensors is associated with a particular basis function and wherein the image-compressing mask comprises a plurality of patterns of occluded regions, each pattern associated with a sensor in the set of sensors, each pattern physically representing the basis function associated with the sensor.
16. The non-transitory machine readable medium of claim 15 , wherein an output of each sensor in the set of sensors is a measurement of light incident on the sensor through the image-compressing mask and is interpreted as a compressed-image coefficient value that is equivalent to the result of a projection of an image of the captured scene onto the basis function.
17. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for analyzing image data captured as a set of compressed-image coefficients, the program comprising sets of instructions for:
receiving, from a compressed-image capture device comprising a set of sensors and an image-compressing mask in front of the set of sensors, a set of compressed-image coefficient values associated with a particular compressively-captured image, each compressed-image coefficient value in the set of compressed-image coefficient values associated with a particular sensor in the set of sensors; and
processing, using a plurality of machine-trained nodes, the received set of compressed-image coefficient values to determine content of the compressively-captured captured image,
wherein the image-compressing mask comprises one of a fixed-pattern mask array printed on a substrate that is coupled to the set of sensors and a fixed-pattern mask array printed on the sensors of the set of sensors.