Compressive sensing based image processing
Some embodiments provide a novel compressive-sensing image capture device and a method of using data captured by the compressive-sensing image capture device. The novel compressive-sensing image capture device includes an array of sensors for detecting electromagnetic radiation. Each sensor in the sensor array has an associated mask that blocks electromagnetic radiation from portions of the sensor. In some embodiments, an array of passive masks is used to block a particular set of areas of each sensor in the sensor array. In some embodiments, the image capture device also includes an array of lenses corresponding to the sensors of the sensor array such that each sensor receives light that passes through a different lens. Some embodiments of the invention provide a dynamic mask array. In some embodiments, a novel machine trained network is provided that processes image capture data captured by the compressive-sensing image capture device to predict solutions to problems.
1 . A compressive-sensing camera comprising:
a set of sensors comprising a plurality of sensors;
a set of lenses comprising a plurality of focusing lenses, wherein at least two lenses focus light on different sets of sensors in the plurality of sensors; and
a liquid crystal display (LCD) mask in front of the set of sensors that produces a plurality of LCD patterns including transparent regions and occluding regions, wherein a first LCD pattern of the plurality of LCD patterns occludes light transmission to a first portion of the set of sensors while allowing light transmission to a second portion of the set of sensors, and wherein a second LCD pattern of the plurality of LCD patterns occludes light transmission to a third portion of the set of sensors while allowing light transmission to a fourth portion of the set of sensors.
2 . The camera of claim 1 , wherein the LCD mask is printed on each sensor of the set of sensors.
3 . The camera of claim 1 , wherein the LCD mask is printed on a substrate associated with the set of sensors.
4 . The camera of claim 3 , wherein the LCD mask is printed on a particular surface of the substrate, wherein the particular surface is nearest to a surface of a particular sensor.
5 . The camera of claim 1 , wherein the LCD mask is printed on the set of lenses.
6 . The camera of claim 1 wherein each sensor of the set of sensors is associated with a particular lens in the set of lenses, each particular lens focusing incident light on a particular sensor associated with the particular lens.
7 . The camera of claim 6 , wherein each sensor in the set of sensors comprises one of a green, red, and blue sensor for capturing data associated with green, red, and blue light respectively.
8 . The camera of claim 1 , wherein a compressed-image capture operation comprises capturing data from each sensor for each different LCD pattern of the plurality of LCD patterns produced by the LCD mask.
9 . The camera of claim 1 , wherein the LCD mask is placed in a path traveled by light from an input scene to the set of sensors.
10 . The camera of claim 1 , wherein each sensor of the set of sensors is associated with a particular lens in the set of lenses, each particular lens focusing light passing through the transparent regions of the LCD mask onto the sensor associated with the lens.
11 . The camera of claim 1 , wherein the set of lenses are condensing lenses that focus light on a particular sensor of the plurality of sensors.
12 . The camera of claim 11 , wherein the condensing lenses are metalenses.
13 . The camera of claim 12 , wherein a particular metalens comprises an array of features that are deposited on a substrate.
14 . The camera of claim 13 , wherein the array of features is made from titanium dioxide or silicon.
15 . The camera of claim 1 , wherein the plurality of LCD patterns from a basis for a frequency domain.
16 . A method for generating compressed images, the method comprising:
focusing, using a set of lenses comprising a plurality of focusing lenses, light on a set of sensors comprising a plurality of sensors, wherein at least two lenses focus light on different sensors in the plurality of sensors, and wherein a liquid crystal display (LCD) mask in front of the plurality of sensors produces a plurality of LCD patterns including transparent regions and occluding regions, wherein a first LCD pattern of the plurality of LCD patterns occludes light transmission to a first portion of the set of sensors while allowing light transmission to a second portion of the set of sensors, and wherein a second LCD pattern of the plurality of LCD patterns occludes light transmission to a third portion of the set of sensors while allowing light transmission to a fourth portion of the set of sensors;
capturing, by portions of sensors of the plurality of sensors associated with the transparent regions of the plurality of LCD patterns and based on the focused light, image data; and
storing the image data in memory.