IP Library Granted Patent US 9,912,884
Granted Patent B2
US 9,912,884 · App. 15/115,270 · Granted Mar 6, 2018

Methods and apparatus for superpixel modulation

Inventors: Tomas Kovacovsky (Bratislava, SK); Michal Maly (Lutila, SK); Jan Zizka (Bratislava, SK)
Assignee: Photoneo, s.r.o.
H04N5/353G01S7/4863G01S17/89H04N5/2352H04N5/378H04N13/0246H04N13/0253H04N13/0296
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,912,884
App. No.
15/115,270
Granted
Mar 6, 2018
Kind
B2
Abstract

In illustrative implementations, a set of separate modulation signals simultaneously modulates a plurality of pixels in a superpixel by a set of separate modulation signals, such that each pixel in the superpixel is modulated by a modulation signal that causes sensitivity of the pixel to vary over time. Each superpixel comprises multiple pixels. In some implementations, the sensitivity of a pixel to incident light is controlled by storage modulation or by light modulation. In some implementations, this invention is used for 3D scanning, i.e., for detection of the 3D position of points in a scene.

Claims (86)

1. A method comprising simultaneously modulating imaging pixels in an imaging superpixel of a light sensor by a set of separate modulation signals, in such a way that each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time, wherein:

(a) the pixels comprise pixel sets and each pixel set comprises multiple pixels to which a single modulation signal is applied;

(b) each of the pixel sets is modulated by a separate modulation signal;

(c) the superpixel includes at least one pixel from each of the pixel sets, such that different pixels in the superpixel are separately modulated; and

(d) the set of modulation signals, taken together, comprise a code, each of the modulation signals representing a digit of the code.

2. A method comprising simultaneously modulating imaging pixels in an imaging superpixel of a light sensor by a set of separate modulation signals, in such a way that each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time, wherein:

(a) the pixels comprise pixel sets and each pixel set comprises multiple pixels to which a single modulation signal is applied;

(b) each of the pixel sets is modulated by a separate modulation signal;

(c) the superpixel includes at least one pixel from each of the pixel sets, such that different pixels in the superpixel are separately modulated;

(d) the modulating occurs during a readout-frame, which readout-frame includes an exposure period; and

(e) each imaging pixel, respectively, in the imaging superpixel includes a photodiode that, during the exposure period, transfers electric charge to only one storage device, in such a way that at multiple times during the exposure period, electric charge is transferred from the photodiode to the storage device.

3. A method comprising simultaneously modulating imaging pixels in an imaging superpixel of a light sensor by a set of separate modulation signals, in such a way that each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time, wherein:

(a) the pixels comprise pixel sets and each pixel set comprises multiple pixels to which a single modulation signal is applied;

(b) each of the pixel sets is modulated by a separate modulation signal;

(c) the superpixel includes at least one pixel from each of the pixel sets, such that different pixels in the superpixel are separately modulated;

(d) the modulating occurs during a readout-frame, which readout-frame includes an exposure period; and

(e) each imaging pixel, respectively, in the imaging superpixel includes a photodiode that, at multiple times during the exposure period, transfers electric charge to a set of storage devices, in such a way that

(i) at any given time at which a transfer of electric charge from the photodiode to a storage device occurs, electric charge is being transferred to only one of the storage devices, and electric charge is not being transferred to any other storage device, and

(ii) electric charge is transferred from the photodiode to each of the storage devices during the exposure period.

4. A method comprising simultaneously modulating imaging pixels in an imaging superpixel of a light sensor by a set of separate modulation signals, in such a way that each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time, wherein:

(a) the pixels comprise pixel sets and each pixel set comprises multiple pixels to which a single modulation signal is applied;

(b) each of the pixel sets is modulated by a separate modulation signal;

(c) the superpixel includes at least one pixel from each of the pixel sets, such that different pixels in the superpixel are separately modulated;

(d) during the modulating of the pixels, one or more light sources illuminate a scene by a pattern of light that changes over time;

(e) the pattern of light sweeps across a scene, in such a way that a point in the scene, when directly illuminated by the pattern of light, reflects light directly to the imaging superpixel; and

(f) the method further comprises one or more computers processing data indicative of measurements recorded by the imaging superpixel and, based in part on the data, calculating 3D spatial coordinates of the point.

5. The method of claim 4 , wherein, when calculating 3D spatial coordinates of the point, the computer performs an algorithm that either:

(a) involves triangulation; or

(b) involves accessing data that was calculated by a computer using an algorithm that involves triangulation.

6. A system comprising, in combination:

(a) a camera that includes

(i) pixel sets, each pixel set comprising multiple imaging pixels, and

(ii) an imaging superpixel; and

(b) one or more signal generators for producing a set of separate modulation signals, in such a way that

(i) each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time,

(ii) a single modulation signal is applied to each pixel set, and

(iii) each of the pixel sets is modulated by a separate modulation signal; wherein

(A) the superpixel includes at least one pixel from each of the pixel sets such that, when the set of modulation signals is produced, different pixels in the superpixel are separately modulated,

(B) the superpixel includes a photodiode, and

(C) a modulation signal, out of the set of modulation signals, causes the photodiode, at multiple times during an exposure period of a read-out frame, to transfer electric charge from the photodiode to only one storage device, which storage device is part of the system and does not receive electric charge from any other photodiode during the exposure period.

7. A system comprising, in combination:

(a) a camera that includes

(i) pixel sets, each pixel set comprising multiple imaging pixels, and

(ii) an imaging superpixel; and

(b) one or more signal generators for producing a set of separate modulation signals, in such a way that

(i) each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time,

(ii) a single modulation signal is applied to each pixel set, and

(iii) each of the pixel sets is modulated by a separate modulation signal; wherein

(A) the superpixel includes at least one pixel from each of the pixel sets such that, when the set of modulation signals is produced, different pixels in the superpixel are separately modulated,

(B) the superpixel includes a photodiode, and

(C) a modulation signal, out of the set of modulation signals, causes the photodiode, at multiple times during an exposure period of a read-out frame, to transfer electric charge from the photodiode to a set of storage devices which are part of the system, in such a way that

(1) at any given time at which a transfer of electric charge from the photodiode to a storage device occurs, electric charge is being transferred to only one of the storage devices, and electric charge is not being transferred to any other storage device, and

(2) electric charge is transferred from the photodiode to each of the storage devices during the exposure period.

8. A system comprising, in combination:

(a) a camera that includes

(i) pixel sets, each pixel set comprising multiple imaging pixels, and

(ii) an imaging superpixel; and

(b) one or more signal generators for producing a set of separate modulation signals, in such a way that

(i) each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time,

(ii) a single modulation signal is applied to each pixel set, and

(iii) each of the pixel sets is modulated by a separate modulation signal; wherein

(A) the superpixel includes at least one pixel from each of the pixel sets such that, when the set of modulation signals is produced, different pixels in the superpixel are separately modulated,

(B) the system further comprises

(1) one or more light sources for illuminating a scene by a pattern of light that sweeps across the scene, in such a way that

(I) a point in the scene, when directly illuminated by the pattern of light, reflects light directly to the imaging superpixel, and

(II) the set of separate modulation signals occur while the scene is illuminated by the pattern of light and

(2) one or more computers for processing data indicative of measurements recorded by the imaging superpixel and for calculating, based on this data, 3D spatial coordinates of the point, by performing an algorithm that either

(I) involves triangulation or

(II) involves accessing data that was calculated by a computer using an algorithm that involves triangulation.

9. A system comprising, in combination:

(a) a camera that includes

(i) pixel sets, each pixel set comprising multiple imaging pixels, and

(ii) an imaging superpixel; and

(b) one or more signal generators for producing a set of separate modulation signals, in such a way that

(i) each respective imaging pixel is modulated by a modulation signal that causes sensitivity of the respective imaging pixel to vary over time,

(ii) a single modulation signal is applied to each pixel set, and

(iii) each of the pixel sets is modulated by a separate modulation signal; wherein

(A) the superpixel includes at least one pixel from each of the pixel sets such that, when the set of modulation signals is produced, different pixels in the superpixel are separately modulated,

(B) the set of modulation signals, taken together, comprise a code, each of the modulation signals representing a digit of the code,

(C) the system includes one or more light sources for producing a sweep of a pattern of light across a scene, which sweep occurs in a time period that consists of a set of subintervals, the total number of subintervals being equal to the number of permutations of the code,

(D) during each subinterval, the modulation signals, taken together, represent only one of the permutations of the code,

(E) intensity of light incident on the imaging superpixel reaches a maximum value for the entire sweep during a specific subinterval, and

(F) the system further comprises one or more computers for

(1) processing data indicative of measurements recorded by the imaging superpixel,

(2) calculating the permutation of the code represented by the modulation signals during the specific subinterval, and

(3) calculating 3D spatial coordinates of a point in the scene.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2025
From: PHOTONEO S.R.O.
To: ZEBRA TECHNOLOGIES SLOVAKIA S.R.O.
Reel/Frame 070627/0600 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2016
From: KOVACOVSKY, TOMAS; MALY, MICHAL; ZIZKA, JAN
To: PHOTONEO S.R.O.
Reel/Frame 039291/0943 →
Continuity (2)
Provisional Application 61947282 · Mar 3, 2014
Related Publication 20170195589A1 · Jul 6, 2017