IP Library Granted Patent US 10,739,447
Granted Patent B2
US 10,739,447 · App. 15/699,623 · Granted Aug 11, 2020

Systems, methods, and media for encoding and decoding signals used in time of flight imaging

Inventors: Felipe Gutierrez Barragan (Madison, WI); Mohit Gupta (Madison, WI); Andreas Velten (Madison, WI); Eric Breitbach (Madison, WI); Shree K. Nayar (New York, NY)
Assignees: Wisconsin Alumni Research Foundation; The Trustees of Columbia University in the City of New York
G01S7/4911G01S7/484G01S7/4865G01S7/4915G01S17/10G01S17/32G01S17/89G06T7/521H04N5/2256H04N5/232G01S17/36G06T2207/10028G06T2207/10152
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,739,447
App. No.
15/699,623
Granted
Aug 11, 2020
Kind
B2
Abstract

In accordance with some embodiments, systems, methods and media for encoding and decoding signals used in time-of-flight imaging are provided. In some embodiments, a method for estimating the depth of a scene is provided, comprising: causing a light source to emit modulated light toward the scene based on a modulation function; causing the image sensor to generate a first value based on the modulated light and a first demodulation function of K modulation functions; causing the image sensor to generate a second value; causing the image sensor to generate a third value; and determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and three correlation functions each including at least one half of a trapezoid wave.

Claims (48)

1. A system for estimating the depth of a scene, the system comprising:

a light source;

an image sensor comprising at least one pixel;

a signal generator configured to output at least:

a first signal corresponding to a modulation function, wherein the modulation function comprises a square wave; and

a second signal corresponding to a first demodulation function of K demodulation functions; and

at least one hardware processor that is programmed to:

cause the light source to emit first modulated light toward the scene with modulation based on the first signal;

cause the image sensor to generate a first value based on the first modulated light received from a portion of the scene and the second signal;

cause the image sensor to generate a second value based on second modulated light received from the portion of the scene, and a third signal corresponding to a second demodulation function of the K demodulation functions;

cause the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and

determine a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

2. The system of claim 1 , wherein K is at least four.

3. The system of claim 1 , wherein the at least one hardware processor is further programmed to cause the light source to emit the first modulated light during a first time period, the second modulated light during a second time period, and the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

4. The system of claim 3 , wherein the at least one hardware processor is further programmed to cause the light source to emit the first modulated light after the second time period has elapsed.

5. The system of claim 1 , wherein the modulation function has a duty cycle of one sixth of an integration period τ.

6. The system of claim 1 , wherein each of the K demodulation functions includes at least one square wave, and the K demodulation functions correspond to K correlation functions that each include at least one half of a trapezoid wave.

7. The system of claim 1 , wherein the signal generator comprises multiple signal generators.

8. A method for estimating the depth of a scene, the method comprising:

causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising a square wave;

causing an image sensor to generate a first value based on the first modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions;

causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;

causing the image sensor to generate a third value based on the third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and

determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

9. The method of claim 8 , wherein K is at least four.

10. The method of claim 8 , wherein causing a light source to emit modulated light toward the scene comprises:

causing the light source to emit the first modulated light during a first time period;

causing the light source to emit the second modulated light during a second time period; and

causing the light source to emit the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

11. The method of claim 10 , further comprising causing the light source to emit the first modulated light after the second time period has elapsed.

12. The method of claim 9 , wherein the modulation function has a duty cycle of one sixth of an integration period τ.

13. The method of claim 9 , wherein each of the K demodulation functions includes at least one square wave, and the K demodulation functions correspond to K correlation functions that each include at least one half of a trapezoid wave.

14. The method of claim 8 , wherein signal generator comprises multiple signal generators.

15. A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating the depth of a scene, the method comprising:

causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising a square wave;

causing an image sensor to generate a first value based on the modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions;

causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;

causing the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and

determining a depth estimate for the portion of the scene based on the first value, the second value, the third value, and at least three correlation functions each including at least one half of a trapezoid wave.

16. The non-transitory computer readable medium of claim 15 , wherein K is at least four.

17. The non-transitory computer readable medium of claim 15 , wherein causing a light source to emit modulated light toward the scene comprises:

causing the light source to emit the modulated light during a first time period;

causing the light source to emit the modulated light during a second time period; and

causing the light source to emit the first modulated light during a third time period, wherein the first value is based on the second modulated light emitted during the first time period, the second value is based on the third modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

18. The non-transitory computer readable medium of claim 17 , wherein the method further comprises causing the light source to emit the first modulated light after the second time period has elapsed.

19. The non-transitory computer readable medium of claim 15 , wherein the modulation function has a duty cycle of one sixth of an integration period τ.

20. The non-transitory computer readable medium of claim 15 , wherein each of the K demodulation functions includes at least one square wave, and the K demodulation functions correspond to K correlation functions that each include at least one half of a trapezoid wave.

21. The non-transitory computer readable medium of claim 15 , wherein the signal generator comprises multiple signal generators.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2020
From: NAYAR, SHREE
To: THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK
Reel/Frame 051630/0689 →
CONFIRMATORY LICENSE Recorded Feb 5, 2018
From: WISCONSIN ALUMNI RESEARCH FOUNDATION
To: NAVY, SECRETARY OF THE UNITED STATES OF AMERICA
Reel/Frame 045281/0430 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2017
From: GUTIERREZ BARRAGAN, FELIPE; GUPTA, MOHIT; VELTEN, ANDREAS; BREITBACH, ERIC
To: WISCONSIN ALUMNI RESEARCH FOUNDATION
Reel/Frame 044027/0181 →
Continuity (2)
Continuation In Part 15492969 · Apr 20, 2017
Related Publication 20190146073A1 · May 16, 2019
Cited By (2)
US 12,387,292 US 12,511,789