IP Library Granted Patent US 10,645,367
Granted Patent B2
US 10,645,367 · App. 15/492,969 · Granted May 5, 2020

Systems, methods, and media for encoding and decoding signals used in time of flight imaging

Inventors: Mohit Gupta (Madison, WI); Eric Breitbach (Madison, WI); Andreas Velten (Madison, WI); Shree K. Nayar (New York, NY)
Assignees: Wisconsin Alumni Research Foundation; The Trustees of Columbia University in the City of New York
H04N13/271G01S7/4915G01S17/32G01S17/89H04N5/232H04N13/161H04N13/254H04N13/275G01S17/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,645,367
App. No.
15/492,969
Granted
May 5, 2020
Kind
B2
Abstract

In accordance with some embodiments, systems, methods and media for encoding and decoding signals used in time-of-flight imaging are provided. In some embodiments, a method for estimating the depth of a scene is provided, comprising: causing a light source to emit modulated light toward the scene based on a modulation function; causing the image sensor to generate a first value based on the modulated light and a first demodulation function of K modulation functions, including at least one trapezoid wave; causing the image sensor to generate a second value; causing the image sensor to generate a third value; and determining a depth estimate for the portion of the scene based on the first value, the second value, and the third value.

Claims (48)

1. A system for estimating the depth of a scene, the system comprising:

a light source;

an image sensor comprising at least one pixel;

a signal generator configured to output at least:

a first signal corresponding to a modulation function, wherein the modulation function comprises an impulse function; and

a second signal corresponding to a first demodulation function of K demodulation functions, wherein the second signal includes at least one trapezoid wave; and

at least one hardware processor that is programmed to:

cause the light source to emit first modulated light toward the scene with modulation based on the first signal;

cause the image sensor to generate a first value based on the first modulated light received from a portion of the scene and the second signal;

cause the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;

cause the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and

determine a depth estimate for the portion of the scene based on the first value, the second value, and the third value.

2. The system of claim 1 , wherein K=4.

3. The system of claim 1 , wherein the at least one hardware processor is further programmed to cause the light source to emit the first modulated light during a first time period, the second modulated light during a second time period, and the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

4. The system of claim 1 , wherein the K demodulation functions correspond to K correlation functions of a Hamiltonian coding scheme.

5. The system of claim 4 , wherein the Hamiltonian coding scheme has a coding curve in a K-dimensional space corresponding to the K correlation functions of the Hamiltonian coding scheme that substantially corresponds to a Hamiltonian cycle on a K-dimensional hypercube, with two vertices omitted.

6. The system of claim 1 , wherein the K demodulation functions and K modulation functions, including the modulation function, characterize a coding scheme having a coding curve with a length in K-dimensional space that is at least five.

7. The system of claim 1 , wherein the signal generator comprises multiple signal generators.

8. A method for estimating the depth of a scene, the method comprising:

causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising an impulse function;

causing an image sensor to generate a first value based on the first modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions and the second signal includes at least one trapezoid wave;

causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;

causing the image sensor to generate a third value based on the third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions of the Hamiltonian coding scheme; and

determining a depth estimate for the portion of the scene based on the first value, the second value, and the third value.

9. The method of claim 8 , wherein K=4.

10. The method of claim 8 , wherein causing a light source to emit modulated light toward the scene comprises:

causing the light source to emit the first modulated light during a first time period;

causing the light source to emit the second modulated light during a second time period; and

causing the light source to emit the third modulated light during a third time period, wherein the first value is based on the modulated light emitted during the first time period, the second value is based on the modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

11. The method of claim 8 , wherein the K demodulation functions correspond to K correlation functions of a Hamiltonian coding scheme.

12. The method of claim 11 , wherein the Hamiltonian coding scheme has a coding curve in a K-dimensional space corresponding to the K correlation functions of the Hamiltonian coding scheme that substantially corresponds to a Hamiltonian cycle on a K-dimensional hypercube, with two vertices omitted.

13. The method of claim 8 , wherein the K demodulation functions and K modulation functions, including the modulation function, characterize a coding scheme having a coding curve with a length in K-dimensional space that is at least five.

14. The method of claim 8 , wherein signal generator comprises multiple signal generators.

15. A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating the depth of a scene, the method comprising:

causing a light source to emit first modulated light toward the scene with modulation based on a first signal output by a signal generator, wherein the first signal corresponds to a modulation function comprising an impulse function;

causing an image sensor to generate a first value based on the modulated light received from a portion of the scene and a second signal output by the signal generator, wherein the second signal corresponds to a first demodulation function of K demodulation functions and the second signal includes at least one trapezoid wave;

causing the image sensor to generate a second value based on second modulated light received from the portion of the scene and a third signal corresponding to a second demodulation function of the K demodulation functions;

causing the image sensor to generate a third value based on third modulated light received from the portion of the scene and a fourth signal corresponding to a third demodulation function of the K demodulation functions; and

determining a depth estimate for the portion of the scene based on the first value, the second value, and the third value.

16. The non-transitory computer readable medium of claim 15 , wherein K=4.

17. The non-transitory computer readable medium of claim 15 , wherein causing a light source to emit modulated light toward the scene comprises:

causing the light source to emit the modulated light during a first time period;

causing the light source to emit the modulated light during a second time period; and

causing the light source to emit the first modulated light during a third time period, wherein the first value is based on the second modulated light emitted during the first time period, the second value is based on the third modulated light emitted during the second time period, and the third value is based on the modulated light emitted during the third time period.

18. The non-transitory computer readable medium of claim 15 , wherein the K demodulation functions correspond to K correlation functions of a Hamiltonian coding scheme.

19. The non-transitory computer readable medium of claim 18 , wherein the Hamiltonian coding scheme has a coding curve in K-dimensional space corresponding to the K correlation functions of the Hamiltonian coding scheme that substantially corresponds to a Hamiltonian cycle on a K-dimensional hypercube, with two vertices omitted.

20. The non-transitory computer readable medium of claim 15 , wherein the K demodulation functions and K modulation functions, including the modulation function, characterize a coding scheme having a coding curve with a length in K-dimensional space that is at least five.

21. The non-transitory computer readable medium of claim 15 , wherein the signal generator comprises multiple signal generators.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2020
From: NAYAR, SHREE
To: THE TRUSTEES OF COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK
Reel/Frame 051630/0689 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 2, 2017
From: GUPTA, MOHIT; VELTEN, ANDREAS; BREITBACH, ERIC
To: WISCONSIN ALUMNI RESEARCH FOUNDATION
Reel/Frame 043751/0807 →
Continuity (1)
Related Publication 20180309970A1 · Oct 25, 2018
Cited By (1)
US 12,511,789