IP Library Granted Patent US 11,474,249
Granted Patent B2
US 11,474,249 · App. 16/556,119 · Granted Oct 18, 2022

Systems, methods, and media for stochastic exposure coding that mitigates multi-camera interference in continuous wave time-of-flight imaging

Inventors: Mohit Gupta (Madison, WI); Jongho Lee (Madison, WI)
Assignee: Wisconsin Alumni Reseach Foundation
G01S17/89G01S17/10G06T7/521
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,474,249
App. No.
16/556,119
Granted
Oct 18, 2022
Kind
B2
Abstract

In accordance with some embodiments, systems, methods and media for stochastic exposure coding for continuous time-of-flight imaging are provided. In some embodiments, a method for estimating the depth of a scene is provided, comprising: stochastically selecting active slots based on a probability p; causing, during active slots, a light source to emit light modulated by a first modulation function toward a scene; causing, during active slots, an image sensor to generate a first, second, and third value based on received light from a portion of the scene and a first, second, and third demodulation function, respectively; inhibiting the light source during inactive slots; determining, for each of the active slots, depth estimates for the portion of the scene based on the first, second, and third value; and determining a depth estimate for the portion of the scene based on the depth estimates for the active slots.

Claims (89)

1. A system for estimating the depth of a scene, the system comprising:

a light source;

an image sensor comprising at least one pixel;

a signal generator configured to output at least:

a first signal corresponding to a modulation function;

at least one hardware processor that is programmed to:

stochastically select, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;

cause, during each of the active slots, the light source to emit first modulated light toward the scene with modulation based on the first signal;

cause, during each of the active slots, the image sensor to generate a first value based on the light received from a portion of the scene and a second signal corresponding to a first demodulation function;

cause, during each of the active slots, the image sensor to generate a second value based on light received from the portion of the scene and a third signal corresponding to a second demodulation function;

cause, during each of the active slots, the image sensor to generate a third value based on light received from the portion of the scene and a fourth signal corresponding to a third demodulation function;

inhibit, during each of the inactive slots, the light source from emitting modulated light toward the scene;

determine, for each of a plurality of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determine, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the plurality of the active slots.

2. The system of claim 1 , the at least one hardware processor that is further programmed to:

determine, for each active slot, a total intensity received by the image sensor during the active slot;

determine that a clash has occurred for each active slot in which the total intensity exceeds a threshold; and

in response to determining that a clash has occurred for a particular active slot, exclude that active slot from the plurality of active slots.

3. The system of claim 2 , wherein the at least one hardware processor is further programmed to:

determine a mean total intensity value based on the total intensity values for each active slot; and

set the threshold based on a standard deviation of the total intensity values and the mean total intensity value.

4. The system of claim 1 , wherein each slot has a duration of in a range of 0.5 ms to 1.0 ms.

5. The system of claim 1 , wherein the at least one hardware processor that is further programmed to:

determine, for all of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determine, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the active slots.

6. The system of claim 1 , wherein the probability p of activating each slot of the plurality of slots is based on the relationship

p

=

1

A

0

,

where A 0 represents a peak power of the light source.

7. A method for estimating the depth of a scene, the method comprising:

stochastically selecting, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;

causing, during each of the active slots, a light source to emit first modulated light toward the scene with modulation based on a first signal corresponding to a first modulation function;

causing, during each of the active slots, an image sensor to generate a first value based on the light received from a portion of the scene and a second signal corresponding to a first demodulation function;

causing, during each of the active slots, the image sensor to generate a second value based on light received from the portion of the scene and a third signal corresponding to a second demodulation function;

causing, during each of the active slots, the image sensor to generate a third value based on light received from the portion of the scene and a fourth signal corresponding to a third demodulation function;

inhibiting, during each of the inactive slots, the light source from emitting modulated light toward the scene;

determining, for each of a plurality of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the plurality of the active slots.

8. The method of claim 7 , further comprising:

determining, for each active slot, a total intensity received by the image sensor during the active slot;

determining that a clash has occurred for each active slot in which the total intensity exceeds a threshold; and

in response to determining that a clash has occurred for a particular active slot, excluding that active slot from the plurality of active slots.

9. The method of claim 8 , further comprising:

determining a mean total intensity value based on the total intensity values for each active slot; and

setting the threshold based on a standard deviation of the total intensity values and the mean total intensity value.

10. The method of claim 7 , wherein each slot has a duration of in a range of 0.5 ms to 1.0 ms.

11. The method of claim 7 , further comprising:

determining, for all of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the active slots.

12. The method of claim 7 , wherein the probability p of activating each slot of the plurality of slots is based on the relationship

p

=

1

A

0

,

where A 0 represents a peak power of the light source.

13. A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for estimating the depth of a scene, the method comprising:

stochastically selecting, from a plurality of slots each corresponding to a portion of a total capture time, a first subset of the plurality of slots as active slots and a second subset of the plurality of slots as inactive slots based on a probability p of activating each slot of the plurality of slots;

causing, during each of the active slots, a light source to emit first modulated light toward the scene with modulation based on a first signal corresponding to a first modulation function;

causing, during each of the active slots, an image sensor to generate a first value based on the light received from a portion of the scene and a second signal corresponding to a first demodulation function;

causing, during each of the active slots, the image sensor to generate a second value based on light received from the portion of the scene and a third signal corresponding to a second demodulation function;

causing, during each of the active slots, the image sensor to generate a third value based on light received from the portion of the scene and a fourth signal corresponding to a third demodulation function;

inhibiting, during each of the inactive slots, the light source from emitting modulated light toward the scene;

determining, for each of a plurality of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the plurality of the active slots.

14. The non-transitory computer-readable medium of claim 13 , wherein the method further comprises:

determining, for each active slot, a total intensity received by the image sensor during the active slot;

determining that a clash has occurred for each active slot in which the total intensity exceeds a threshold; and

in response to determining that a clash has occurred for a particular active slot, excluding that active slot from the plurality of active slots.

15. The non-transitory computer-readable medium of claim 14 , wherein the method further comprises:

determining a mean total intensity value based on the total intensity values for each active slot; and

setting the threshold based on a standard deviation of the total intensity values and the mean total intensity value.

16. The non-transitory computer-readable medium of claim 13 , wherein each slot has a duration of in a range of 0.5 ms to 1.0 ms.

17. The non-transitory computer-readable medium of claim 13 , wherein the method further comprises:

determining, for all of the active slots, a depth estimate for the portion of the scene based on the first value, the second value, and the third value; and

determining, for the total capture time, a depth estimate for the portion of the scene based on the depth estimates for each of the active slots.

18. The non-transitory computer-readable medium of claim 13 , wherein the probability p of activating each slot of the plurality of slots is based on the relationship

p

=

1

A

0

,

where A 0 represents a peak power of the light source.

Assignments (2)
CONFIRMATORY LICENSE Recorded Aug 28, 2020
From: WISCONSIN ALUMNI RESEARCH FOUNDATION
To: NAVY, SECRETARY OF THE UNITED STATES OF AMERICA
Reel/Frame 053652/0869 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2019
From: GUPTA, MOHIT; LEE, JONGHO
To: WISCONSIN ALUMNI RESEARCH FOUNDATION
Reel/Frame 050429/0726 →
Continuity (1)
Related Publication 20210063576A1 · Mar 4, 2021