IP Library › Granted Patent US 10,215,856
Granted Patent B1
US 10,215,856 · App. 15/822,237 · Granted Feb 26, 2019

Time of flight camera

Inventor: Zhanping Xu (Sunnyvale, CA)
Assignee: Microsoft Technology Licensing, LLC
G01S17/36G01S7/497G01S7/4915G01S17/89H04N5/2256H04N5/2354
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,215,856
App. No.
15/822,237
Granted
Feb 26, 2019
Kind
B1
Abstract

A method for determining whether a distance that a CW-TOF range camera provides for a scene is degraded by multipath interference (MPI) comprising operating the camera to determine a propagation phase delay and a phase delay coefficient for each of a plurality of modulation frequencies of light that illuminates the scene and using the phase delay coefficient and/or the phase delay to determine whether a distance provided by the camera is compromised by MPI.

Claims (37)

1. A continuous wave time of flight (CW-TOF) range camera comprising:

a light source operable to transmit light to illuminate a scene that the camera images;

a photosensor having pixels configured to register amounts of light reflected by features in the scene from the transmitted light; and

a controller configured to:

control the light source to illuminate the scene with light modulated at each of a plurality of M different modulation frequencies ω m , 1≤m≤M;

for each ω m acquire frames of the photosensor that image the scene and provide for each pixel of the plurality of pixels a voltage useable to provide a range image of the scene;

for a given pixel p(i,j) and for each ω m process the frames to determine a propagation phase delay Θ(i,j,ω m ) and a phase delay coefficient B(i,j,ω m )* associated with the voltages that the frames provide for the given pixel; and

use Θ(i,j,ω m ) and/or B(i,j,ω m )* for at least two different ω m to determine whether Θ(i,j,ω m ) is degraded by multipath interference (MPI).

2. The CW-TOF range camera according to claim 1 wherein to use Θ(i,j,ω m ) for different ω m the controller is configured to determine a ratio RΘ(i,j,ω 1 ,ω 2 ) that is dependent on Θ(i,j,ω m ) evaluated for each of at least two different frequencies ω 1 and ω 2 of the plurality of frequencies.

3. The CW-TOF range camera according to claim 2 wherein RΘ(i,j,ω 1 ,ω 2 ) comprises Θ(i,j,ω 2 )/Θ(i,j,ω 1 ).

4. The CW-TOF range camera according to claim 3 wherein the controller is configured to determine whether Θ(i,j,ω m ) is degraded by MPI based on comparing RΘ(i,j,ω 1 ,ω 2 ) to ω 2 /ω 1 .

5. The CW-TOF range camera according to claim 4 wherein comparing RΘ(i,j,ω 1 ,ω 2 ) to ω 2 /ω 1 comprises determining a value for a probability function that is a function of RΘ(i,j,ω 1 ,ω 2 ) and ω 2 /ω 1 .

6. The CW-TOF range camera according to claim 2 wherein M≥3 and RΘ(i,j,ω 1 ,ω 2 )=[ω 1 /(ω 2 +ω 3 )][(Θ(i,j,ω 2 )+(Θ(i,j,ω 3 ))/Θ(i,j,ω 1 )].

7. The CW-TOF range camera according to claim 6 wherein the controller is configured to determine whether Θ(i,j,ω m ) is degraded by MPI based on comparing RΘ(i,j,ω 1 ,ω 2 ) to 1.

8. The CW-TOF range camera according to claim 7 wherein comparing RΘ(i,j,ω 1 ,ω 2 ) to 1 comprises determining a value for a probability function that is a function of RΘ(i,j,ω 1 ,ω 2 ) and 1.

9. The CW-TOF range camera according to claim 1 wherein to use B(i,j,ω m )* for different ω m the controller is configured to determine a ratio RB(i,j,ω 1 ,ω 2 ), that is dependent on B(i,j,ω m )* evaluated for each of at least two different frequencies, ω 1 and ω 2 of the plurality of frequencies.

10. The CW-TOF range camera according to claim 9 wherein RB(i,j,ω 1 ,ω 2 ) is a function of B(i,j,ω 2 )*/B(i,j,ω 1 )*.

11. The CW-TOF range camera according to claim 10 wherein the controller is configured to determine whether Θ(i,j,ω m ) is degraded by MPI based on comparing RB(i,j,ω 1 ,w 2 )* with 1.

12. The CW-TOF range camera according to claim 11 wherein comparing RB(i,j,ω 1 ,ω 2 ) to 1 comprises determining a value for a probability function that is a function of RB(i,j,ω 1 ,ω 2 ) and 1.

13. The CW-TOF range camera according to claim 9 wherein M≥3 and RB(i,j,ω 1 ,ω 2 )*=0.5(B(i,j,ω 2 )*+B(i,j,ω 3 )*)/B(i,j,ω 1 )*.

14. The CW-TOF range camera according to claim 13 wherein the controller is configured to indicate whether Θ(i,j,ω m ) is degraded by MPI based on comparing RB(i,j,ω 1 ,ω 2 ) with 1.

15. The CW-TOF range camera according to claim 14 wherein comparing RB(i,j,ω 1 ,ω 2 ) to 1 comprises determining a value for a probability function that is a function of RB(i,j,ω 1 ,ω 2 ) and 1.

16. The CW-TOF range camera according to claim 9 wherein determining RB(i,j,ω 1 ,ω 2 ) comprises:

determining a selection of pixels p(i′,j′) of the plurality of pixels in the photosensor that are relatively free of MPI;

determining an average of phase delay coefficients B(i′,j′,ω 1 )* for pixels in the selection;

normalizing B(i,j,ω 1 )* to the average; and

using the normalized B(i,j,ω 1 )* in place of B(i,j,ω 1 )* to determine RB(i,j,ω 1 ,ω 2 ).

17. The CW-TOF range camera according to claim 16 wherein a pixel p(i′,j′) is determined to be relatively free of MPI if a value of the phase delay coefficient B(i′,j′,ω 1 )* for the pixel is relatively large.

18. The CW-TOF range camera according to claim 17 wherein determining whether the phase delay coefficient B(i′,j′,ω 1 )* for pixel p(i′,j′) is relatively large comprises:

determining a maximum and a standard deviation σ B (ω 1 ) for the magnitudes of the phase delay coefficients for the pixels in the photosensor at frequency ω 1 ;

determining that the phase delay coefficient B(i′,j′,ω 1 )* for a pixel p(i′,j′) of the plurality of pixels in the photosensor is relatively large if it is within nσ B (ω m ) of the determined maximum phase delay coefficient, where n is an integer.

19. A method for determining whether a distance that a continuous wave time of flight (CW-TOF) range camera provides for a scene that the camera images is degraded by multipath interference (MPI), the method comprising:

transmitting light modulated at each of a plurality of M different modulation frequencies ω m , 1≤m≤M to illuminate the scene;

for each ω m determining voltages based on photocharge that the camera accumulates responsive to light reflected by the scene from light transmitted at ω m ;

processing the voltages for each ω m to determine a propagation phase delay Θ(ω m ) and a phase delay coefficient B(ω m )* that characterize the determined voltages; and

using the Θ(ω m ) and/or B(ω m )* determined for at least two different ω m to determine whether a phase delay Θ(ω m′ ) 1≤m′≤M and thereby the distance is degraded by multipath interference (MPI).

20. The method according to claim 19 wherein using the Θ(ω m ) and/or B(ω m )* for different ω m comprises determining a ratio RΘ(ω 1 ,ω 2 ) that is dependent on Θ(ω m ) and/or determining a ratio RB(ω 1 ,ω 2 ) that is dependent on B(ω m )*, evaluated for each of at least two different frequencies ω 1 and ω 2 of the plurality of frequencies.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2017
From: XU, ZHANPING
To: MICROSOFT TECHNOLOGY LICENSING, LLC.
Reel/Frame 044220/0592 →
Cited By (1)
US 12,230,649