IP Library Granted Patent US 10,762,651
Granted Patent B2
US 10,762,651 · App. 15/721,640 · Granted Sep 1, 2020

Real time calibration for time-of-flight depth measurement

Inventors: David Cohen (Nesher, IL); Assaf Pellman (Bet-Yehoshua, IL); Shai Mazor (Binyamina, IL); Erez Tadmor (Atlit, IL); Giora Yahav (Haifa, IL)
Assignee: Magic Leap, Inc.
G06T7/521G01S7/483G01S7/4813G01S7/497G01S17/08G01S17/89G06T2207/10004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,762,651
App. No.
15/721,640
Granted
Sep 1, 2020
Kind
B2
Abstract

A method for determining a distance to a target object includes transmitting light pulses to illuminate the target object and sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the transmitted light pulses. The feedback optical device includes a preset reference depth. The method includes calibrating time-of-flight (TOF) depth measurement reference information based on the sensed light in the first region of the pixel array. The method further includes sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses. The distance of the target object is determined based on the sensed reflected light and the calibrated TOF measurement reference information.

Claims (39)

1. A time-of-flight (TOF) imaging system, comprising:

an illuminator to transmit light pulses to illuminate a target object for determining a distance to the target object;

an image sensor having a light-sensitive pixel array to receive optical signals from the light pulses, the pixel arrays including an active region and a feedback region;

an optical feedback device for directing a portion of the light from the illuminator to the feedback region of the pixel array, the optical feedback device including a preset reference depth;

wherein the imaging system is configured to:

transmit light pulses to illuminate a target object;

sense, in the feedback region of the pixel array, light from the optical feedback device, using a sequence of shutter windows that includes delay times representing a range of depth;

calibrate time-of-flight (TOF) depth measurement reference information based on the sensed light in the feedback region of the pixel array;

sense, in the active region of the light-sensitive pixel array, light reflected from the target object; and

determine the distance of the target object based on the sensed reflected light and the calibrated TOF measurement reference information.

2. The system of claim 1 , wherein the feedback region is in an edge region of light-sensitive pixel array.

3. The system of claim 1 , wherein light-sensitive pixel array further comprises an isolation region separating the active region and the feedback sensing region.

4. The system of claim 1 , wherein the feedback region of the pixel array comprises from one to 10×10 pixels.

5. The system of claim 1 , wherein the optical feedback device comprises a reflector.

6. The system of claim 1 , wherein the optical feedback device comprises an optical fiber.

7. The system of claim 6 , wherein the optical fiber is configured to couple light from inside an illuminator housing to the feedback region at a corner of the pixel array.

8. The system of claim 1 , wherein the TOF measurement reference information comprises a look-up table correlating distance of an object to a ratio between two sampled light signals using two shutters with different time delays.

9. The system of claim 1 , wherein sensing light reflected from the target object comprises sensing the light using two shutters with different time delays.

10. The system of claim 1 , wherein the calibration takes about 1/300 of a frame period.

11. The system of claim 1 , further comprising multiple illumination sources and corresponding optical feedback regions.

12. The system of claim 1 , wherein the imaging system is configured to calibrate time-of-flight (TOF) depth measurement reference information after sensing, in the active region of the light-sensitive pixel array, light reflected from the target object.

13. In a digital camera characterized by a preset frame rate, a method comprising: in a single frame period,

transmitting light pulses to illuminate a target object;

sensing, in a first region of a light-sensitive pixel array, light provided from an optical feedback device that receives a portion of the transmitted light pulses, the feedback optical device including a preset reference depth, wherein the light from the optical feedback device is sampled using a sequence of shutter windows that includes delay times representing a range of distance;

calibrating time-of-flight (TOF) depth measurement reference information based on the sensed light in the first region of the pixel array;

sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses; and

determining a distance of the target object based on the sensed reflected light and the calibrated TOF measurement reference information.

14. The method of claim 13 , wherein the optical feedback device comprises an optical fiber.

15. The method of claim 13 , wherein the optical fiber is configured to couple light from inside an illuminator housing to the first region at a corner of the pixel array.

16. The method of claim 13 , wherein, wherein the TOF measurement reference information comprises a look-up table correlating distance of an object to a ratio between two sampled light signals using two shutters with different time delays.

17. The method of claim 13 , wherein sensing light reflected from the target object comprises sensing light using two shutters with different time delays.

18. A method for calibrating a time-of-flight (TOF) camera system, comprising:

transmitting light pulses to illuminate a target object for determining a distance to the target object;

sensing, in a first region of a light-sensitive pixel array, light provided from a feedback optic device that receives a portion of the transmitted light pulses, the feedback optical device including a preset reference depth;

sensing, in a second region of the light-sensitive pixel array, light reflected from the target object from the transmitted light pulses, using a sequence of shutter windows that includes delay times representing a range of depth;

calibrating time-of-flight depth measurement information based on the light provided from the feedback optical device; and

determining the distance of the target object based on light reflected from the target object and the calibrated time-of-flight depth measurement information.

19. The method of claim 18 , wherein sensing light reflected from the target object comprises sensing light using two shutters with different time delays.

20. The method of claim 18 , wherein the TOF measurement reference information comprises a look-up table correlating distance of an object to a ratio between two sampled light signals using two shutters with different time delays.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2019
From: COHEN, DAVID; PELLMAN, ASAF; MAZOR, SHAI; TADMOR, EREZ; YAHAV, GIORA
To: MAGIC LEAP, INC.
Reel/Frame 051154/0706 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
Continuity (2)
Provisional Application 62402770 · Sep 30, 2016
Related Publication 20180096489A1 · Apr 5, 2018