IP Library Granted Patent US 9,330,324
Granted Patent B2
US 9,330,324 · App. 13/541,775 · Granted May 3, 2016

Error compensation in three-dimensional mapping

Inventors: Daniel Cohen (Tel-Aviv, IL); Dmitri Rais (Ramat-Gan, IL); Erez Sali (Savion, IL); Niv Galezer (Tel-Aviv, IL); Alexander Shpunt (Tel-Aviv, IL)
Assignee: APPLE INC.
G06K9/2036G06T7/0057H04N13/0253H04N13/0271G06K2209/40G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,330,324
App. No.
13/541,775
Granted
May 3, 2016
Kind
B2
Abstract

A method for forming a three-dimensional (3D) map of an object, including illuminating the object from a light source so as to project a pattern onto the object, capturing an image of the pattern using an array of detector elements, and processing the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets including at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array. The method further includes computing a correction factor in response to the first offset, applying the correction factor to the second offset so as to find a corrected offset, and computing depth coordinates of the object in response to the corrected offset.

Claims (49)

1. A method for forming a three-dimensional (3D) map of an object, comprising:

illuminating the object from a light source so as to project a pattern onto the object;

capturing an image of the pattern on the object using an array of detector elements;

processing the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets comprising at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array;

computing a geometrical correction factor in response to the first offset;

applying the geometrical correction factor to the second offset so as to find a corrected offset; and

computing depth coordinates of the object in response to the corrected offset.

2. The method according to claim 1 , wherein the at least first offset of the first element comprises a plurality of first offsets of respective first elements.

3. The method according to claim 2 , and wherein computing the geometrical correction factor comprises fitting the plurality of first offsets to an equation so as to determine constants of the equation.

4. The method according to claim 3 , wherein the corrected offset comprises a corrected second offset, and wherein applying the geometrical correction factor comprises using at least one of the constants to find the corrected second offset.

5. The method according to claim 3 , wherein the equation comprises a two-dimensional linear equation.

6. The method according to claim 3 , wherein the equation comprises a two-dimensional quadratic equation.

7. The method according to claim 3 , and comprising using the equation to predict a subsequent value of the first offset.

8. The method according to claim 1 , and comprising correcting at least one of the offsets for distortion in the image.

9. The method according to claim 8 , wherein correcting for the distortion comprises capturing a reference image of the pattern, storing parameters of the reference image in a look-up table, and using the look-up table to correct the at least one of the offsets.

10. The method according to claim 1 , and comprising correcting the at least first offset for a geometrical misalignment between the light source and the array of detectors.

11. The method according to claim 10 , wherein correcting for the geometrical misalignment comprises using a captured reference image to determine the misalignment.

12. The method according to claim 10 , wherein correcting the at least first offset for the geometrical misalignment comprises correcting for a change in the geometrical misalignment.

13. The method according to claim 1 , wherein the first offsets comprise a deviation in response to a change of wavelength of the light source, and wherein the geometrical correction factor is configured to correct for the change of wavelength.

14. The method according to claim 1 , wherein the first offsets comprise a deviation in response to a change of focal length in optics forming the image, and wherein the geometrical correction factor is configured to correct for the change of focal length.

15. The method according to claim 14 , and comprising measuring a temperature of the optics, and determining the change in focal length in response to the temperature.

16. The method according to claim 1 , wherein a line defined by the light source and the array is parallel to one of the mutually-perpendicular directions.

17. The method according to claim 1 , wherein the pattern comprises primary speckles.

18. The method according to claim 1 , wherein the pattern comprises a regular geometric pattern.

19. An apparatus for forming a three-dimensional (3D) map of an object, comprising:

a light source configured to illuminate the object so as to project a pattern onto the object;

an array of detector elements configured to capture an image of the pattern on the object; and

a processor which is configured to:

process the captured image so as to measure respective offsets of elements of the pattern in the captured image relative to a reference pattern, the offsets comprising at least a first offset of a first element of the pattern and a second offset of a second element of the pattern, measured respectively in first and second, mutually-perpendicular directions in a plane of the array,

compute a geometrical correction factor in response to the first offset;

apply the geometrical correction factor to the second offset so as to find a corrected offset; and

compute depth coordinates of the object in response to the corrected offset.

20. The apparatus according to claim 19 , wherein the at least first offset of the first element comprises a plurality of first offsets of respective first elements.

21. The apparatus according to claim 20 , wherein computing the geometrical correction factor comprises fitting the plurality of first offsets to an equation so as to determine constants of the equation.

22. The apparatus according to claim 21 , wherein the corrected offset comprises a corrected second offset, and wherein applying the geometrical correction factor comprises using at least one of the constants to find the corrected second offset.

23. The apparatus according to claim 21 , wherein the equation comprises a two-dimensional linear equation.

24. The apparatus according to claim 21 , wherein the equation comprises a two-dimensional quadratic equation.

25. The apparatus according to claim 21 , and comprising using the equation to predict a subsequent value of the first offset.

26. The apparatus according to claim 19 , wherein the processor is configured to correct at least one of the offsets for distortion in the image.

27. The apparatus according to claim 26 , wherein correcting for the distortion comprises capturing a reference image of the pattern, storing parameters of the reference image in a look-up table, and using the look-up table to correct the at least one of the offsets.

28. The apparatus according to claim 19 , and comprising correcting the at least first offset for a geometrical misalignment between the light source and the array of detectors.

29. The apparatus according to claim 28 , wherein correcting for the geometrical misalignment comprises using a captured reference image to determine the misalignment.

30. The apparatus according to claim 28 , wherein correcting the at least first offset for the geometrical misalignment comprises correcting for a change in the geometrical misalignment.

31. The apparatus according to claim 19 , wherein the first offsets comprise a deviation in response to a change of wavelength of the light source, and wherein the geometrical correction factor is configured to correct for the change of wavelength.

32. The apparatus according to claim 19 , and comprising optics forming the image, and wherein the first offsets comprise a deviation in response to a change of focal length in the optics, and wherein the geometrical correction factor is configured to correct for the change of focal length.

33. The apparatus according to claim 32 , and comprising a temperature sensor configured to measure a temperature of the optics, and wherein the processor is configured to determine the change in focal length in response to the temperature.

34. The apparatus according to claim 19 , wherein a line defined by the light source and the array is parallel to one of the mutually-perpendicular directions.

35. The apparatus according to claim 19 , wherein the pattern comprises primary speckles.

36. The apparatus according to claim 19 , wherein the pattern comprises a regular geometric pattern.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION # 13840451 AND REPLACE IT WITH CORRECT APPLICATION # 13810451 PREVIOUSLY RECORDED ON REEL 034293 FRAME 0092. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded May 11, 2015
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 035624/0091 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2014
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 034293/0092 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 5, 2012
From: COHEN, DANIEL; RAIS, DMITRI; SALI, EREZ; GALEZER, NIV; SHPUNT, ALEXANDER
To: PRIMESENSE LTD.
Reel/Frame 028490/0074 →
Continuity (6)
Continuation In Part 12605340 · Oct 25, 2009
Continuation In Part 12282517
Continuation In Part PCTIL2006000335 · Mar 14, 2006
Provisional Application 60785187 · Mar 24, 2006
Provisional Application 60724903 · Oct 11, 2005
Related Publication 20120281240A1 · Nov 8, 2012