IP Library Granted Patent US 12,361,576
Granted Patent B2
US 12,361,576 · App. 17/885,573 · Granted Jul 15, 2025

Proximity sensing using structured light

Inventors: Jonathan Hauser (Ramat Gan, IL); Assaf Avraham (Givatayim, IL); Dror Leshem (Maale Efraim, IL); Refael Della Pergola (Jerusalem, IL); Shay Keren-Zur (Tel Aviv, IL)
Assignee: Apple Inc.
G06T7/521G06T7/74
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,361,576
App. No.
17/885,573
Granted
Jul 15, 2025
Kind
B2
Abstract

Sensing apparatus includes a projector, which projects toward a target a pattern of spots extending over a predefined angular range about a projection axis. An imaging assembly includes an image sensor and objective optics, which form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis. A processor is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range.

Claims (28)

1. Sensing apparatus, comprising:

a projector, which is configured to project toward a target a pattern of spots extending over a predefined angular range about a projection axis;

an imaging assembly, comprising an image sensor and objective optics, which are configured to form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and

a processor, which is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range,

wherein the processor is configured to find, responsively to the detected part of the pattern, a location of the edge of the angular range in the image, and to estimate the distance to the target within the second span responsively to the location of the edge by a process of triangulation based on the location of the edge in the image and an offset between the projection axis and the imaging axis.

2. The apparatus according to claim 1 , wherein the processor is further configured to detect, when the pattern is absent from the image, that the target is located within a third span of distances, nearer to the apparatus than the second span.

3. The apparatus according to claim 2 , wherein within the second span of the distances, the image contains bars formed by the spots projected onto the target, and wherein the processor is configured to detect whether the target is in the second span or the third span of the distances responsively to a presence or absence of the bars in the image.

4. The apparatus according to claim 1 , wherein the projector comprises a diffractive optical element (DOE), and wherein the spots correspond to respective diffraction orders of the DOE.

5. The apparatus according to claim 1 , wherein the processor is configured to find the location of the edge by identifying one or more local extrema in an intensity profile of the image.

6. Sensing apparatus, comprising:

a projector, which is configured to project toward a target a pattern of spots extending over a predefined angular range about a projection axis;

an imaging assembly, comprising an image sensor and objective optics, which are configured to form on the image sensor an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and

a processor, which is configured to process signals output by the image sensor so as generate, while the target is within a first span of distances from the apparatus, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the apparatus than the first span, a distance to the target from the apparatus by detecting in the image a part of the pattern located at an edge of the angular range,

wherein the processor is configured to store reference data indicating respective expected locations in the image of the spots at the edge of the angular range for each of a plurality of values of the distance within the second span, and to estimate the distance to the target within the second span by matching actual locations of the spots in the image of the pattern to the expected locations of the spots in the stored reference data.

7. The apparatus according to claim 6 , wherein the processor is configured to find respective displacements between the actual locations and the expected locations of each of the spots for two or more of the values of the distance of the reference data, and to select a value of the distance to the target responsively to the respective displacements.

8. The apparatus according to claim 7 , wherein the processor is configured to compute respective scores for the two or more of the values of the distance based on the respective displacements, and to choose the value of the distance, responsively to the respective scores, at which the expected locations of the spots optimally match the actual locations.

9. The apparatus according to claim 6 , wherein the imaging axis is offset relative to the projection axis along a baseline direction, and wherein the spots in the projected pattern are arranged along rows that are angled obliquely relative to the baseline direction.

10. A method for sensing, comprising:

projecting from a projector toward a target a pattern of spots extending over a predefined angular range about a projection axis;

capturing an image of the target along an imaging axis, which is offset transversely relative to the projection axis; and

processing the image so as generate, while the target is within a first span of distances of the projector, a depth map of the target responsively to shifts of the spots in the image, and to estimate, while the target is within a second span of distances, nearer to the projector than the first span, a distance to the target by detecting in the image a part of the pattern located at an edge of the angular range,

wherein processing the image comprises finding, responsively to the detected part of the pattern, a location of the edge of the angular range in the image, and estimating the distance to the target within the second span by applying a process of triangulation based on the location of the edge in the image and an offset between the projection axis and the imaging axis.

11. The method according to claim 10 , wherein processing the image comprises detecting, when the pattern is absent from the image, that the target is located within a third span of distances, nearer to the projector than the second span.

12. The method according to claim 11 , wherein within the second span of the distances, the image contains bars formed by the spots projected onto the target, and wherein processing the image comprises detecting whether the target is in the second span or the third span of the distances responsively to a presence or absence of the bars in the image.

13. The method according to claim 10 , wherein the projector comprises a diffractive optical element (DOE), and wherein the spots correspond to respective diffraction orders of the DOE.

14. The method according to claim 10 , wherein processing the image comprises storing reference data indicating respective expected locations in the image of the spots at the edge of the angular range for each of a plurality of values of the distance within the second span, and estimating the distance to the target within the second span by matching actual locations of the spots in the image of the pattern to the expected locations of the spots in the stored reference data.

15. The method according to claim 14 , wherein estimating the distance comprises finding respective displacements between the actual locations and the expected locations of each of the spots for two or more of the values of the distance of the reference data, and selecting a value of the distance to the target responsively to the respective displacements.

16. The method according to claim 14 , wherein the imaging axis is offset relative to the projection axis along a baseline direction, and wherein the spots in the projected pattern are arranged along rows that are angled obliquely relative to the baseline direction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 16, 2022
From: HAUSER, JONATHAN; AVRAHAM, ASSAF; LESHEM, DROR; DELLA PERGOLA, REFAEL; KEREN-ZUR, SHAY
To: APPLE INC.
Reel/Frame 060813/0679 →
Continuity (2)
Provisional Application 63241002 · Sep 6, 2021
Related Publication 20230073962A1 · Mar 9, 2023
References Cited (7)
US 8384997B2 · Shpunt et al. · 2013 [cited by applicant]
US 8456517B2 · Spektor et al. · 2013 [cited by applicant]
US 10303866B1 · Van Os · 2019 [cited by examiner]
US 20200015899A1 · Scheib et al. · 2020 [cited by applicant]
US 20200289216A1 · Denlinger et al. · 2020 [cited by applicant]
US 20210165098A1 · Kimura · 2021 [cited by examiner]
Alnahhas et al., U.S. Appl. No. 17/549,917, filed Dec. 14, 2021. [cited by applicant]