IP Library › Granted Patent US 12666145
Granted Patent B2
US 12666145 · App. 18/957,796 · Granted Jun 23, 2026

Depth sensing techniques for virtual, augmented, and mixed reality systems

Inventors: Brian Keith Smith (Wellington, FL); Koon Keong Shee (Miramar, FL); Gregory Michael Link (Charlotte, NC)
Assignee: Magic Leap, Inc.
H04N23/667G01S17/894G06F3/00H04N13/139H04N13/296H04N23/959
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12666145
App. No.
18/957,796
Granted
Jun 23, 2026
Kind
B2
Abstract

Techniques for operating a depth sensor are discussed. A first sequence of operation steps and a second sequence of operation steps can be stored in memory on the depth sensor to define, respectively, a first depth sensing mode of operation and a second depth sensing mode of operation. In response to a first request for depth measurement(s) according to the first depth sensing mode of operation, the depth sensor can operate in the first mode of operation by executing the first sequence of operation steps. In response to a second request for depth measurement(s) according to the second depth sensing mode of operation, and without performing an additional configuration operation, the depth sensor can operate in the second mode of operation by executing the second sequence of operation steps.

Claims (62)

1 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a short-range and high frame rate mode of operation.

2 . The method of claim 1 , wherein the depth sensor is a time-of-flight camera.

3 . The method of claim 1 , wherein the first depth sensing mode of operation corresponds to a first distance range of measurements and the second depth sensing mode of operation corresponds to a second distance range of measurements which is different from the first distance range of measurements.

4 . The method of claim 1 , wherein the one or more first depth measurements and the one or more second depth measurements are usable in a virtual, augmented, or mixed reality display system.

5 . The method of claim 1 , wherein the first depth sensing mode of operation corresponds to a first frame rate and the second depth sensing mode of operation corresponds to a second frame rate which is different than the first frame rate.

6 . The method of claim 1 , wherein the first depth sensing mode of operation corresponds to a first frame rate and the second depth sensing mode of operation corresponds to a second frame rate which is slower than the first frame rate.

7 . The method of claim 1 , wherein at least one of the second sequence of operation steps comprises a delay.

8 . The method of claim 1 , further comprising:

receiving at least one configuration request; and

in response to the at least one configuration request, storing, in the memory, the first sequence of operation steps and the second sequence of operations steps.

9 . The method of claim 1 , wherein the short-range and high frame rate mode of operation is used for sensing depths at ranges less than about 2 m with frame rates greater than about 20 Hz.

10 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a short-range and low frame rate mode of operation.

11 . The method of claim 10 , wherein the short-range and low frame rate mode of operation is used for sensing depths at ranges less than about 2 m with frame rates less than about 20 Hz.

12 . The method of claim 10 , wherein the depth sensor is a time-of-flight camera.

13 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a long-range and high frame rate mode of operation.

14 . The method of claim 13 , wherein the long-range and high frame rate mode of operation is used for sensing depths at ranges from about 2-4 m with frame rates greater than about 20 Hz.

15 . The method of claim 13 , wherein the depth sensor is a time-of-flight camera.

16 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a long-range and low frame rate mode of operation.

17 . The method of claim 16 , wherein the long-range and low frame rate mode of operation is used for sensing depths at ranges from about 2-4 m with frame rates less than about 20 Hz.

18 . The method of claim 16 , wherein the depth sensor is a time-of-flight camera.

19 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein:

the first depth sensing mode of operation is a short-range and high frame rate mode of operation;

the second depth sensing mode of operation is a short-range and low frame rate mode of operation;

the one or more common operation steps include steps to: i) capture a short-range intensity sub-frame using a short exposure, and ii) capture four phase sub-frames;

the first sequence of operation steps includes a first delay; and

the second sequence of operation steps includes a second delay that is longer than the first delay.

20 . A method for operating a depth sensor, the method comprising:

receiving, by the depth sensor, a first request for one or more first depth measurements according to a first depth sensing mode of operation of the depth sensor, wherein the first depth sensing mode of operation is defined by a first sequence of operation steps stored in memory on the depth sensor, the first sequence of operation steps including one or more common operation steps;

in response to the first request, performing the first mode of operation by executing the first sequence of operation steps to generate the one or more first depth measurements;

receiving, by the depth sensor, a second request for one or more second depth measurements according to a second depth sensing mode of operation of the depth sensor, wherein the second depth sensing mode of operation is defined by a combination of the one or more common operations steps and a second sequence of operation steps stored in the memory on the depth sensor; and

in response to the second request and without storing additional operation steps in the memory, performing the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the one or more second depth measurements,

wherein:

the first depth sensing mode of operation is a long-range and high frame rate mode of operation;

the second depth sensing mode of operation is a long-range and low frame rate mode of operation;

the one or more common operation steps include steps to: i) capture a long-range intensity sub-frame using a long exposure, ii) capture four phase sub-frames for a first illumination modulation frequency, and iii) capture four phase sub-frames for a second illumination modulation frequency;

the first sequence of operation steps includes a first delay; and

the second sequence of operation steps includes a second delay that is longer than the first delay.