IP Library › Granted Patent US 12,620,178
Granted Patent B1
US 12,620,178 · App. 18/142,322 · Granted May 5, 2026

Supplementing depth information for anchoring

Inventors: Rudy Poot (Mill Valley, CA); Alvin Chung (Santa Clara, CA); Tomas Alvarez Rodriguez (Boulder, CO)
Assignee: APPLE INC.
G06T19/00G06T2200/24G06T2210/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,620,178
App. No.
18/142,322
Granted
May 5, 2026
Kind
B1
Abstract

According to various implementations, a method includes obtaining pose data that indicates a plurality of poses of the electronic device within a physical environment. The method includes obtaining a representation of a three-dimensional (3D) environment. The representation of the 3D environment includes a plurality of planes. Each of the plurality of planes defines a plurality of points in xy space. The representation of the 3D environment does not include z space (e.g., depth) information. The method includes anchoring the representation of the 3D environment to a physical anchor point within the physical environment based on the pose data. The method includes anchoring a computer-generated object to the representation of the 3D environment based on the pose data. For example, the pose data includes z space (e.g., depth) information, which is used to anchor the computer-generated object to the representation of the 3D environment.

Claims (47)

1 . A method comprising:

at an electronic device with one or more processors, a non-transitory memory, and a display:

obtaining pose data that indicates a plurality of poses of the electronic device within a physical environment;

obtaining a representation of a three-dimensional (3D) environment, wherein the representation of the 3D environment includes a first plurality of planes, wherein each of the first plurality of planes defines a plurality of points in xy space, and wherein the representation is a modified version of the 3D environment that lacks z space information;

anchoring the representation of the 3D environment to a physical anchor point within the physical environment based on z space information indicated in the pose data; and

anchoring a computer-generated object to the representation of the 3D environment based on the pose data.

2 . The method of claim 1 , wherein none of the first plurality of planes defines a point in z space.

3 . The method of claim 2 , wherein the pose data includes 3D positional information regarding the physical environment, wherein the 3D positional information includes a point in the z space, and wherein anchoring the computer-generated object to the representation of the 3D environment is based on the point in the z space.

4 . The method of claim 1 , wherein obtaining the pose data includes generating a 3D map that characterizes the physical environment.

5 . The method of claim 1 , wherein obtaining the pose data includes generating a 3D point cloud that characterizes the physical environment.

6 . The method of claim 1 ,

wherein anchoring the representation of the 3D environment to the physical anchor point includes:

rendering the representation of the 3D environment based on the pose data in order to generate a first portion of display data, and

displaying, on the display, the first portion of the display data; and

wherein anchoring the computer-generated object to the representation of the 3D environment includes:

rendering the computer-generated object based on the pose data in order to generate a second portion of the display data, and

displaying, on the display, the second portion of the display data while displaying the first portion of the display data.

7 . The method of claim 1 , wherein the representation of the 3D environment corresponds to a 3D representation of a virtual environment.

8 . The method of claim 1 , wherein the representation of the 3D environment corresponds to a 3D representation of a physical environment.

9 . The method of claim 1 , wherein anchoring the representation of the 3D environment to the physical anchor point is substantially concurrent with anchoring the computer-generated object to the representation of the 3D environment.

10 . The method of claim 1 , wherein the electronic device further comprises an image sensor that captures image data of the physical environment, wherein the electronic device further comprises a positional sensor that generates positional sensor data characterizing a position of the electronic device, and wherein obtaining the pose data includes determining the pose data based on the image data and the positional sensor data.

11 . The method of claim 10 , further comprising:

detecting, based on the positional sensor data, a positional change of the electronic device; and

in response to detecting the positional change:

maintaining anchoring the representation of the 3D environment to the physical anchor point by anchoring a second plurality of planes of the representation of the 3D environment based on the pose data, wherein each of the second plurality of planes defines a plurality of points in the xy space; and

maintaining anchoring the computer-generated object to the representation of the 3D environment by anchoring the computer-generated object to the second plurality of planes of the representation of the 3D environment.

12 . The method of claim 10 , wherein determining the pose data includes applying simultaneous localization and mapping (SLAM) to the image data and the positional sensor data.

13 . The method of claim 1 , wherein a corresponding 3D environment defines a plurality of points in xyz space, and wherein the corresponding 3D environment is more graphically complex than the representation of the 3D environment.

14 . The method of 13 , wherein rendering the corresponding 3D environment is associated with a first amount of resource utilization by the electronic device, and wherein rendering the representation of the 3D environment is associated with a second amount of resource utilization by the electronic device that is less than the first amount of resource utilization.

15 . The method of claim 1 , further comprising:

detecting a first user input that specifies the physical anchor point, wherein anchoring the representation of the 3D environment to the physical anchor point is in response to detecting the first user input.

16 . The method of claim 15 , further comprising detecting a second user input that requests anchoring the computer-generated object to the representation of the 3D environment, wherein anchoring the computer-generated object to the representation of the 3D environment is in response to detecting the second user input.

17 . An electronic device comprising:

a display;

a non-transitory memory; and

one or more processors to:

obtain pose data that indicates a plurality of poses of the electronic device within a physical environment;

obtain a representation of a 3D environment, wherein the representation of the 3D environment includes a first plurality of planes, wherein each of the first plurality of planes defines a plurality of points in xy space, and wherein the representation is a modified version of the 3D environment that lacks z space information;

anchor the representation of the 3D environment to a physical anchor point within the physical environment based on z space information indicated in the pose data; and

anchor a computer-generated object to the representation of the 3D environment based on the pose data.

18 . The electronic device of claim 17 , wherein none of the first plurality of planes defines a point in z space, wherein the pose data indicates a point in the z space, and wherein anchoring the computer-generated object to the representation of the 3D environment is based on the point in the z space.

19 . The electronic device of claim 17 , wherein the electronic device further comprises an image sensor that captures image data of the physical environment, wherein the electronic device further comprises a positional sensor that generates positional sensor data characterizing a position of the electronic device, and wherein obtaining the pose data includes determining the pose data based on the image data and the positional sensor data.

20 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device including a display, cause the electronic device to:

obtain pose data that indicates a plurality of poses of the electronic device within a physical environment;

obtain a representation of a 3D environment, wherein the representation of the 3D environment includes a first plurality of planes, wherein each of the first plurality of planes defines a plurality of points in xy space, and wherein the representation is a modified version of the 3D environment that lacks z space information;

anchor the representation of the 3D environment to a physical anchor point within the physical environment based on z space information indicated in the pose data; and

anchor a computer-generated object to the representation of the 3D environment based on the pose data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2023
From: POOT, RUDY; CHUNG, ALVIN; ALVAREZ RODRIGUEZ, THOMAS
To: APPLE INC.
Reel/Frame 063585/0985 →
Continuity (1)
Provisional Application 63337860 · May 3, 2022
References Cited (7)
US 10740960B2 · Stachniak · 2020 [cited by examiner]
US 10832417B1 · Tzur · 2020 [cited by applicant]
US 20210042958A1 · Engel et al. · 2021 [cited by applicant]
US 20210183161A1 · Upendran et al. · 2021 [cited by applicant]
US 20220229534A1 · Terre · 2022 [cited by examiner]
US 20220292715A1 · Liu · 2022 [cited by examiner]
US 20230315383A1 · Canberk · 2023 [cited by examiner]