IP Library › Granted Patent US 12,561,042
Granted Patent B2
US 12,561,042 · App. 18/664,073 · Granted Feb 24, 2026

Devices, methods, and graphical user interfaces for interacting with three-dimensional environments

Inventors: Israel Pastrana Vicente (Spring, TX); Jonathan Ravasz (Sunnyvale, CA); Hugo D. Verweij (Portola Valley, CA); Hana Z. Wang (Novato, CA); Kirsty Keatch (Cupertino, CA)
Assignee: APPLE INC.
G06F3/04815G06F3/012G06F3/0482G06F3/0487G06F3/04817
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,561,042
App. No.
18/664,073
Granted
Feb 24, 2026
Kind
B2
Abstract

A computer system detects an input to invoke a home menu user interface. In response to detecting the input, the computer system displays, via one or more display generation components, the home menu user interface in a three-dimensional environment, including: if a viewpoint of a user in the three-dimensional environment had a first elevation relative to a reference plane in the three-dimensional environment, displaying the home menu user interface at a first height in the three-dimensional environment; and, if the viewpoint of the user in the three-dimensional environment had a second elevation relative to the reference plane in the three-dimensional environment, the second elevation being different from the first elevation, displaying the home menu user interface at a second height in the three-dimensional environment, the second height being different from the first height.

Claims (109)

1 . A method, comprising:

at a computer system that is in communication with one or more display generation components and one or more input devices:

detecting a first input to the computer system to invoke a home menu user interface; and

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment, including:

in accordance with a determination that a viewpoint of a user in the three-dimensional environment had a first elevation relative to a reference plane in the three-dimensional environment, displaying the home menu user interface at a first height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second elevation relative to the reference plane in the three-dimensional environment, wherein the second elevation is different from the first elevation, displaying the home menu user interface at a second height in the three-dimensional environment, wherein the second height is different from the first height.

2 . The method of claim 1 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a press input on the one or more input devices.

3 . The method of claim 1 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a hand gesture.

4 . The method of claim 1 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a third elevation relative to the reference plane in the three-dimensional environment and the third elevation is within a first range of elevations, wherein the third elevation is different from the first elevation, displaying the home menu user interface at the first height in the three-dimensional environment.

5 . The method of claim 4 , wherein the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

6 . The method of claim 1 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fourth elevation relative to the reference plane in the three-dimensional environment, wherein the fourth elevation is different from the second elevation and both the fourth elevation and the second elevation are within a second range of elevations, displaying the home menu user interface at a third height in the three-dimensional environment, wherein the third height is different from the second height.

7 . The method of claim 6 , wherein the second range includes elevations higher than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

8 . The method of claim 1 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fifth elevation relative to the reference plane in the three-dimensional environment that is within a third range of elevations, displaying the home menu user interface at a fourth height in the three-dimensional environment, a plane of the home menu user interface at the fourth height being perpendicular to a horizontal plane in the three-dimensional environment.

9 . The method of claim 8 , wherein the third range of elevations is different from a first range of elevations and different from a second range of elevations, the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment and the second range includes elevations higher than the horizontal plane extending from the viewpoint of the user in the three-dimensional environment.

10 . The method of claim 8 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment, includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a sixth elevation relative to the reference plane in the three-dimensional environment, wherein the sixth elevation is different from the fifth elevation, and the sixth elevation is within a fourth range of elevations, wherein the fourth range of elevations is different from the third range of elevations, displaying the home menu user interface such that a plane of the home menu user interface is tilted toward a viewpoint of the user in the three-dimensional environment.

11 . The method of claim 1 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a first rotation about a vertical axis perpendicular to the reference plane, displaying the home menu user interface at a first rotational position in the three-dimensional environment corresponding to the first rotation; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second rotation about the vertical axis perpendicular to the reference plane, the second rotation being different from the first rotation, displaying the home menu user interface at a second rotational position in the three-dimensional environment corresponding to the second rotation, wherein the second rotational position is different from the first rotational position.

12 . The method of claim 1 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had an eighth elevation relative to a reference plane in the three-dimensional environment and a gaze of the user is directed at a first spatial location, displaying the home menu user interface at a seventh height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had the eighth elevation relative to the reference plane in the three-dimensional environment and the gaze of the user is directed at a second spatial location distinct from the first spatial location, displaying the home menu user interface at the seventh height in the three-dimensional environment.

13 . The method of claim 1 , wherein displaying the home menu user interface at the first height in the three-dimensional environment includes displaying the home menu user interface at a respective height below a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

14 . The method of claim 1 , wherein displaying the home menu user interface includes displaying icons in the home menu user interface progressively in a concentric arrangement.

15 . The method of claim 1 , wherein displaying the home menu user interface includes increasing visual emphasis of icons displayed in the home menu user interface over time.

16 . The method of claim 1 , wherein an elevation of a viewpoint of the user with respect to the reference plane corresponds to an elevation of a user's head with respect to the reference plane.

17 . The method of claim 1 , including:

ceasing to display the home menu user interface;

detecting a second input to the computer system to invoke the home menu user interface after the home menu user interface has ceased to be displayed; and

in response to detecting the second input, in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a ninth elevation relative to the reference plane in the three-dimensional environment, wherein the ninth elevation is different from the first elevation and from the second elevation, displaying the home menu user interface at an eighth height in the three-dimensional environment.

18 . The method of claim 1 including:

detecting a third input corresponding to a request to cease to display the home menu user interface; and

in response to detecting the third input, ceasing to display the home menu user interface.

19 . The method of claim 18 , wherein ceasing to display the home menu user interface in response to detecting the third input includes ceasing to display the home menu user interface by transitioning icons displayed in the home menu user interface through a plurality of intermediate states with increasing amounts of deemphasis of the icons over time.

20 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more display generation components and one or more input devices, the one or more programs including instructions for:

detecting a first input to the computer system to invoke a home menu user interface; and

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment, including:

in accordance with a determination that a viewpoint of a user in the three-dimensional environment had a first elevation relative to a reference plane in the three-dimensional environment, displaying the home menu user interface at a first height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second elevation relative to the reference plane in the three-dimensional environment, wherein the second elevation is different from the first elevation, displaying the home menu user interface at a second height in the three-dimensional environment, wherein the second height is different from the first height.

21 . The non-transitory computer-readable storage medium of claim 20 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a press input on the one or more input devices.

22 . The non-transitory computer-readable storage medium of claim 20 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a hand gesture.

23 . The non-transitory computer-readable storage medium of claim 20 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a third elevation relative to the reference plane in the three-dimensional environment and the third elevation is within a first range of elevations, wherein the third elevation is different from the first elevation, displaying the home menu user interface at the first height in the three-dimensional environment.

24 . The non-transitory computer-readable storage medium of claim 23 , wherein the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

25 . The non-transitory computer-readable storage medium of claim 20 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fourth elevation relative to the reference plane in the three-dimensional environment, wherein the fourth elevation is different from the second elevation and both the fourth elevation and the second elevation are within a second range of elevations, displaying the home menu user interface at a third height in the three-dimensional environment, wherein the third height is different from the second height.

26 . The non-transitory computer-readable storage medium of claim 25 , wherein the second range includes elevations higher than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

27 . The non-transitory computer-readable storage medium of claim 20 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fifth elevation relative to the reference plane in the three-dimensional environment that is within a third range of elevations, displaying the home menu user interface at a fourth height in the three-dimensional environment, a plane of the home menu user interface at the fourth height being perpendicular to a horizontal plane in the three-dimensional environment.

28 . The non-transitory computer-readable storage medium of claim 27 , wherein the third range of elevations is different from a first range of elevations and different from a second range of elevations, the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment and the second range includes elevations higher than the horizontal plane extending from the viewpoint of the user in the three-dimensional environment.

29 . The non-transitory computer-readable storage medium of claim 20 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a first rotation about a vertical axis perpendicular to the reference plane, displaying the home menu user interface at a first rotational position in the three-dimensional environment corresponding to the first rotation; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second rotation about the vertical axis perpendicular to the reference plane, the second rotation being different from the first rotation, displaying the home menu user interface at a second rotational position in the three-dimensional environment corresponding to the second rotation, wherein the second rotational position is different from the first rotational position.

30 . The non-transitory computer-readable storage medium of claim 20 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had an eighth elevation relative to a reference plane in the three-dimensional environment and a gaze of the user is directed at a first spatial location, displaying the home menu user interface at a seventh height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had the eighth elevation relative to the reference plane in the three-dimensional environment and the gaze of the user is directed at a second spatial location distinct from the first spatial location, displaying the home menu user interface at the seventh height in the three-dimensional environment.

31 . The non-transitory computer-readable storage medium of claim 20 , wherein displaying the home menu user interface at the first height in the three-dimensional environment includes displaying the home menu user interface at a respective height below a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

32 . The non-transitory computer-readable storage medium of claim 20 , wherein the one or more programs include instructions for:

ceasing to display the home menu user interface;

detecting a second input to the computer system to invoke the home menu user interface after the home menu user interface has ceased to be displayed; and

in response to detecting the second input, in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a ninth elevation relative to the reference plane in the three-dimensional environment, wherein the ninth elevation is different from the first elevation and from the second elevation, displaying the home menu user interface at an eighth height in the three-dimensional environment.

33 . A computer system that is in communication with one or more display generation components and one or more input devices, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

detecting a first input to the computer system to invoke a home menu user interface; and

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment, including:

in accordance with a determination that a viewpoint of a user in the three-dimensional environment had a first elevation relative to a reference plane in the three-dimensional environment, displaying the home menu user interface at a first height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second elevation relative to the reference plane in the three-dimensional environment, wherein the second elevation is different from the first elevation, displaying the home menu user interface at a second height in the three-dimensional environment, wherein the second height is different from the first height.

34 . The computer system of claim 33 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a press input on the one or more input devices.

35 . The computer system of claim 33 , wherein detecting the first input to the computer system to invoke the home menu user interface includes detecting a hand gesture.

36 . The computer system of claim 33 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a third elevation relative to the reference plane in the three-dimensional environment and the third elevation is within a first range of elevations, wherein the third elevation is different from the first elevation, displaying the home menu user interface at the first height in the three-dimensional environment.

37 . The computer system of claim 36 , wherein the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

38 . The computer system of claim 33 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fourth elevation relative to the reference plane in the three-dimensional environment, wherein the fourth elevation is different from the second elevation and both the fourth elevation and the second elevation are within a second range of elevations, displaying the home menu user interface at a third height in the three-dimensional environment, wherein the third height is different from the second height.

39 . The computer system of claim 38 , wherein the second range includes elevations higher than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

40 . The computer system of claim 33 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in the three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a fifth elevation relative to the reference plane in the three-dimensional environment that is within a third range of elevations, displaying the home menu user interface at a fourth height in the three-dimensional environment, a plane of the home menu user interface at the fourth height being perpendicular to a horizontal plane in the three-dimensional environment.

41 . The computer system of claim 40 , wherein the third range of elevations is different from a first range of elevations and different from a second range of elevations, the first range of elevations includes elevations that are parallel to or lower than a horizontal plane extending from a viewpoint of a user in the three-dimensional environment and the second range includes elevations higher than the horizontal plane extending from the viewpoint of the user in the three-dimensional environment.

42 . The computer system of claim 33 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a first rotation about a vertical axis perpendicular to the reference plane, displaying the home menu user interface at a first rotational position in the three-dimensional environment corresponding to the first rotation; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a second rotation about the vertical axis perpendicular to the reference plane, the second rotation being different from the first rotation, displaying the home menu user interface at a second rotational position in the three-dimensional environment corresponding to the second rotation, wherein the second rotational position is different from the first rotational position.

43 . The computer system of claim 33 , wherein:

in response to detecting the first input, displaying via the one or more display generation components the home menu user interface in a three-dimensional environment includes:

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had an eighth elevation relative to a reference plane in the three-dimensional environment and a gaze of the user is directed at a first spatial location, displaying the home menu user interface at a seventh height in the three-dimensional environment; and

in accordance with a determination that the viewpoint of the user in the three-dimensional environment had the eighth elevation relative to the reference plane in the three-dimensional environment and the gaze of the user is directed at a second spatial location distinct from the first spatial location, displaying the home menu user interface at the seventh height in the three-dimensional environment.

44 . The computer system of claim 33 , wherein displaying the home menu user interface at the first height in the three-dimensional environment includes displaying the home menu user interface at a respective height below a horizontal plane extending from a viewpoint of a user in the three-dimensional environment.

45 . The computer system of claim 33 , wherein the one or more programs include instructions for:

ceasing to display the home menu user interface;

detecting a second input to the computer system to invoke the home menu user interface after the home menu user interface has ceased to be displayed; and

in response to detecting the second input, in accordance with a determination that the viewpoint of the user in the three-dimensional environment had a ninth elevation relative to the reference plane in the three-dimensional environment, wherein the ninth elevation is different from the first elevation and from the second elevation, displaying the home menu user interface at an eighth height in the three-dimensional environment.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 3, 2025
From: VERWEIJ, HUGO D.; WANG, HANA Z.; KEATCH, KIRSTY
To: APPLE INC.
Reel/Frame 070386/0341 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2024
From: PASTRANA VICENTE, ISRAEL; DEDONATO, AMY E.; ALONSO RUIZ, MARCOS; BROUGHTON, LEE S.; LYONS, RICHARD D.; SORRENTINO, WILLIAM A., III; LEMAY, STEPHEN O.; OWEN, JAMES J.; ESTANY RODRIGUEZ, MIQUEL; CHAND, JESSE; DASCOLA, JONATHAN R.; SCHNORR, CHRISTIAN; TAYLOR, ZOEY C.; RAVASZ, JONATHAN; HASKIN, HARLAN B.; CHAWDA, VINAY; BOESEL, BENJAMIN H.; KAWASHIMA, IEYUKI; MCKENZIE, CHRISTOPHER D.; HYLAK, BENJAMIN; GITTER, NATHAN; KIM, NAHCKJOON; MONSMA, OWEN; STAUBER, MATAN; PRICE, DANIELLE M.
To: APPLE INC.
Reel/Frame 068938/0786 →
Continuity (5)
Provisional Application 63646801 · May 13, 2024
Provisional Application 63528336 · Jul 21, 2023
Provisional Application 63470912 · Jun 4, 2023
Provisional Application 63466963 · May 16, 2023
Related Publication 20240411421A1 · Dec 12, 2024
References Cited (12)
US 20140215383A1 · Park-Ekecs et al. · 2014 [cited by applicant]
US 20160026242A1 · Burns · 2016 [cited by examiner]
US 20160231883A1 · Zambetti et al. · 2016 [cited by applicant]
US 20210286509A1 · Tyler et al. · 2021 [cited by applicant]
US 20230315247A1 · Pastrana et al. · 2023 [cited by applicant]
US 20240036646A1 · Yu · 2024 [cited by examiner]
AU 2020239727A1 · 2021 [cited by applicant]
Balkan Aral et al., “Parallax Scrolling” in “Flash 3D Cheats Most Wanted”, https://dx.doi.org/10.1007/978-1-4302-0814-3_5>, Jan. 1, 2003, 4 pages. [cited by applicant]
David Antonio Gomez Jauregui et al., “Design and Evaluation of 3D Cursors and Motion Parallax for the Exploration of Desktop Virtual Environments”, 3D User Interfaces (3DUI), 2012 IEEE Symposium on IEEE, Mar. 4, 2012, 4… [cited by applicant]
Keytreeuk: “Smart Vision—Interactive Augmented Reality, Hand Tracking using Kinects SAP HANA, NetWeaver Cloud”, https://www.youtube.com/watch?v=87ggsGUK5iA, Nov. 25, 2012, 2 pages. [cited by applicant]
International Search Report and Written Opinion, dated Nov. 25, 2024, received in International Patent Application No. PCT/US2024/029514, which corresponds with U.S. Appl. No. 18/664,073, 31 pages. [cited by applicant]
Invitation to Pay, dated Oct. 10, 2025, received in International Patent Application No. PCT/US2025/032759, which corresponds with U.S. Appl. No. 19/229,726, 17 pages. [cited by applicant]