IP Library › Granted Patent US 12,380,653

Granted Patent B2

US 12,380,653 · App. 17/655,521 · Granted Aug 5, 2025

Devices, methods, and graphical user interfaces for maps

Inventors: Fiona P. O'Leary (Mountain View, CA); Matthew J. Sundstrom (Campbell, CA); Stephen O. Lemay (Palo Alto, CA); Julian K. Missig (Burlingame, CA)

Assignee: Apple Inc.

G06T19/003 G01C21/3664 G06F3/013 G06F3/04815 G06T19/006 G06F3/04883 G06T2200/24

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12,380,653
App. No.: 17/655,521
Filed: Mar 18, 2022
Granted: Aug 5, 2025
Kind: B2
Examiner: HARRISON, CHANTE E
Art Unit: 2615
USPC: 345/419

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

In some embodiments, an electronic device concurrently presents a navigation user interface element with a respective physical location designated and a content element including content corresponding to the respective physical location. In some embodiments, an electronic device presents navigation from a first physical location to a second physical location with reduced visual prominence in a content element in response to an input corresponding to a request to present content corresponding to the second physical location.

Claims (160)

1. A method, comprising:

at an electronic device in communication with a display generation component and one or more input devices:

displaying, via the display generation component, a three-dimensional environment including a user interface that includes:

first respective content corresponding to a first view of a first physical location, wherein the first respective content is displayed at a first respective location in the three-dimensional environment, and wherein the first respective content corresponding to the first view of the first physical location is concurrently displayed with a representation of a physical environment of the electronic device and occupies a first portion of a display area of the display generation component; and

a navigation user interface element with a first location designated within the navigation user interface element, the first location within the navigation user interface element corresponding to the first physical location, wherein the navigation user interface element is displayed in the three-dimensional environment between the first respective location at which the first respective content is displayed and a viewpoint of a user in the three-dimensional environment, and

while displaying, via the display generation component, the user interface and the navigation user interface element, detecting, via the one or more input devices, a user input directed towards the navigation user interface element, the user input corresponding to a request to designate a second location corresponding to a second physical location;

in response to detecting the user input, updating the user interface to include second respective content corresponding to a second view of the second physical location, wherein the second respective content is displayed at the first respective location in the three-dimensional environment; and

while displaying, via the display generation component, the first respective content corresponding to the first view of the first physical location occupying the first portion of the display area concurrently with the representation of the physical environment, detecting, via the one or more input devices, a respective input corresponding to a request to obscure display of the representation of the physical environment; and

in response to detecting the respective input, obscuring display of the representation of the physical environment, including updating the first respective content to:

correspond to a third view, greater than the first view of the first physical location; and

occupy a second portion, greater than the first portion, of the display area of the display generation component.

2. The method of claim 1 , wherein:

the three-dimensional environment includes a representation of a surface in a physical environment of the electronic device, and

the navigation user interface element is displayed at a location in the three-dimensional environment corresponding to the representation of the surface.

3. The method of claim 1 , wherein the navigation user interface element is displayed at a location in the three-dimensional environment that does not correspond to a surface in a physical environment of the electronic device.

4. The method of claim 1 , wherein the first respective content is surrounded by a representation of a physical environment of the electronic device, and a boundary between the first respective content and the representation of the physical environment of the electronic device includes a gradual visual transition between the first respective content and the representation of the physical environment of the electronic device.

5. The method of claim 1 , further comprising:

while displaying the first respective content corresponding to the first view of the first physical location, detecting, via the one or more input devices, movement of the viewpoint of the user of the electronic device in the three-dimensional environment; and

in response to detecting the movement of the viewpoint of the user of the electronic device in the three-dimensional environment, in accordance with a determination that a gaze of the user was directed to the first respective content when the movement of the viewpoint of the user was detected, updating display of the first view of the first physical location corresponding to the first respective content with a simulated parallax effect in accordance with the movement of the viewpoint of the user.

6. The method of claim 1 , further comprising:

while displaying the first respective content corresponding to the first view of the first physical location, detecting, via the one or more input devices, movement of the electronic device; and

in response to detecting the movement of the electronic device, in accordance with a determination that the movement of the electronic device satisfies one or more criteria, displaying, via the display generation component, additional content corresponding to the first physical location.

7. The method of claim 1 , further comprising:

while displaying the first respective content corresponding to the first view of the first physical location, detecting, via the one or more input devices, an input directed to a selectable user interface element displayed in the user interface; and

in response to detecting the input directed to the selectable user interface element, displaying third respective content corresponding to a respective view of a respective physical location, wherein the third respective content is displayed at the first respective location in the three-dimensional environment.

8. The method of claim 1 , further comprising:

while displaying the first respective content corresponding to the first view of the first physical location, detecting movement of the electronic device relative to the three-dimensional environment; and

in accordance with a determination that the movement of the electronic device relative to the three-dimensional environment satisfies one or more criteria, displaying, via the display generation component, additional content corresponding to the first physical location; and

in accordance with a determination that the movement of the electronic device relative to the three-dimensional environment does not satisfy the one or more criteria, forgoing displaying of the additional content corresponding to the first physical location.

9. The method of claim 8 , wherein the one or more criteria include a criterion that is satisfied based on a change in an orientation of the electronic device and/or a criterion that is satisfied based on whether a gaze of the user is directed to one or more respective objects in the three-dimensional environment.

10. The method of claim 1 , wherein the first respective content corresponds to a first field of view having a first orientation, and the method further comprises:

while displaying the first respective content and displaying, in the navigation user interface element, an indication of the first field of view, detecting, via the one or more input devices, an input corresponding to a request to display third respective content that corresponds to a second field of view having a second orientation; and

in response detecting the input corresponding to the request to display the third respective content that corresponds to the second field of view having the second orientation:

displaying, via the display generation component, the third respective content; and

updating, in the navigation user interface element, the indication of the first field of view to be an indication of the second field of view.

11. The method of claim 1 , further comprising:

in accordance with a determination that a physical environment of the electronic device satisfies one or more criteria, including a criterion that is satisfied based on an amount of unobstructed space around the electronic device in the physical environment of the electronic device, displaying the navigation user interface element at a first size in the three-dimensional environment; and

in accordance with a determination that the physical environment of the electronic device does not satisfy the one or more criteria, displaying the navigation user interface element at a second size, different from the first size, in the three-dimensional environment.

12. The method of claim 1 , wherein the navigation user interface element includes a first respective portion displayed in three dimensions in the three-dimensional environment, and a second respective portion displayed in two dimensions in the three-dimensional environment, wherein the first respective portion is closer to the viewpoint of the user than the second respective portion.

13. The method of claim 1 , wherein:

while displaying the first respective content:

the navigation user interface element corresponds to a first physical region including the first physical location, and

the navigation user interface element includes an indication of a topography of the first physical region.

14. The method of claim 1 , wherein the navigation user interface element includes respective content displayed in three dimensions, and the method further comprises:

while displaying the navigation user interface element that includes the respective content displayed in three dimensions, detecting, via the one or more input devices, movement of a predefined portion of the user; and

in response to detecting the movement of the predefined portion of the user:

in accordance with a determination that the predefined portion of the user is at a location corresponding to the respective content displayed in three dimensions in the navigation user interface element, updating the navigation user interface element to visually de-emphasize the respective content displayed in three dimensions.

15. The method of claim 1 , further comprising:

displaying, within the navigation user interface element, an indication of a second electronic device displayed at a respective location within the navigation user interface element corresponding to a respective physical location of the second electronic device;

while displaying the indication of the second electronic device in the navigation user interface element, detecting, via the one or more input devices, selection of the indication of the second electronic device; and

in response to detecting the selection of the indication of the second electronic device, displaying, via the display generation component, content captured by the second electronic device of the respective physical location of the second electronic device.

16. The method of claim 1 , wherein the first respective content includes a two-dimensional representation of an object located at the first physical location, the method further comprising:

concurrently displaying the first respective content including the two-dimensional representation of the object, and a three-dimensional representation of the object.

17. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via a display generation component, a three-dimensional environment including a user interface that includes:

while displaying, via the display generation component, the user interface and the navigation user interface element, detecting, via one or more input devices, a user input directed towards the navigation user interface element, the user input corresponding to a request to designate a second location corresponding to a second physical location;

in response to detecting the respective input, obscuring display of the representation of the physical environment, including updating the first respective content to:

correspond to a third view, greater than the first view of the first physical location; and

occupy a second portion, greater than the first portion, of the display area of the display generation component.

18. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:

displaying, via a display generation component, a three-dimensional environment including a user interface that includes:

in response to detecting the respective input, obscuring display of the representation of the physical environment, including updating the first respective content to:

correspond to a third view, greater than the first view of the first physical location; and

occupy a second portion, greater than the first portion, of the display area of the display generation component.

19. The electronic device of claim 17 , wherein:

the three-dimensional environment includes a representation of a surface in a physical environment of the electronic device, and

the navigation user interface element is displayed at a location in the three-dimensional environment corresponding to the representation of the surface.

20. The electronic device of claim 17 , wherein the navigation user interface element is displayed at a location in the three-dimensional environment that does not correspond to a surface in a physical environment of the electronic device.

21. The electronic device of claim 17 , wherein the first respective content is surrounded by a representation of a physical environment of the electronic device, and a boundary between the first respective content and the representation of the physical environment of the electronic device includes a gradual visual transition between the first respective content and the representation of the physical environment of the electronic device.

22. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

23. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

while displaying the first respective content corresponding to the first view of the first physical location, detecting, via the one or more input devices, movement of the electronic device; and

24. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

25. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

26. The electronic device of claim 25 , wherein the one or more criteria include a criterion that is satisfied based on a change in an orientation of the electronic device and/or a criterion that is satisfied based on whether a gaze of the user is directed to one or more respective objects in the three-dimensional environment.

27. The electronic device of claim 17 , wherein the first respective content corresponds to a first field of view having a first orientation, and the one or more programs further include instructions for:

in response detecting the input corresponding to the request to display the third respective content that corresponds to the second field of view having the second orientation:

displaying, via the display generation component, the third respective content; and

updating, in the navigation user interface element, the indication of the first field of view to be an indication of the second field of view.

28. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

29. The electronic device of claim 17 , wherein the navigation user interface element includes a first respective portion displayed in three dimensions in the three-dimensional environment, and a second respective portion displayed in two dimensions in the three-dimensional environment, wherein the first respective portion is closer to the viewpoint of the user than the second respective portion.

30. The electronic device of claim 17 , wherein:

while displaying the first respective content:

the navigation user interface element corresponds to a first physical region including the first physical location, and

the navigation user interface element includes an indication of a topography of the first physical region.

31. The electronic device of claim 17 , wherein the navigation user interface element includes respective content displayed in three dimensions, and the one or more programs further include instructions for:

in response to detecting the movement of the predefined portion of the user:

32. The electronic device of claim 17 , wherein the one or more programs further include instructions for:

33. The electronic device of claim 17 , wherein the first respective content includes a two-dimensional representation of an object located at the first physical location, and the one or more programs further include instructions for:

concurrently displaying the first respective content including the two-dimensional representation of the object, and a three-dimensional representation of the object.

34. The non-transitory computer readable storage medium of claim 18 , wherein:

the three-dimensional environment includes a representation of a surface in a physical environment of the electronic device, and

the navigation user interface element is displayed at a location in the three-dimensional environment corresponding to the representation of the surface.

35. The non-transitory computer readable storage medium of claim 18 , wherein the navigation user interface element is displayed at a location in the three-dimensional environment that does not correspond to a surface in a physical environment of the electronic device.

36. The non-transitory computer readable storage medium of claim 18 , wherein the first respective content is surrounded by a representation of a physical environment of the electronic device, and a boundary between the first respective content and the representation of the physical environment of the electronic device includes a gradual visual transition between the first respective content and the representation of the physical environment of the electronic device.

37. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

38. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

while displaying the first respective content corresponding to the first view of the first physical location, detecting, via the one or more input devices, movement of the electronic device; and

39. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

40. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

41. The non-transitory computer readable storage medium of claim 40 , wherein the one or more criteria include a criterion that is satisfied based on a change in an orientation of the electronic device and/or a criterion that is satisfied based on whether a gaze of the user is directed to one or more respective objects in the three-dimensional environment.

42. The non-transitory computer readable storage medium of claim 18 , wherein the first respective content corresponds to a first field of view having a first orientation, and the method further comprises:

in response detecting the input corresponding to the request to display the third respective content that corresponds to the second field of view having the second orientation:

displaying, via the display generation component, the third respective content; and

updating, in the navigation user interface element, the indication of the first field of view to be an indication of the second field of view.

43. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

44. The non-transitory computer readable storage medium of claim 18 , wherein the navigation user interface element includes a first respective portion displayed in three dimensions in the three-dimensional environment, and a second respective portion displayed in two dimensions in the three-dimensional environment, wherein the first respective portion is closer to the viewpoint of the user than the second respective portion.

45. The non-transitory computer readable storage medium of claim 18 , wherein:

while displaying the first respective content:

the navigation user interface element corresponds to a first physical region including the first physical location, and

the navigation user interface element includes an indication of a topography of the first physical region.

46. The non-transitory computer readable storage medium of claim 18 , wherein the navigation user interface element includes respective content displayed in three dimensions, and the method further comprises:

in response to detecting the movement of the predefined portion of the user:

47. The non-transitory computer readable storage medium of claim 18 , wherein the method further comprises:

48. The non-transitory computer readable storage medium of claim 18 , wherein the first respective content includes a two-dimensional representation of an object located at the first physical location, and the method further comprises:

concurrently displaying the first respective content including the two-dimensional representation of the object, and a three-dimensional representation of the object.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 21, 2022

From: O'LEARY, FIONA P.; SUNDSTROM, MATTHEW J.; LEMAY, STEPHEN O.; MISSIG, JULIAN K.

To: APPLE INC.

Reel/Frame 059330/0467 →

Continuity (2)

Provisional Application 63164296 · Mar 22, 2021

Related Publication 20220301264A1 · Sep 22, 2022

References Cited (65)

US 6323846B1 · Westerman et al. · 2001 [cited by applicant]

US 6570557B1 · Westerman et al. · 2003 [cited by applicant]

US 6677932B1 · Westerman · 2004 [cited by applicant]

US 7614008B2 · Ording · 2009 [cited by applicant]

US 7633076B2 · Huppi et al. · 2009 [cited by applicant]

US 7653883B2 · Hotelling et al. · 2010 [cited by applicant]

US 7657849B2 · Chaudhri et al. · 2010 [cited by applicant]

US 7663607B2 · Hotelling et al. · 2010 [cited by applicant]

US 7844914B2 · Andre et al. · 2010 [cited by applicant]

US 7957762B2 · Herz et al. · 2011 [cited by applicant]

US 8006002B2 · Kalayjian et al. · 2011 [cited by applicant]

US 8239784B2 · Hotelling et al. · 2012 [cited by applicant]

US 8279180B2 · Hotelling et al. · 2012 [cited by applicant]

US 8381135B2 · Hotelling et al. · 2013 [cited by applicant]

US 8436872B2 · Wright · 2013 [cited by examiner]

US 8479122B2 · Hotelling et al. · 2013 [cited by applicant]

US 9164975B2 · Milewski · 2015 [cited by examiner]

US 9196072B2 · Oh · 2015 [cited by examiner]

US 9348458B2 · Hotelling et al. · 2016 [cited by applicant]

US 9437047B2 · Chang · 2016 [cited by examiner]

US 9933937B2 · Lemay et al. · 2018 [cited by applicant]

US 10768421B1 · Rosenberg et al. · 2020 [cited by applicant]

US 11861136B1 · Faulkner · 2024 [cited by examiner]

US 11922588B2 · Fillhardt · 2024 [cited by examiner]

US 20020015024A1 · Westerman et al. · 2002 [cited by applicant]

US 20050190059A1 · Wehrenberg · 2005 [cited by applicant]

US 20060017692A1 · Wehrenberg et al. · 2006 [cited by applicant]

US 20060033724A1 · Chaudhri et al. · 2006 [cited by applicant]

US 20060197753A1 · Hotelling · 2006 [cited by applicant]

US 20080186255A1 · Cohen · 2008 [cited by examiner]

US 20090049408A1 · Naaman et al. · 2009 [cited by applicant]

US 20090319181A1 · Khosravy et al. · 2009 [cited by applicant]

US 20100115459A1 · Kinnunen et al. · 2010 [cited by applicant]

US 20110164163A1 · Bilbrey · 2011 [cited by examiner]

US 20120194547A1 · Johnson et al. · 2012 [cited by applicant]

US 20130263016A1 · Lehtiniemi · 2013 [cited by examiner]

US 20140063058A1 · Fialho · 2014 [cited by examiner]

US 20150262428A1 · Tatzgern et al. · 2015 [cited by applicant]

US 20160127690A1 · Kaehler · 2016 [cited by examiner]

US 20160133044A1 · Lynch · 2016 [cited by applicant]

US 20170147180A1 · Yoon · 2017 [cited by examiner]

US 20180136815A1 · Tomizuka et al. · 2018 [cited by applicant]

US 20180218214A1 · Pestun · 2018 [cited by examiner]

US 20190325654A1 · Stisser · 2019 [cited by examiner]

US 20210173340A1 · Kim · 2021 [cited by examiner]

US 20210248674A1 · Ogunbunmi · 2021 [cited by examiner]

US 20210272367A1 · Richter · 2021 [cited by examiner]

US 20220292783A1 · Jayaram · 2022 [cited by examiner]

US 20220292784A1 · Jayaram · 2022 [cited by examiner]

US 20220292785A1 · Jayaram · 2022 [cited by examiner]

US 20220295032A1 · Jayaram · 2022 [cited by examiner]

US 20220295040A1 · Jayaram · 2022 [cited by examiner]

US 20220295139A1 · Jayaram · 2022 [cited by examiner]

US 20230260240A1 · Jayaram · 2023 [cited by examiner]

US 20230377295A1 · Angevine · 2023 [cited by examiner]

US 20240403080A1 · Laurita et al. · 2024 [cited by applicant]

US 20240404207A1 · Laurita et al. · 2024 [cited by applicant]

CN 104981681A · 2015 [cited by examiner]

CN 117043722A · 2023 [cited by examiner]

EP 3688726B1 · 2023 [cited by examiner]

WO 2013169849A2 · 2013 [cited by applicant]

WO 2014105276A1 · 2014 [cited by applicant]

International Search Report received for PCT Patent Application No. PCT/US2022/071228, mailed on Aug. 25, 2022, 6 pages. [cited by applicant]

International Search Report received for PCT Patent Application No. PCT/US2024/032140, mailed on Sep. 10, 2024, 4 pages. [cited by applicant]

International Search Report received for PCT Patent Application No. PCT/US2024/032423, mailed on Sep. 25, 2024, 5 pages. [cited by applicant]