IP Library › Granted Patent US 11,017,608

Granted Patent B2

US 11,017,608 · App. 16/929,027 · Granted May 25, 2021

Devices, methods, and graphical user interfaces for displaying objects in 3D context

Inventors: Grant R. Paul (San Francisco, CA); Nicolas V. Scapel (Sunnyvale, CA); David Lui (San Jose, CA); Xiao Jin Yu (Sunnyvale, CA); Giancarlo Yerkes (Menlo Park, CA); Kyle E. Fisher (Sunnyvale, CA)

Assignee: APPLE INC.

G06T19/006 G06F3/013 G06F3/0488 G06F3/04815 G06F3/04842 G06F3/04845 G06F3/165

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,017,608
App. No.: 16/929,027
Granted: May 25, 2021
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

An electronic device, while displaying a first user interface region and a first media item, detects a first input corresponding to a sharing user interface. In response, the device displays the sharing user interface, which includes a second user interface object for initiating a process for displaying a second representation of a first media item in an augmented reality environment. In response to detecting a sequence of one or more inputs including selection of the second user interface object, the device displays the second representation of the first media item in a second user interface that includes content of at least a portion of a field of view of the device's one or more cameras.

Claims (129)

1. A method, comprising:

at an electronic device including a display generation component, one or more input devices, and one or more cameras:

displaying, by the display generation component:

a first user interface region;

a first representation of a first media item in the first user interface region; and

a first user interface object for displaying a sharing user interface;

while displaying the first user interface region, detecting a first input corresponding to the first user interface object;

in response to detecting the first input corresponding to the first user interface object, displaying the sharing user interface that includes a second user interface object for initiating a process for displaying a second representation of the first media item in an augmented reality environment;

while the sharing user interface is displayed, detecting a sequence of one or more inputs including selection of the second user interface object; and

in response to detecting the sequence of one or more inputs including selection of the second user interface object, displaying the second representation of the first media item in a second user interface that includes content of at least a portion of a field of view of the one or more cameras.

2. The method of claim 1 , wherein the displayed second representation of the first media item overlays a respective plane in the field of view of the one or more cameras.

3. The method of claim 2 , including:

while displaying the second representation of the first media item in the second user interface, detecting an input for rotating the second representation of the first media item; and

in response to the input for rotating the second representation of the first media item, rotating the second representation of the first media item about an axis that is normal to the respective plane in the field of view of the one or more cameras over which the first media item is displayed.

4. The method of claim 1 , including:

detecting first movement of the electronic device that adjusts the field of view of the one or more cameras; and

in response to detecting the first movement of the electronic device, adjusting the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and a respective plane in the field of view of the one or more cameras.

5. The method of claim 1 , including:

detecting selection of a first representation of a second media item; and

in response to detecting the sequence of one or more inputs including selection of the second user interface object, displaying a second representation of the second media item in the second user interface that includes the content of at least a portion of the field of view of the one or more cameras.

6. The method of claim 5 , wherein the selection of the second media item is detected while the sharing user interface is displayed.

7. The method of claim 5 , including:

while displaying the first user interface region, displaying the first representation of the second media item; and

wherein the selection of the second media item is detected while the first user interface region is displayed.

8. The method of claim 5 , wherein:

selection of the first media item is detected at a first time;

selection of the second media item is detected at a second time; and

wherein displaying the second representation of the first media item in the second user interface and displaying the second representation of the second media item in the second user interface includes:

in accordance with a determination that the first time is prior to the second time, displaying the second representation of the first media item and the second representation of the second media item with a first order; and

in accordance with a determination that the second time is prior to the first time, displaying the second representation of the first media item and the second representation of the second media item with a second order that is distinct from the first order.

9. The method of claim 5 , including:

while displaying the second representation of the first media item in the second user interface and displaying the second representation of the second media item in the second user interface, detecting an input for manipulating a second representation of a respective media item; and

in response to detecting the input for manipulating the second representation of the respective media item:

in accordance with a determination that the input for manipulating the second representation of the respective media item is directed to the second representation of the first media item, manipulating the second representation of the first media item based on the first input; and

in accordance with a determination that the input for manipulating the second representation of the respective media item is directed to the second representation of the second media item, manipulating the second representation of the second media item based on the first input.

10. The method of claim 1 , wherein the sharing user interface includes a third user interface object for transmitting the first media item to a remote device.

11. The method of claim 1 , wherein the second representation of the first media item is displayed with an orientation that is perpendicular to a plane in the field of view of the one or more cameras.

12. The method of claim 11 , including:

while a first face of the second representation of the first media item is displayed, detecting a second movement of the electronic device that adjusts the field of view of the one or more cameras,

in response to detecting the second movement of the electronic device, adjusting the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and the plane in the field of view of the one or more cameras, wherein:

after the second movement, a second face, opposite the first face, of the second representation of the first media item is displayed; and

the second face of the second representation of the first media item is a reversed version of the first face of the second representation of the first media item.

13. A computer system, comprising:

a display generation component;

one or more input devices;

one or more cameras;

one or more processors; and

memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, by the display generation component:

a first user interface region;

a first representation of a first media item in the first user interface region; and

a first user interface object for displaying a sharing user interface;

while displaying the first user interface region, detecting a first input corresponding to the first user interface object;

while the sharing user interface is displayed, detecting a sequence of one or more inputs including selection of the second user interface object; and

14. The computer system of claim 13 , wherein the displayed second representation of the first media item overlays a respective plane in the field of view of the one or more cameras.

15. The computer system of claim 14 , wherein the one or more programs include instructions for:

while displaying the second representation of the first media item in the second user interface, detecting an input for rotating the second representation of the first media item; and

16. The computer system of claim 13 , wherein the one or more programs include instructions for:

detecting first movement of the computer system that adjusts the field of view of the one or more cameras; and

in response to detecting the first movement of the computer system, adjusting the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and a respective plane in the field of view of the one or more cameras.

17. The computer system of claim 13 , wherein the one or more programs include instructions for:

detecting selection of a first representation of a second media item; and

18. The computer system of claim 17 , wherein the selection of the second media item is detected while the sharing user interface is displayed.

19. The computer system of claim 17 , wherein the one or more programs include instructions for:

while displaying the first user interface region, displaying the first representation of the second media item; and

wherein the selection of the second media item is detected while the first user interface region is displayed.

20. The computer system of claim 17 , wherein:

selection of the first media item is detected at a first time;

selection of the second media item is detected at a second time; and

21. The computer system of claim 17 , wherein the one or more programs include instructions for:

in response to detecting the input for manipulating the second representation of the respective media item:

22. The computer system of claim 13 , wherein the sharing user interface includes a third user interface object for transmitting the first media item to a remote device.

23. The computer system of claim 13 , wherein the second representation of the first media item is displayed with an orientation that is perpendicular to a plane in the field of view of the one or more cameras.

24. The computer system of claim 23 , wherein the one or more programs include instructions for:

while a first face of the second representation of the first media item is displayed, detecting a second movement of the computer system that adjusts the field of view of the one or more cameras,

in response to detecting the second movement of the computer system, adjusting the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and the plane in the field of view of the one or more cameras, wherein:

after the second movement, a second face, opposite the first face, of the second representation of the first media item is displayed; and

the second face of the second representation of the first media item is a reversed version of the first face of the second representation of the first media item.

25. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a computer system with a display generation component, one or more input devices, and one or more cameras, cause the computer system to:

display, by the display generation component:

a first user interface region;

a first representation of a first media item in the first user interface region; and

a first user interface object for displaying a sharing user interface;

while displaying the first user interface region, detect a first input corresponding to the first user interface object;

in response to detecting the first input corresponding to the first user interface object, display the sharing user interface that includes a second user interface object for initiating a process for displaying a second representation of the first media item in an augmented reality environment;

while the sharing user interface is displayed, detect a sequence of one or more inputs including selection of the second user interface object; and

in response to detecting the sequence of one or more inputs including selection of the second user interface object, display the second representation of the first media item in a second user interface that includes content of at least a portion of a field of view of the one or more cameras.

26. The non-transitory computer readable storage medium of claim 25 , wherein the displayed second representation of the first media item overlays a respective plane in the field of view of the one or more cameras.

27. The non-transitory computer readable storage medium of claim 26 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

while displaying the second representation of the first media item in the second user interface, detect an input for rotating the second representation of the first media item; and

in response to the input for rotating the second representation of the first media item, rotate the second representation of the first media item about an axis that is normal to the respective plane in the field of view of the one or more cameras over which the first media item is displayed.

28. The non-transitory computer readable storage medium of claim 25 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

detect first movement of the computer system that adjusts the field of view of the one or more cameras; and

in response to detecting the first movement of the computer system, adjust the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and a respective plane in the field of view of the one or more cameras.

29. The non-transitory computer readable storage medium of claim 25 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

detect selection of a first representation of a second media item; and

in response to detecting the sequence of one or more inputs including selection of the second user interface object, display a second representation of the second media item in the second user interface that includes the content of at least a portion of the field of view of the one or more cameras.

30. The non-transitory computer readable storage medium of claim 29 , wherein the selection of the second media item is detected while the sharing user interface is displayed.

31. The non-transitory computer readable storage medium of claim 29 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

while displaying the first user interface region, display the first representation of the second media item; and

wherein the selection of the second media item is detected while the first user interface region is displayed.

32. The non-transitory computer readable storage medium of claim 29 , wherein:

selection of the first media item is detected at a first time;

selection of the second media item is detected at a second time; and

33. The non-transitory computer readable storage medium of claim 29 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

while displaying the second representation of the first media item in the second user interface and displaying the second representation of the second media item in the second user interface, detect an input for manipulating a second representation of a respective media item; and

in response to detecting the input for manipulating the second representation of the respective media item:

in accordance with a determination that the input for manipulating the second representation of the respective media item is directed to the second representation of the first media item, manipulate the second representation of the first media item based on the first input; and

in accordance with a determination that the input for manipulating the second representation of the respective media item is directed to the second representation of the second media item, manipulate the second representation of the second media item based on the first input.

34. The non-transitory computer readable storage medium of claim 25 , wherein the sharing user interface includes a third user interface object for transmitting the first media item to a remote device.

35. The non-transitory computer readable storage medium of claim 25 , wherein the second representation of the first media item is displayed with an orientation that is perpendicular to a plane in the field of view of the one or more cameras.

36. The non-transitory computer readable storage medium of claim 35 , wherein the one or more programs include instructions, which, when executed by the computer system, cause the computer system to:

while a first face of the second representation of the first media item is displayed, detect a second movement of the computer system that adjusts the field of view of the one or more cameras,

in response to detecting the second movement of the computer system, adjust the second representation of the first media item in accordance with a fixed spatial relationship between the second representation of the first media item and the plane in the field of view of the one or more cameras, wherein:

after the second movement, a second face, opposite the first face, of the second representation of the first media item is displayed; and

the second face of the second representation of the first media item is a reversed version of the first face of the second representation of the first media item.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 2, 2021

From: YERKES, GIANCARLO; FISHER, KYLE E.

To: APPLE INC.

Reel/Frame 055119/0640 →

Continuity (4)

Continuation 16581685 · Sep 24, 2019

Provisional Application 62855973 · Jun 1, 2019

Provisional Application 62844010 · May 6, 2019

Related Publication 20200357184A1 · Nov 12, 2020

Cited By (1)

US 12,399,601