IP Library › Granted Patent US 10,762,716

Granted Patent B1

US 10,762,716 · App. 16/581,685 · Granted Sep 1, 2020

Devices, methods, and graphical user interfaces for displaying objects in 3D contexts

Inventors: Grant R. Paul (San Francisco, CA); Nicolas V. Scapel (Sunnyvale, CA); David Lui (San Jose, CA); Xiao Jin Yu (Sunnyvale, CA); Hon-ming Chen (San Jose, CA)

Assignee: APPLE INC.

G06T19/006 G06F3/04815 G06F3/04842

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,762,716
App. No.: 16/581,685
Filed: Sep 24, 2019
Granted: Sep 1, 2020
Kind: B1
Examiner: WU, CHONG
Art Unit: 2613
USPC: 345/633

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

An electronic device detects a first manipulation input directed to a respective virtual object of a plurality of virtual objects. In response to detecting the first manipulation input, in accordance with a determination that the plurality of virtual objects are displayed with at least a portion of a field of view of the one or more cameras, the device changes a display property of the respective virtual object based on the first manipulation input, and, in accordance with a determination that the plurality of virtual objects are displayed in a user interface that does not include the field of view of the one or more cameras, the electronic device changes a display property of the plurality of virtual objects based on the first manipulation input.

Claims (144)

1. A method, comprising:

at an electronic device including a display generation component, one or more input devices, and one or more cameras:

displaying, by the display generation component, a plurality of virtual objects that include a first virtual object and a second virtual object;

while displaying the plurality of virtual objects, detecting a first manipulation input directed to the first virtual object of the plurality of virtual objects; and

in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects:

in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that includes at least a portion of a field of view of the one or more cameras, changing a display property of the first virtual object based on the first manipulation input without changing a display property of the second virtual object based on the first manipulation input; and

in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras:

changing the display property of the first virtual object based on the first manipulation input; and

changing the display property of the second virtual object based on the first manipulation input.

2. The method of claim 1 , wherein:

while the plurality of virtual objects are displayed in the user interface region that does not include the field of view of the one or more cameras, the first virtual object and the second virtual object of the plurality of virtual objects are each displayed with a first set of display characteristics.

3. The method of claim 2 , wherein:

the first set of display characteristics includes a height ordering characteristic; and

displaying the plurality of virtual objects in accordance with the height ordering characteristic includes ordering the first virtual object and the second virtual object of the plurality of virtual objects in accordance with respective heights of the first virtual object and the second virtual object.

4. The method of claim 2 , wherein:

the first set of display characteristics includes a spacing characteristic; and

displaying the plurality of virtual objects in accordance with the spacing characteristic includes positioning the first virtual object and the second virtual object of the plurality of virtual objects with a predefined distance between respective boundaries of the first virtual object and the second virtual object.

5. The method of claim 4 , wherein:

the first virtual object of the plurality of virtual objects has a first boundary that corresponds to a static state of the first virtual object;

the first virtual object is associated with a first action that causes at least a portion of the first virtual object to extend beyond the first boundary;

the first virtual object has a second boundary that corresponds to an extended range of the first virtual object during the first action;

the second virtual object of the plurality of virtual objects is adjacent to the first virtual object and has a third boundary that corresponds to a static state of the second virtual object; and

the third boundary of the second virtual object includes the second boundary of the first virtual object.

6. The method of claim 2 , including:

while displaying the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras, after changing the display property of the first virtual object based on the first manipulation input, detecting a request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras; and

in response to detecting the request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, displaying the user interface region that does not include the field of view of the one or more cameras, including displaying the first virtual object and the second virtual object with the first set of display characteristics.

7. The method of claim 6 , including:

after changing the display property of the first virtual object and changing the display property of the second virtual object based on the first manipulation input, while displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, detecting a request to re-display the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras; and

in response to detecting the request to re-display the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras, re-displaying the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras, including re-displaying the first virtual object and the second virtual object with the display property of the first virtual object and the display property of the second virtual object as changed based on the first manipulation input.

8. The method of claim 1 , wherein:

the first manipulation input is a rotation input.

9. The method of claim 1 , including:

prior to displaying the plurality of virtual objects, displaying a selection user interface;

while the selection user interface is displayed, detecting selection of the plurality of virtual objects that include the first virtual object and the second virtual object;

while the plurality of virtual objects are selected, receiving a request to display the plurality of virtual objects; and

in response to receiving the request to display the plurality of virtual objects, displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras.

10. The method of claim 9 , wherein:

the first virtual object of the plurality of virtual objects has a first defined plane type;

the second virtual object of the plurality of virtual objects has a second defined plane type that is distinct from the first defined plane type; and

in response to receiving the request to display the plurality of virtual objects while displaying the first virtual object and the second virtual object in the user interface region that includes the field of view of the one or more cameras:

the first virtual object of the plurality of virtual objects is displayed at a fixed position relative to a first plane that corresponds to the first defined plane type; and

the second virtual object of the plurality of virtual objects is displayed at a fixed position relative to a second plane that corresponds to the second defined plane type that is distinct from the first defined plane type.

11. The method of claim 10 , including:

while displaying the first virtual object and the second virtual object in the user interface region that includes the field of view of the one or more cameras, displaying the first virtual object at the fixed position relative to the first plane, and displaying the second virtual object at the fixed position relative to the second plane, detecting a rotation input; and

in response to detecting the rotation input:

in accordance with a determination that the rotation input is directed to the first virtual object, rotating the first virtual object about a first axis; and

in accordance with a determination that the rotation input is directed to the second virtual object, rotating the second virtual object about a second axis that is distinct from the first axis.

12. A computer system, comprising:

a display generation component;

one or more input devices;

one or more cameras;

one or more processors; and

memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, by the display generation component, a plurality of virtual objects that include a first virtual object and a second virtual object;

while displaying the plurality of virtual objects, detecting a first manipulation input directed to the first virtual object of the plurality of virtual objects; and

in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects:

in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras;

changing the display property of the first virtual object based on the first manipulation input; and

changing the display property of the second virtual object based on the first manipulation input.

13. The computer system of claim 12 , wherein:

14. The computer system of claim 13 , wherein:

the first set of display characteristics includes a height ordering characteristic; and

15. The computer system of claim 13 , wherein:

the first set of display characteristics includes a spacing characteristic; and

16. The computer system of claim 15 , wherein:

the first virtual object of the plurality of virtual objects has a first boundary that corresponds to a static state of the first virtual object;

the first virtual object is associated with a first action that causes at least a portion of the first virtual object to extend beyond the first boundary;

the first virtual object has a second boundary that corresponds to an extended range of the first virtual object during the first action;

the second virtual object of the plurality of virtual objects is adjacent to the first virtual object and has a third boundary that corresponds to a static state of the second virtual object; and

the third boundary of the second virtual object includes the second boundary of the first virtual object.

17. The computer system of claim 13 , wherein the one or more programs include instructions for:

18. The computer system of claim 17 , wherein the one or more programs include instructions for:

after changing the display property of the first virtual object and changing the display property of the second virtual object based on the first manipulation input, while displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, detecting a request to re-display the plurality of virtual objects in a user interface region that includes at least a portion of a field of view of the one or more cameras; and

in response to detecting the request to re-display the plurality of virtual objects with at least a portion of a field of view of the one or more cameras, re-displaying the plurality of virtual objects in a user interface region that includes at least a portion of a field of view of the one or more cameras, including re-displaying the first virtual object and the second virtual object with the display property of the first virtual object and the display property of the second virtual object as changed based on the first manipulation input.

19. The computer system of claim 12 , wherein:

the first manipulation input is a rotation input.

20. The computer system of claim 12 , wherein the one or more programs include instructions for:

prior to displaying the plurality of virtual objects, displaying a selection user interface;

while the selection user interface is displayed, detecting selection of the plurality of virtual objects including the first virtual object and the second virtual object;

while the plurality of virtual objects are selected, receiving a request to display the plurality of virtual objects; and

21. The computer system of claim 20 , wherein:

the first virtual object of the plurality of virtual objects has a first defined plane type;

the second virtual object of the plurality of virtual objects has a second defined plane type that is distinct from the first defined plane type; and

in response to receiving the request to display the plurality of virtual objects:

the first virtual object of the plurality of virtual objects is displayed at a fixed position relative to a first plane that corresponds to the first defined plane type; and

22. The computer system of claim 21 , wherein the one or more programs include instructions for:

while the first virtual object is displayed at the fixed position relative to the first plane and the second virtual object is displayed at the fixed position relative to the second plane, detecting a rotation input; and

in response to detecting the rotation input:

in accordance with a determination that the rotation input is directed to the first virtual object, rotating the first virtual object about a first axis; and

in accordance with a determination that the rotation input is directed to the second virtual object, rotating the second virtual object about a second axis that is distinct from the first axis.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a computer system with a display generation component, one or more input devices, and one or more cameras, cause the computer system to:

display, by the display generation component, a plurality of virtual objects that include a first virtual object and a second virtual object;

while displaying the plurality of virtual objects, detect a first manipulation input directed to the first virtual object of the plurality of virtual objects; and

in response to detecting the first manipulation input directed to the first virtual object of the plurality of virtual objects:

in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that includes at least a portion of a field of view of the one or more cameras, change a display property of the first virtual object based on the first manipulation input without changing a display property of the second virtual object based on the first manipulation input; and

in accordance with a determination that the plurality of virtual objects are displayed in a user interface region that does not include the field of view of the one or more cameras:

change the display property of the first virtual object based on the first manipulation input; and

change the display property of the second virtual object based on the first manipulation input.

24. The computer readable storage medium of claim 23 , wherein:

25. The computer readable storage medium of claim 24 , wherein:

the first set of display characteristics includes a height ordering characteristic; and

26. The computer readable storage medium of claim 24 , wherein:

the first set of display characteristics includes a spacing characteristic; and

27. The computer readable storage medium of claim 26 , wherein:

the first virtual object of the plurality of virtual objects has a first boundary that corresponds to a static state of the first virtual object;

the first virtual object is associated with a first action that causes at least a portion of the first virtual object to extend beyond the first boundary;

the first virtual object has a second boundary that corresponds to an extended range of the first virtual object during the first action;

the second virtual object of the plurality of virtual objects is adjacent to the first virtual object and has a third boundary that corresponds to a static state of the second virtual object; and

the third boundary of the second virtual object includes the second boundary of the first virtual object.

28. The computer readable storage medium of claim 24 , wherein the one or more programs include instructions that, when executed by the computer system, cause the computer system to:

while displaying the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras, after changing the display property of the first virtual object based on the first manipulation input, detect a request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras; and

in response to detecting the request to display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, display the user interface region that does not include the field of view of the one or more cameras, including displaying the first virtual object and the second virtual object with the first set of display characteristics.

29. The computer readable storage medium of claim 28 , wherein the one or more programs include instructions that, when executed by the computer system, cause the computer system to:

after changing the display property of the first virtual object and changing the display property of the second virtual object based on the first manipulation input, while displaying the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras, detect a request to re-display the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras; and

in response to detecting the request to re-display the plurality of virtual objects with at least a portion of a field of view of the one or more cameras, re-display the plurality of virtual objects in the user interface region that includes at least a portion of a field of view of the one or more cameras, including re-displaying the first virtual object and the second virtual object with the display property of the first virtual object and the display property of the second virtual object as changed based on the first manipulation input.

30. The computer readable storage medium of claim 23 , wherein:

the first manipulation input is a rotation input.

31. The computer readable storage medium of claim 23 , wherein the one or more programs include instructions that, when executed by the computer system, cause the computer system to:

prior to displaying the plurality of virtual objects, display a selection user interface;

while the selection user interface is displayed, detect selection of the plurality of virtual objects including the first virtual object and the second virtual object;

while the plurality of virtual objects are selected, receive a request to display the plurality of virtual objects; and

in response to receiving the request to display the plurality of virtual objects, display the plurality of virtual objects in the user interface region that does not include the field of view of the one or more cameras.

32. The computer readable storage medium of claim 31 , wherein:

the first virtual object of the plurality of virtual objects has a first defined plane type;

the second virtual object of the plurality of virtual objects has a second defined plane type that is distinct from the first defined plane type; and

in response to receiving the request to display the plurality of virtual objects:

the first virtual object of the plurality of virtual objects is displayed at a fixed position relative to a first plane that corresponds to the first defined plane type; and

33. The computer readable storage medium of claim 32 , wherein the one or more programs include instructions that, when executed by the computer system, cause the computer system to:

in response to detecting the rotation input:

in accordance with a determination that the rotation input is directed to the first virtual object, rotate the first virtual object about a first axis; and

in accordance with a determination that the rotation input is directed to the second virtual object, rotate the second virtual object about a second axis that is distinct from the first axis.

Assignments (1)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2020

From: PAUL, GRANT R.; SCAPEL, NICOLAS V.; LUI, DAVID; YU, XIAO JIN; CHEN, HON-MING

To: APPLE INC.

Reel/Frame 052435/0510 →

Continuity (2)

Provisional Application 62855973 · Jun 1, 2019

Provisional Application 62844010 · May 6, 2019

Cited By (58)

US 1,139,455 US 12,200,342 US 12,236,515 US 12,236,546 US 12,242,706 US 12,265,705 US 12,283,020 US 12,287,921 US 12,288,285 US 12,307,066 US 12,307,067 US 12,307,580 US 12,333,083 US 12,405,700 US 12,406,451 US 12,437,494 US 12,443,273 US 12,456,271 US 12,461,641 US 12,462,498 US 12,469,207 US 12,475,635 US 12,511,847 US 12,524,142 US 12,524,956 US 12,524,977 US 12,535,931 US 12,536,762 US 12,541,280 US 12,542,862 US 12,566,532 US 12,572,198 US 12,592,043 US 12,596,472 US 12,602,154 US 12,608,079 US 12,608,115 US 12,608,877 US 12,608,890 US 12,619,303 US 12,619,701 US 12,620,187 US 12,625,546 US 12,625,608 US 12,632,157 US 12,632,170 US 12,633,044 US 12,645,778 US 12,663,907 US 12,675,958 US 12,681,587 US 12,687,922 US 12,688,657 US 12,688,660 US 12,705,009 US 12,711,329 US 12,737,093 US 12,737,107