Granted Patent B2

US 12,242,706 · App. 17/812,965 · Granted Mar 4, 2025

Inventors: David A. Lipton (San Mateo, CA); Ryan S. Burgoyne (Lyons, CO); Michelle Chua (Seattle, WA); Zachary Z. Becker (Kirkland, WA); Karen N. Wong (Sunnyvale, CA); Eric G. Thivierge (Merchantville, NJ); Mahdi Nabiyouni (Emeryville, CA); Eric Chiu (San Jose, CA); Tyler L. Casella (Martinez, CA)

Assignee: Apple Inc.

G06F3/04815G02B27/017G06F3/013

Abstract

Claims (138)

1. A method comprising:

at a first electronic device in communication with a display and one or more input devices:

displaying, via the display, a three-dimensional environment including a three-dimensional preview, wherein the three-dimensional preview includes:

a first object; and

one or more user interface elements;

while displaying the three-dimensional preview including the first object and the one or more user interface elements, receiving, via the one or more input devices, a first input corresponding to a selection of the first object;

in accordance with a determination that the three-dimensional preview is operating in a first mode, causing the first object to perform a first action in accordance with a determination that the first object is configured to perform the first action in accordance with the selection, wherein the first action includes movement of the first object in the three-dimensional preview; and

in accordance with a determination that the three-dimensional preview is operating in a second mode, different than the first mode, changing an appearance of the first object in the three-dimensional preview to indicate the selection of the first object;

wherein:

the first electronic device is in further communication with a second electronic device;

the second electronic device is configured to display a content creation application including a two-dimensional representation of the first object; and

the three-dimensional preview includes a tray, and the first object is disposed on a surface on the tray.

2. The method of claim 1 , wherein:

the first electronic device is in further communication with a second electronic device; and

the second electronic device is configured to display a content creation application including a two-dimensional representation of the first object.

3. The method of claim 2 , wherein:

the second electronic device is a laptop computer, a desktop computer, or a tablet computer; and

the first electronic device is a head-mounted display device.

4. The method of claim 2 , wherein causing the first object to perform the first action comprises:

presenting a first animation of the first object including the movement in the three-dimensional environment; and

in accordance with the determination that the three-dimensional preview is operating in the first mode, in accordance with the determination that the first object is configured to perform the first action, concurrently displaying the first animation of the two-dimensional representation of the first object at the second electronic device.

5. The method of claim 2 , further comprising:

in accordance with a determination that the three-dimensional preview is operating in the second mode, changing the appearance of the two-dimensional representation of the first object displayed at the second electronic device to indicate the selection of the first object.

6. The method of claim 5 , wherein:

changing the appearance of the first object in the three-dimensional environment comprises highlighting the first object in the three-dimensional environment; and

changing the appearance of the two-dimensional representation of the first object displayed at the second electronic device comprises highlighting the two-dimensional representation of the first object displayed at the second electronic device.

7. The method of claim 6 , further comprising:

while concurrently highlighting the first object in the three-dimensional environment and the two-dimensional representation of the first object displayed at the second electronic device, receiving a second input at the second electronic device; and

in accordance with the second input, updating a characteristic of the two-dimensional representation of the first object displayed at the second electronic device and updating the characteristic of the first object in the three-dimensional environment displayed at the first electronic device.

8. The method of claim 1 , wherein the one or more user interface elements are displayed within a tool bar associated with the tray.

9. The method of claim 1 , further comprising:

capturing, via the one or more input devices, at least portions of a real-world environment including the second electronic device and the content creation application displayed on the second electronic device;

wherein the three-dimensional environment further includes a representation of the captured portions of the real-world environment, including:

a representation of the second electronic device; and

a representation of the content creation application.

10. The method of claim 9 , wherein the one or more user interface elements comprises:

a first user interface element that is selectable to cause the three-dimensional preview to operate in the first mode;

a second user interface element that is selectable to cause the three-dimensional preview to cease operating in the first mode and to operate in the second mode;

a third user interface element that is selectable to scale the first object from a first size to a second size, larger than the first size;

a fourth user interface element that is selectable to cause the first object to scrub through the first action; and/or

a fifth user interface element that is selectable to cause display of a full-scale representation of the first object.

11. The method of claim 10 , wherein displaying the full-scale representation comprises:

occluding at least portions of the three-dimensional environment and displaying the full-scale representation of the first object over the occluded portions of the three-dimensional environment.

12. The method of claim 11 , wherein displaying the full-scale representation comprises:

concurrently displaying in the three-dimensional environment:

the representation of the three-dimensional preview; and

the representation of the second electronic device including the representation of the content creation application and the two-dimensional representation of the first object.

13. The method of claim 1 , further comprising:

receiving, via the one or more input devices, a second input;

in accordance with a determination that the second input corresponds to a request to move the three-dimensional preview, moving the three-dimensional preview within the three-dimensional environment in accordance with the second input;

in accordance with a determination that the second input corresponds to a request to change a size of the tray, changing the size of the tray from a first size to a second size, different from the first size in accordance with the second input;

in accordance with a determination that the second input corresponds to a request to rotate the tray, rotating the tray and the first object disposed on the surface of the tray within the three-dimensional environment in accordance with the second input; and

in accordance with a determination that the second input corresponds to a request to elevate the tray, moving the first object from a first height to a second height, offset from an initial position of the tray within the three-dimensional environment in accordance with the second input.

14. The method of claim 13 , wherein:

the determination that the second input corresponds to the request to rotate the tray is in accordance with a determination that the second input includes:

an interaction input provided by a predetermined portion of a user of the first electronic device for at least a threshold amount of time while a gaze of the user is directed toward the tray, wherein a representation of a pivot point is displayed at a respective location on the tray in response to detecting the interaction input; and

movement of the predetermined portion of the user of the first electronic device in a respective direction relative to the representation of the pivot point on the tray while maintaining the interaction input; and

rotating the tray and the first object disposed on the surface of the tray within the three-dimensional environment in accordance with the second input includes rotation of the tray and the first object disposed on the surface of the tray about an axis through the pivot point.

15. The method of claim 14 , further comprising:

ceasing displaying the representation of the pivot point on the tray in response to a release of the interaction input provided by the predetermined portion of the user.

16. The method of claim 1 , further comprising:

receiving, via the one or more input devices, a second input corresponding to a request to manipulate the first object in the three-dimensional environment;

in accordance with a determination that the second input corresponds to a request to change a size of the first object, changing the size of the first object from a first size to a second size, different than the first size in accordance with the second input; and

in accordance with a determination that the second input corresponds to a request to reposition the first object on the tray, moving the first object from a first location to a second location, different than the first location, on the tray in accordance with the second input.

17. The method of claim 16 , wherein:

the determination that the second input corresponds to the request to change the size of the first object is in accordance with a determination that the second input includes:

an interaction input provided by a predetermined portion of a user of the first electronic device for at least a threshold amount of time while a gaze of the user is directed toward the tray, wherein a representation of a pivot point is displayed at a respective location on the tray in response to detecting the interaction input;

movement of the predetermined portion of the user of the first electronic device in a respective direction relative to the representation of the pivot point on the tray; and

changing the size of the tray from the first size to the second size, different from the first size in accordance with the second input is based on the amount of the movement of the predetermined portion of the user of the first electronic device.

18. The method of claim 16 , further comprising:

in accordance with a determination that the second input corresponds to a request to reposition the first object outside a boundary of the tray:

ceasing to display the first object; and

changing an appearance of the three-dimensional preview, wherein changing the appearance of the three-dimensional preview includes:

highlighting a portion of a boundary of the tray corresponding to the second location within the three-dimensional environment; and/or

displaying one or more vertical indicators above the boundary of the tray corresponding to the second location within the environment.

19. An electronic device comprising:

one or more processors;

memory; and

one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising:

displaying, via a display, a three-dimensional environment including a three-dimensional preview, wherein the three-dimensional preview includes:

a first object; and

one or more user interface elements;

while displaying the three-dimensional preview including the first object and the one or more user interface elements, receiving, via one or more input devices, a first input corresponding to a selection of the first object;

in accordance with a determination that the three-dimensional preview is operating in a first mode, causing the first object to perform a first action in accordance with a determination that the first object is configured to perform the first action in accordance with the selection, wherein the first action includes movement of the first object in the three-dimensional preview; and

in accordance with a determination that the three-dimensional preview is operating in a second mode, different than the first mode, changing an appearance of the first object in the three-dimensional preview to indicate the selection of the first object;

wherein:

the electronic device is in further communication with a second electronic device;

the second electronic device is configured to display a content creation application including a two-dimensional representation of the first object; and

the three-dimensional preview includes a tray, and the first object is disposed on a surface on the tray.

20. The electronic device of claim 19 , wherein causing the first object to perform the first action comprises:

presenting a first animation of the first object including the movement in the three-dimensional environment; and

in accordance with the determination that the three-dimensional preview is operating in the first mode, in accordance with the determination that the first object is configured to perform the first action, concurrently displaying the first animation of the two-dimensional representation of the first object at the second electronic device.

21. The electronic device of claim 19 , wherein the method further comprises:

in accordance with a determination that the three-dimensional preview is operating in the second mode, changing the appearance of the two-dimensional representation of the first object displayed at the second electronic device to indicate the selection of the first object.

22. The electronic device of claim 21 , wherein:

changing the appearance of the first object in the three-dimensional environment comprises highlighting the first object in the three-dimensional environment; and

changing the appearance of the two-dimensional representation of the first object displayed at the second electronic device comprises highlighting the two-dimensional representation of the first object displayed at the second electronic device.

23. The electronic device of claim 22 , wherein the method further comprises:

while concurrently highlighting the first object in the three-dimensional environment and the two-dimensional representation of the first object displayed at the second electronic device, receiving a second input at the second electronic device; and

in accordance with the second input, updating a characteristic of the two-dimensional representation of the first object displayed at the second electronic device and updating the characteristic of the first object in the three-dimensional environment displayed at the electronic device.

24. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to perform a method comprising:

displaying, via a display, a three-dimensional environment including a three-dimensional preview, wherein the three-dimensional preview includes:

a first object; and

one or more user interface elements;

while displaying the three-dimensional preview including the first object and the one or more user interface elements, receiving, via one or more input devices, a first input corresponding to a selection of the first object;

in accordance with a determination that the three-dimensional preview is operating in a first mode, causing the first object to perform a first action in accordance with a determination that the first object is configured to perform the first action in accordance with the selection, wherein the first action includes movement of the first object in the three-dimensional preview; and

in accordance with a determination that the three-dimensional preview is operating in a second mode, different than the first mode, changing an appearance of the first object in the three-dimensional preview to indicate the selection of the first object;

wherein:

the first electronic device is in further communication with a second electronic device;

the second electronic device is configured to display a content creation application including a two-dimensional representation of the first object; and

the three-dimensional preview includes a tray, and the first object is disposed on a surface on the tray.

25. The non-transitory computer readable storage medium of claim 24 , wherein the method further comprises:

capturing, via the one or more input devices, at least portions of a real-world environment including the second electronic device and the content creation application displayed on the second electronic device;

wherein the three-dimensional environment further includes a representation of the captured portions of the real-world environment, including:

a representation of the second electronic device; and

a representation of the content creation application.

26. The non-transitory computer readable storage medium of claim 24 , wherein the method further comprises:

receiving, via the one or more input devices, a second input;

in accordance with a determination that the second input corresponds to a request to move the three-dimensional preview, moving the three-dimensional preview within the three-dimensional environment in accordance with the second input;

in accordance with a determination that the second input corresponds to a request to change a size of the tray, changing the size of the tray from a first size to a second size, different from the first size in accordance with the second input;

in accordance with a determination that the second input corresponds to a request to rotate the tray, rotating the tray and the first object disposed on the surface of the tray within the three-dimensional environment in accordance with the second input; and

in accordance with a determination that the second input corresponds to a request to elevate the tray, moving the first object from a first height to a second height, offset from an initial position of the tray within the three-dimensional environment in accordance with the second input.

27. The non-transitory computer readable storage medium of claim 26 , wherein:

the determination that the second input corresponds to the request to rotate the tray is in accordance with a determination that the second input includes:

an interaction input provided by a predetermined portion of a user of the first electronic device for at least a threshold amount of time while a gaze of the user is directed toward the tray, wherein a representation of a pivot point is displayed at a respective location on the tray in response to detecting the interaction input; and

movement of the predetermined portion of the user of the first electronic device in a respective direction relative to the representation of the pivot point on the tray while maintaining the interaction input; and

rotating the tray and the first object disposed on the surface of the tray within the three-dimensional environment in accordance with the second input includes rotation of the tray and the first object disposed on the surface of the tray about an axis through the pivot point.

28. The non-transitory computer readable storage medium of claim 27 , wherein the method further comprises:

ceasing displaying the representation of the pivot point on the tray in response to a release of the interaction input provided by the predetermined portion of the user.

29. The electronic device of claim 19 , wherein the method further comprises:

receiving, via the one or more input devices, a second input corresponding to a request to manipulate the first object in the three-dimensional environment;

in accordance with a determination that the second input corresponds to a request to change a size of the first object, changing the size of the first object from a first size to a second size, different than the first size in accordance with the second input; and

in accordance with a determination that the second input corresponds to a request to reposition the first object on the tray, moving the first object from a first location to a second location, different than the first location, on the tray in accordance with the second input.

30. The electronic device of claim 29 , wherein:

the determination that the second input corresponds to the request to change the size of the first object is in accordance with a determination that the second input includes:

an interaction input provided by a predetermined portion of a user of the electronic device for at least a threshold amount of time while a gaze of the user is directed toward the tray, wherein a representation of a pivot point is displayed at a respective location on the tray in response to detecting the interaction input;

movement of the predetermined portion of the user of the electronic device in a respective direction relative to the representation of the pivot point on the tray; and

changing the size of the tray from the first size to the second size, different from the first size in accordance with the second input is based on the amount of the movement of the predetermined portion of the user of the electronic device.

Assignments (1)

Continuity (2)

References Cited (190)