Granted Patent B2

US 12,663,875 · App. 19/092,989 · Granted Jun 23, 2026

Inventors: William D. Lindmeier (San Francisco, CA); Tony Kobayashi (Berkeley, CA); Alexis H. Palangie (Palo Alto, CA); Carmine Elvezio (San Francisco, CA); Matthew J. Sundstrom (Campbell, CA)

Assignee: Apple Inc.

G06F3/017G06F3/0346G06F3/04815G06F3/04845

Abstract

Claims (98)

1 . A method comprising:

at a computer system in communication with a display generation component and one or more input devices:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input directed to the virtual object, wherein the first user input includes input from a physical object, and the first user input corresponds to a request to move the virtual object; and

while detecting the first user input:

in accordance with a determination that the first user input satisfies one or more first criteria, including a criterion that is satisfied when a location corresponding to the virtual object is within a threshold distance of the physical object, moving the virtual object in accordance with the first user input with a first degree of constraint on a movement of the virtual object; and

in accordance with a determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, moving the virtual object in accordance with the first user input with a second degree of constraint on the movement of the virtual object, wherein the second degree of constraint on the movement of the virtual object is greater than the first degree of constraint on the movement of the virtual object.

2 . The method of claim 1 , wherein the second degree of constraint on the movement of the virtual object allows for three degrees of movement of the virtual object.

3 . The method of claim 1 , wherein the first degree of constraint on the movement of the virtual object allows for six degrees of movement of the virtual object.

4 . The method of claim 1 , wherein the second degree of constraint on the movement of the virtual object allows for rotation of the virtual object.

5 . The method of claim 1 , comprising while detecting the first user input:

in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, displaying, in the user interface, a visual indication corresponding to the second degree of constraint on the movement of the virtual object, Indication rotates around object with movement.

6 . The method of claim 5 , wherein the visual indication moves relative to the virtual object in a rotational manner during the first user input in accordance with the movement of the virtual object.

7 . The method of claim 5 , wherein the first user input includes a selection input detected from the physical object, and the movement of the virtual object is in accordance with a portion of the first user input detected after the selection input, the method further comprising:

in response to detecting the selection input from the physical object, updating a visual appearance of the visual indication in accordance with the selection input.

8 . The method of claim 1 , comprising before detecting the first user input:

displaying, via the display generation component, the virtual object with a respective spatial relationship relative to a displayed virtual surface in the user interface.

9 . The method of claim 8 , comprising:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, detecting, via the one or more input devices, the first user input that satisfies the one or more first criteria; and

in response to detecting the first user input, moving the virtual object in accordance with the first user input with the first degree of constraint on the movement of the virtual object.

10 . The method of claim 8 , comprising:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, detecting, via the one or more input devices, attention of a user directed to a region of the user interface associated with the displayed virtual surface; and

in response to detecting the attention of the user directed to the region associated with the virtual surface, displaying, in the user interface, a virtual element that is interactable to move the virtual surface and the virtual object relative to the user interface.

11 . The method of claim 8 , comprising:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, detecting, via the one or more input devices, a second user input that corresponds to a request to rotate the displayed virtual surface; and

in response to detecting the second user input, rotating the virtual surface and the virtual object in the user interface in accordance with the second user input.

12 . The method of claim 11 , comprising:

before detecting the second user input and while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that attention of a user of the computer system is not directed to a region of the user interface associated with the virtual surface, and in accordance with a determination that a location of a second physical object relative to the virtual object satisfies one or more second criteria, displaying, via the display generation component, the virtual surface comprising a plurality of visual indications that indicate a degree of rotation of the virtual surface, wherein the plurality of visual indications are displayed at a first position along the displayed virtual surface corresponding to a current location of the second physical object relative to the virtual surface;

while displaying the plurality of visual indications at the first position along the virtual surface, detecting, via the one or more input devices, movement of the second physical object relative to the displayed virtual surface; and

in response to detecting the movement of the second physical object relative to the virtual surface, displaying the plurality of visual indications at a second position, different from the first position, along the virtual surface that corresponds to the current location of the second physical object relative to the virtual surface.

13 . The method of claim 12 , wherein the second user input includes a selection input detected from the second physical object, and the rotation of the virtual object is in accordance with a portion of the second user input detected after the selection input, the method further comprising:

in response to detecting the selection input in the second user input, updating a visual appearance of the plurality of visual indications to indicate that the selection input has been detected.

14 . The method of claim 8 , comprising:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that the physical object has a first location relative to the virtual surface, displaying, in the user interface, a visual indication at a first position relative to the virtual surface corresponding to the first location of the physical object relative to the virtual surface; and

in accordance with a determination that the physical object has a second location, different from the first location, relative to the virtual surface, displaying, in the user interface, the visual indication at a second position, different from the first position, relative to the virtual surface corresponding to the second location of the physical object relative to the displayed virtual surface.

15 . The method of claim 8 , wherein the first user input is detected when displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, the method further comprising:

in response to detecting the first user input that satisfies the one or more first criteria, ceasing display of the virtual surface in the user interface.

16 . The method of claim 15 , comprising:

while display of the virtual surface is ceased in the user interface, detecting, via the one or more input devices, an end of the first user input; and in response to detecting the end of the first user input, redisplaying, in the user interface, the virtual surface, wherein the virtual object and the end of the first user input has the respective spatial relationship relative to the redisplayed virtual surface.

17 . The method of claim 16 , wherein when the first user input is detected, the virtual surface has a second respective spatial relationship relative to the user interface, and redisplaying the virtual surface in the user interface in response to the end of the first user input includes displaying the virtual surface with the second respective spatial relationship relative to the user interface.

18 . The method of claim 8 , comprising:

while detecting the first user input, in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, maintaining displaying of the virtual surface in the user interface.

19 . The method of claim 8 , wherein moving the virtual object with the second degree of constraint in accordance with the first user input includes moving the virtual object based on a first simulated physical property associated with the virtual object.

20 . The method of claim 19 , wherein moving the virtual object with the first degree of constraint in accordance with the first user input includes moving the virtual object in a manner that is not based on the first simulated physical property.

21 . The method of claim 1 , wherein the one or more first criteria include a criterion that is satisfied when the physical object is within a field of view of a viewpoint of a user of the computer system.

22 . A computer system that is in communication with a display generation component and one or more input devices, the computer system comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input directed to the virtual object, wherein the first user input includes input from a physical object, and the first user input corresponds to a request to move the virtual object; and

while detecting the first user input:

in accordance with a determination that the first user input satisfies one or more first criteria, including a criterion that is satisfied when a location corresponding to the virtual object is within a threshold distance of the physical object, moving the virtual object in accordance with the first user input with a first degree of constraint on a movement of the virtual object; and

in accordance with a determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, moving the virtual object in accordance with the first user input with a second degree of constraint on the movement of the virtual object, wherein the second degree of constraint on the movement of the virtual object is greater than the first degree of constraint on the movement of the virtual object.

23 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a computer system that is in communication with a display generation component and one or more input devices, cause the computer system to perform a method comprising:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input directed to the virtual object, wherein the first user input includes input from a physical object, and the first user input corresponds to a request to move the virtual object; and

while detecting the first user input:

in accordance with a determination that the first user input satisfies one or more first criteria, including a criterion that is satisfied when a location corresponding to the virtual object is within a threshold distance of the physical object, moving the virtual object in accordance with the first user input with a first degree of constraint on a movement of the virtual object; and

in accordance with a determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, moving the virtual object in accordance with the first user input with a second degree of constraint on the movement of the virtual object, wherein the second degree of constraint on the movement of the virtual object is greater than the first degree of constraint on the movement of the virtual object.

24 . The computer system of claim 22 , wherein the one or more programs further include instructions for, while detecting the first user input:

in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, displaying, in the user interface, a visual indication corresponding to the second degree of constraint on the movement of the virtual object.

25 . The computer system of claim 22 , wherein the one or more programs further include instructions for, before detecting the first user input:

displaying, via the display generation component, the virtual object with a respective spatial relationship relative to a displayed virtual surface in the user interface.

26 . The computer system of claim 25 , wherein the one or more programs further include instructions for:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, detecting, via the one or more input devices, a second user input that corresponds to a request to rotate the displayed virtual surface; and

in response to detecting the second user input, rotating the virtual surface and the virtual object in the user interface in accordance with the second user input.

27 . The computer system of claim 26 , wherein the one or more programs further include instructions for:

before detecting the second user input and while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that attention of a user of the computer system is not directed to a region of the user interface associated with the virtual surface, and in accordance with a determination that a location of a second physical object relative to the virtual object satisfies one or more second criteria, displaying, via the display generation component, the virtual surface comprising a plurality of visual indications that indicate a degree of rotation of the virtual surface, wherein the plurality of visual indications are displayed at a first position along the displayed virtual surface corresponding to a current location of the second physical object relative to the virtual surface;

while displaying the plurality of visual indications at the first position along the virtual surface, detecting, via the one or more input devices, movement of the second physical object relative to the displayed virtual surface; and

in response to detecting the movement of the second physical object relative to the virtual surface, displaying the plurality of visual indications at a second position, different from the first position, along the virtual surface that corresponds to the current location of the second physical object relative to the virtual surface.

28 . The computer system of claim 25 , wherein the one or more programs further include instructions for:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that the physical object has a first location relative to the virtual surface, displaying, in the user interface, a visual indication at a first position relative to the virtual surface corresponding to the first location of the physical object relative to the virtual surface; and

in accordance with a determination that the physical object has a second location, different from the first location, relative to the virtual surface, displaying, in the user interface, the visual indication at a second position, different from the first position, relative to the virtual surface corresponding to the second location of the physical object relative to the displayed virtual surface.

29 . The computer system of claim 25 , wherein the one or more programs further include instructions for:

while detecting the first user input, in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, maintaining displaying of the virtual surface in the user interface.

30 . The non-transitory computer readable storage medium of claim 23 , wherein the method further comprises, while detecting the first user input:

in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, displaying, in the user interface, a visual indication corresponding to the second degree of constraint on the movement of the virtual object.

31 . The non-transitory computer readable storage medium of claim 23 , wherein the method further comprises, before detecting the first user input:

displaying, via the display generation component, the virtual object with a respective spatial relationship relative to a displayed virtual surface in the user interface.

32 . The non-transitory computer readable storage medium of claim 31 , wherein the method further comprises:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface, detecting, via the one or more input devices, a second user input that corresponds to a request to rotate the displayed virtual surface; and

in response to detecting the second user input, rotating the virtual surface and the virtual object in the user interface in accordance with the second user input.

33 . The non-transitory computer readable storage medium of claim 32 , wherein the method further comprises:

before detecting the second user input and while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that attention of a user of the computer system is not directed to a region of the user interface associated with the virtual surface, and in accordance with a determination that a location of a second physical object relative to the virtual object satisfies one or more second criteria, displaying, via the display generation component, the virtual surface comprising a plurality of visual indications that indicate a degree of rotation of the virtual surface, wherein the plurality of visual indications are displayed at a first position along the displayed virtual surface corresponding to a current location of the second physical object relative to the virtual surface;

while displaying the plurality of visual indications at the first position along the virtual surface, detecting, via the one or more input devices, movement of the second physical object relative to the displayed virtual surface; and

in response to detecting the movement of the second physical object relative to the virtual surface, displaying the plurality of visual indications at a second position, different from the first position, along the virtual surface that corresponds to the current location of the second physical object relative to the virtual surface.

34 . The non-transitory computer readable storage medium of claim 31 , wherein the method further comprises:

while displaying the virtual object with the respective spatial relationship relative to the displayed virtual surface:

in accordance with a determination that the physical object has a first location relative to the virtual surface, displaying, in the user interface, a visual indication at a first position relative to the virtual surface corresponding to the first location of the physical object relative to the virtual surface; and

in accordance with a determination that the physical object has a second location, different from the first location, relative to the virtual surface, displaying, in the user interface, the visual indication at a second position, different from the first position, relative to the virtual surface corresponding to the second location of the physical object relative to the displayed virtual surface.

35 . The non-transitory computer readable storage medium of claim 31 , wherein the method further comprises:

while detecting the first user input, in accordance with the determination that the first user input does not satisfy the one or more first criteria because the virtual object is not within the threshold distance of the physical object, maintaining displaying of the virtual surface in the user interface.

Continuity (4)

References Cited (400)