Granted Patent B2

US 12,625,608 · App. 18/473,197 · Granted May 12, 2026

Inventors: Israel Pastrana Vicente (Spring, TX); Evgenii Krivoruchko (San Francisco, CA); Kristi E. Bauerly (Los Altos, CA); Lorena S. Pazmino (San Francisco, CA)

Assignee: Apple Inc.

G06F3/04886G06F3/011G06F3/017G06F3/0481G06F3/04812G06F3/04815G06F3/04817G06F3/0484G06F3/04842G06F3/04845G06F3/0487G06F3/0488G06F3/013

Abstract

Claims (141)

1 . A method comprising:

at a computer system in communication with a display generation component and one or more input devices:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input that includes attention directed towards a first portion of the virtual object; and

while detecting the first user input:

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when the first user input includes a respective type of input from a first portion of a user at a location that corresponds to the first portion of the virtual object, displaying, via the display generation component, a first visual feedback indicating a location of the attention of the user in the first portion of the virtual object; and

in accordance with a determination that one or more second criteria are satisfied, including a criterion that is satisfied when the first user input includes the attention of the user directed towards the first portion of the virtual object without the respective type of input from the user directed towards the first portion of the virtual object, displaying second visual feedback that is different from the first visual feedback without displaying the first visual feedback in the first portion of the virtual object.

2 . The method of claim 1 , wherein displaying, via the display generation component, the second visual feedback includes displaying the second visual feedback having a shape resulting from masking a first shape corresponding to the attention of the user with a second shape of the virtual object, and displaying, via the display generation component, the first visual feedback includes displaying the first visual feedback having a shape that does not result from masking the first shape with the second shape.

3 . The method of claim 1 , further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a change in position of the first portion of the user relative to the virtual object; and

in response to detecting the change in position of the first portion of the user relative to the virtual object:

in accordance with a determination that the one or more first criteria were satisfied, changing a visual appearance of the first visual feedback by a first amount; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing changing a visual appearance of the second visual feedback by the first amount.

4 . The method of claim 3 , further comprising:

in response to detecting the change in position of the first portion of the user relative to the virtual object, and in accordance with the determination that the one or more second criteria were satisfied, forgoing changing the visual appearance of the second visual feedback based on a change in position of the first portion of the user relative to the virtual object.

5 . The method of claim 3 , wherein changing the visual appearance of the first visual feedback includes changing a level of brightness of the first visual feedback.

6 . The method of claim 3 , wherein changing the visual appearance of the first visual feedback includes changing a size of the first visual feedback.

7 . The method of claim 3 , wherein changing the visual appearance of the first visual feedback includes changing an amount of blur of the first visual feedback.

8 . The method of claim 1 , wherein the one or more second criteria include a criterion that is satisfied when the virtual object is a selectable object, and is not satisfied when the virtual object is not a selectable object.

9 . The method of claim 1 , wherein the one or more second criteria include a criterion that is satisfied when the first portion of the user is in a ready state.

10 . The method of claim 1 , further comprising:

while displaying the second visual feedback in the first portion of the virtual object in response to detecting the first user input:

detecting, via the one or more input devices, a second user input corresponding to an indirect input from the first portion of the user; and

in response to detecting the second user input, interacting with the virtual object based on the second user input.

11 . The method of claim 1 , further comprising:

while displaying the first visual feedback in the first portion of the virtual object in response to detecting the first user input:

detecting, via the one or more input devices, a second user input corresponding to a direct input from the first portion of the user; and

in response to detecting the second user input, interacting with the virtual object based on the second user input.

12 . The method of claim 1 , further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input, selecting the selectable button, including: in accordance with a determination that the one or more first criteria were satisfied when the second input was detected, moving the virtual object away from a viewpoint of the user in accordance with the second input; and

in accordance with a determination that the one or more second criteria were satisfied when the second input was detected, forgoing moving the virtual object away from the viewpoint of the user.

13 . The method of claim 1 , further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input:

selecting the selectable button;

in accordance with a determination that the one or more first criteria were satisfied, displaying a visual indication around at least a portion of a perimeter of the virtual object indicating that the virtual object has been selected; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing displaying the visual indication around the portion of the perimeter of the virtual object.

14 . The method of claim 1 , further comprising:

while displaying the second visual feedback at the first portion of the virtual object, detecting second user input that corresponds to movement of the first portion of the user toward the virtual object; and

in response to detecting the second user input, and in accordance with a determination that first portion of the user has moved within a threshold distance of the virtual object, ceasing display of the second visual feedback and displaying the first visual feedback in a respective portion of the virtual object.

15 . The method of claim 1 , wherein:

displaying the first visual feedback includes:

in accordance with a determination that the virtual object has a first size, displaying the first visual feedback having a first visual appearance; and

in accordance with a determination that the virtual object has a second size, displaying the first visual feedback having the first visual appearance; and

displaying the second visual feedback includes:

in accordance with a determination the virtual object has the first size, displaying the second visual feedback having a second visual appearance; and

in accordance with a determination that the virtual object has the second size, displaying the second visual feedback having a third visual appearance different from the second visual appearance.

16 . The method of claim 1 , further comprising:

while displaying, in a three-dimensional environment, a respective virtual object having a first size relative to the three-dimensional environment, detecting, via the one or more input devices, a second user input that includes attention directed towards a first portion of the respective virtual object; and

in response to detecting the second user input, in accordance with a determination that one or more third criteria are satisfied, displaying the respective virtual object having a second size, greater than the first size, relative to the three-dimensional environment.

17 . The method of claim 16 , wherein the second user input includes gaze of the user directed to the respective virtual object without input from the user other than the gaze of the user, and the one or more third criteria include a criterion that is satisfied when the second user input includes gaze of the user directed toward the respective virtual object without input other than the gaze from the user.

18 . The method of claim 17 , wherein the one or more third criteria include a criterion that is satisfied when gaze of the user is directed toward the respective virtual object, and a criterion that is satisfied when the second user input includes the respective type of input from the first portion of the user at a location that corresponds to the respective virtual object, the method further comprising:

while displaying the respective virtual object having the second size relative to the three-dimensional environment, detecting, via the one or more input devices, that the gaze of the user is no longer directed toward the respective virtual object and that the first portion of the user is no longer providing the respective type of input at the location corresponding to the respective virtual object; and

in response to detecting that the gaze of the user is no longer directed toward the respective virtual object and that the first portion of the user is no longer providing the respective type of input at the location corresponding to the respective virtual object, displaying, in the three-dimensional environment, the respective virtual object having the first size relative to the three-dimensional environment.

19 . The method of claim 18 , further comprising:

while displaying the respective virtual object having the second size relative to the three-dimensional environment, detecting, via the one or more input devices, that the first portion of the user is no longer providing the respective type of input at the location corresponding to the respective virtual object but that the gaze of the user is directed toward the respective virtual object; and

in response to detecting that the first portion of the user is no longer providing the respective type of input at the location corresponding to the respective virtual object but that the gaze of the user is directed toward the respective virtual object, maintaining, in the three-dimensional environment, display of the respective virtual object having the second size relative to the three-dimensional environment.

20 . The method of claim 18 , further comprising:

while displaying the respective virtual object having the second size relative to the three-dimensional environment, detecting, via the one or more input devices, that the gaze of the user is no longer directed toward the respective virtual object but that the first portion of the user is providing the respective type of input at the location corresponding to the respective virtual object; and

in response to detecting that that the gaze of the user is no longer directed toward the respective virtual object but that the first portion of the user is providing the respective type of input at the location corresponding to the respective virtual object, maintaining, in the three-dimensional environment, display of the respective virtual object having the second size relative to the three-dimensional environment.

21 . The method of claim 1 , wherein:

the first user input including a respective type of input from a first portion of the user at a location that corresponds to the first portion of the virtual object comprises hand-based attention directed towards the first portion of the virtual object; and

the first user input including the attention of the user directed towards the first portion of the virtual object without the respective type of input from the user directed towards the first portion of the virtual object comprises gaze-based attention directed towards the first portion of the virtual object, without hand-based attention directed towards the first portion of the virtual object.

22 . A computer system that is in communication with a display generation component and one or more input devices, the computer system comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input that includes attention directed towards a first portion of the virtual object; and

while detecting the first user input:

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when the first user input includes a respective type of input from a first portion of a user at a location that corresponds to the first portion of the virtual object, displaying, via the display generation component, a first visual feedback indicating a location of the attention of the user in the first portion of the virtual object; and

in accordance with a determination that one or more second criteria are satisfied, including a criterion that is satisfied when the first user input includes the attention of the user directed towards the first portion of the virtual object without the respective type of input from the user directed towards the first portion of the virtual object, displaying second visual feedback that is different from the first visual feedback without displaying the first visual feedback in the first portion of the virtual object.

23 . The computer system of claim 22 , the one or more programs further including instructions for:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a change in position of the first portion of the user relative to the virtual object; and

in response to detecting the change in position of the first portion of the user relative to the virtual object:

in accordance with a determination that the one or more first criteria were satisfied, changing a visual appearance of the first visual feedback by a first amount; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing changing a visual appearance of the second visual feedback by the first amount.

24 . The computer system of claim 23 , the one or more programs further including instructions for:

in response to detecting the change in position of the first portion of the user relative to the virtual object, and in accordance with the determination that the one or more second criteria were satisfied, forgoing changing the visual appearance of the second visual feedback based on a change in position of the first portion of the user relative to the virtual object.

25 . The computer system of claim 22 , the one or more programs further including instructions for:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input, selecting the selectable button, including:

in accordance with a determination that the one or more first criteria were satisfied when the second input was detected, moving the virtual object away from a viewpoint of the user in accordance with the second input; and

in accordance with a determination that the one or more second criteria were satisfied when the second input was detected, forgoing moving the virtual object away from the viewpoint of the user.

26 . The computer system of claim 22 , the one or more programs further including instructions for:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input:

selecting the selectable button;

in accordance with a determination that the one or more first criteria were satisfied, displaying a visual indication around at least a portion of a perimeter of the virtual object indicating that the virtual object has been selected; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing displaying the visual indication around the portion of the perimeter of the virtual object.

27 . The computer system of claim 22 , the one or more programs further including instructions for:

while displaying the second visual feedback at the first portion of the virtual object, detecting second user input that corresponds to movement of the first portion of the user toward the virtual object; and

in response to detecting the second user input, and in accordance with a determination that first portion of the user has moved within a threshold distance of the virtual object, ceasing display of the second visual feedback and displaying the first visual feedback in a respective portion of the virtual object.

28 . The computer system of claim 22 , wherein:

displaying the first visual feedback includes:

in accordance with a determination that the virtual object has a first size, displaying the first visual feedback having a first visual appearance; and

in accordance with a determination that the virtual object has a second size, displaying the first visual feedback having the first visual appearance; and

displaying the second visual feedback includes:

in accordance with a determination the virtual object has the first size, displaying the second visual feedback having a second visual appearance; and

in accordance with a determination that the virtual object has the second size, displaying the second visual feedback having a third visual appearance different from the second visual appearance.

29 . The computer system of claim 22 , the one or more programs further including instructions for:

while displaying, in a three-dimensional environment, a respective virtual object having a first size relative to the three-dimensional environment, detecting, via the one or more input devices, a second user input that includes attention directed towards a first portion of the respective virtual object; and

in response to detecting the second user input, in accordance with a determination that one or more third criteria are satisfied, displaying the respective virtual object having a second size, greater than the first size, relative to the three-dimensional environment.

30 . A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a computer system that is in communication with a display generation component and one or more input devices, cause the computer system to perform a method comprising:

displaying, via the display generation component, a user interface including a virtual object;

while displaying the user interface, detecting, via the one or more input devices, a first user input that includes attention directed towards a first portion of the virtual object; and

while detecting the first user input:

in accordance with a determination that one or more first criteria are satisfied, including a criterion that is satisfied when the first user input includes a respective type of input from a first portion of a user at a location that corresponds to the first portion of the virtual object, displaying, via the display generation component, a first visual feedback indicating a location of the attention of the user in the first portion of the virtual object; and

in accordance with a determination that one or more second criteria are satisfied, including a criterion that is satisfied when the first user input includes the attention of the user directed towards the first portion of the virtual object without the respective type of input from the user directed towards the first portion of the virtual object, displaying second visual feedback that is different from the first visual feedback without displaying the first visual feedback in the first portion of the virtual object.

31 . The non-transitory computer readable storage medium of claim 22 , the method further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a change in position of the first portion of the user relative to the virtual object; and

in response to detecting the change in position of the first portion of the user relative to the virtual object:

in accordance with a determination that the one or more first criteria were satisfied, changing a visual appearance of the first visual feedback by a first amount; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing changing a visual appearance of the second visual feedback by the first amount.

32 . The non-transitory computer readable storage medium of claim 31 , the method further comprising:

in response to detecting the change in position of the first portion of the user relative to the virtual object, and in accordance with the determination that the one or more second criteria were satisfied, forgoing changing the visual appearance of the second visual feedback based on a change in position of the first portion of the user relative to the virtual object.

33 . The non-transitory computer readable storage medium of claim 30 , the method further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input, selecting the selectable button, including:

in accordance with a determination that the one or more first criteria were satisfied when the second input was detected, moving the virtual object away from a viewpoint of the user in accordance with the second input; and

in accordance with a determination that the one or more second criteria were satisfied when the second input was detected, forgoing moving the virtual object away from the viewpoint of the user.

34 . The non-transitory computer readable storage medium of claim 30 , the method further comprising:

while the one or more first criteria or the one or more second criteria are satisfied, detecting, via the one or more input devices, a second input corresponding to selection of the virtual object, wherein the virtual object is a selectable button; and

in response to detecting the second input:

selecting the selectable button;

in accordance with a determination that the one or more first criteria were satisfied, displaying a visual indication around at least a portion of a perimeter of the virtual object indicating that the virtual object has been selected; and

in accordance with a determination that the one or more second criteria were satisfied, forgoing displaying the visual indication around the portion of the perimeter of the virtual object.

35 . The non-transitory computer readable storage medium of claim 30 , the method further comprising:

while displaying the second visual feedback at the first portion of the virtual object, detecting second user input that corresponds to movement of the first portion of the user toward the virtual object; and

in response to detecting the second user input, and in accordance with a determination that first portion of the user has moved within a threshold distance of the virtual object, ceasing display of the second visual feedback and displaying the first visual feedback in a respective portion of the virtual object.

36 . The non-transitory computer readable storage medium of claim 30 , wherein:

displaying the first visual feedback includes:

in accordance with a determination that the virtual object has a first size, displaying the first visual feedback having a first visual appearance; and

in accordance with a determination that the virtual object has a second size, displaying the first visual feedback having the first visual appearance; and

displaying the second visual feedback includes:

in accordance with a determination the virtual object has the first size, displaying the second visual feedback having a second visual appearance; and

in accordance with a determination that the virtual object has the second size, displaying the second visual feedback having a third visual appearance different from the second visual appearance.

37 . The non-transitory computer readable storage medium of claim 30 , the method further comprising:

while displaying, in a three-dimensional environment, a respective virtual object having a first size relative to the three-dimensional environment, detecting, via the one or more input devices, a second user input that includes attention directed towards a first portion of the respective virtual object; and

in response to detecting the second user input, in accordance with a determination that one or more third criteria are satisfied, displaying the respective virtual object having a second size, greater than the first size, relative to the three-dimensional environment.

Assignments (1)

Continuity (5)

References Cited (400)