IP Library › Granted Patent US 12,125,149
Granted Patent B2
US 12,125,149 · App. 17/667,350 · Granted Oct 22, 2024

Interfaces for presenting avatars in three-dimensional environments

Inventors: Rupert Thomas Guy Burton (San Francisco, CA); Gary Ian Butcher (Los Gatos, CA); Jason Rickwald (Santa Cruz, CA); Giancarlo Yerkes (San Carlos, CA); Kristi Elaine Schmidt Bauerly (Los Altos, CA); Dorian D. Dargan (Oakland, CA)
Assignee: Apple Inc.
G06T19/006G06T13/40G06V40/168G06V40/174
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,125,149
App. No.
17/667,350
Granted
Oct 22, 2024
Kind
B2
Abstract

In some embodiments, a computer system displays user interfaces for enrolling one or more features of a user of a computer system. In some embodiments, a computer system displays visual effects associated with a virtual avatar in an XR environment. In some embodiments, a computer system displays objects having different visual characteristics in an XR environment. In some embodiments, a computer system switches between different presentation modes associated with a user represented in an XR environment. In some embodiments, a computer system displays a virtual avatar in an XR environment.

Claims (93)

1. A computer system, wherein the computer system is in communication with a display generation component and one or more cameras, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

during an enrollment process that includes capturing facial data of a user via the one or more cameras, displaying, via the display generation component, an enrollment interface for enrolling one or more features of the user, including:

outputting a first facial expression prompt instructing the user to position one or more of the user's facial features in a first specific facial expression;

after outputting the first facial expression prompt, capturing a first set of facial data of the user via the one or more cameras;

in accordance with a determination that the first set of facial data of the user satisfies a first set of facial expression criteria, wherein the first set of facial expression criteria include a requirement that the first set of facial data corresponds to the first specific facial expression in order for the first set of facial expression criteria to be met, outputting confirmation that the first set of facial data of the user corresponds to the first specific facial expression;

outputting a second facial expression prompt instructing the user to position one or more of the user's facial features in a second specific facial expression different from the first specific facial expression;

after outputting the second facial expression prompt, capturing a second set of facial data of the user via the one or more cameras; and

in accordance with a determination that the second set of facial data of the user satisfies a second set of facial expression criteria, wherein the second set of facial expression criteria include a requirement that the second set of facial data corresponds to the second specific facial expression that is different from the first facial expression in order for the second set of facial expression criteria to be met, outputting confirmation that the second set of facial data of the user corresponds to the second specific facial expression.

2. The computer system of claim 1 , wherein:

the first facial expression prompt is outputted in accordance with a determination that a first set of enrollment criteria is not met, and

the second facial expression prompt is outputted in accordance with a determination that the first set of enrollment criteria is met and a second set of enrollment criteria is not met.

3. The computer system of claim 1 , the one or more programs further including instructions for:

after capturing the first set of facial data of the user via the one or more cameras, ceasing display of the first facial expression prompt; and

after capturing the second set of facial data of the user via the one or more cameras, ceasing display of the second facial expression prompt.

4. The computer system of claim 1 , wherein the first or second specific facial expression is selected from a group consisting of a smile, a frown, a squint, and a surprised expression.

5. The computer system of claim 1 , wherein the second facial expression prompt includes a prompt for the user to speak a set of one or more words.

6. The computer system of claim 1 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a third prompt to change a position of the user's head.

7. The computer system of claim 6 , wherein the third prompt is outputted before at least one of the first facial expression prompt or the second facial expression prompt.

8. The computer system of claim 1 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fourth prompt to change a position of the one or more cameras relative to the user's head while keeping the user's head stationary.

9. The computer system of claim 1 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fifth prompt to indicate a height of the user.

10. The computer system of claim 1 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a sixth prompt to remove a set of glasses from the user's face for at least a portion of the enrollment process.

11. The computer system of claim 1 , wherein:

an avatar is generated using at least a portion of the facial data captured during the enrollment process, and

the avatar is displayed using an external computer system that is different from the computer system.

12. The computer system of claim 1 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a seventh prompt to capture a pose of a non-facial feature of the user.

13. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component and one or more cameras, the one or more programs including instructions for:

during an enrollment process that includes capturing facial data of a user via the one or more cameras, displaying, via the display generation component, an enrollment interface for enrolling one or more features of the user, including:

outputting a first facial expression prompt instructing the user to position one or more of the user's facial features in a first specific facial expression;

after outputting the first facial expression prompt, capturing a first set of facial data of the user via the one or more cameras;

in accordance with a determination that the first set of facial data of the user satisfies a first set of facial expression criteria, wherein the first set of facial expression criteria include a requirement that the first set of facial data corresponds to the first specific facial expression in order for the first set of facial expression criteria to be met, outputting confirmation that the first set of facial data of the user corresponds to the first specific facial expression;

outputting a second facial expression prompt instructing the user to position one or more of the user's facial features in a second specific facial expression different from the first specific facial expression;

after outputting the second facial expression prompt, capturing a second set of facial data of the user via the one or more cameras; and

in accordance with a determination that the second set of facial data of the user satisfies a second set of facial expression criteria, wherein the second set of facial expression criteria include a requirement that the second set of facial data corresponds to the second specific facial expression that is different from the first facial expression in order for the second set of facial expression criteria to be met, outputting confirmation that the second set of facial data of the user corresponds to the second specific facial expression.

14. The non-transitory computer-readable storage medium of claim 13 , wherein:

the first facial expression prompt is outputted in accordance with a determination that a first set of enrollment criteria is not met, and

the second facial expression prompt is outputted in accordance with a determination that the first set of enrollment criteria is met and a second set of enrollment criteria is not met.

15. The non-transitory computer-readable storage medium of claim 13 , the one or more programs further including instructions for:

after capturing the first set of facial data of the user via the one or more cameras, ceasing display of the first facial expression prompt; and

after capturing the second set of facial data of the user via the one or more cameras, ceasing display of the second facial expression prompt.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the first or second specific facial expression is selected from a group consisting of a smile, a frown, a squint, and a surprised expression.

17. The non-transitory computer-readable storage medium of claim 13 , wherein the second facial expression prompt includes a prompt for the user to speak a set of one or more words.

18. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a third prompt to change a position of the user's head.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the third prompt is outputted before at least one of the first facial expression prompt or the second facial expression prompt.

20. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fourth prompt to change a position of the one or more cameras relative to the user's head while keeping the user's head stationary.

21. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fifth prompt to indicate a height of the user.

22. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a sixth prompt to remove a set of glasses from the user's face for at least a portion of the enrollment process.

23. The non-transitory computer-readable storage medium of claim 13 , wherein:

an avatar is generated using at least a portion of the facial data captured during the enrollment process, and

the avatar is displayed using an external computer system that is different from the computer system.

24. The non-transitory computer-readable storage medium of claim 13 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a seventh prompt to capture a pose of a non-facial feature of the user.

25. A method comprising:

at a computer system that is in communication with a display generation component and one or more cameras:

during an enrollment process that includes capturing facial data of a user via the one or more cameras, displaying, via the display generation component, an enrollment interface for enrolling one or more features of the user, including:

outputting a first facial expression prompt instructing the user to position one or more of the user's facial features in a first specific facial expression;

after outputting the first facial expression prompt, capturing a first set of facial data of the user via the one or more cameras;

in accordance with a determination that the first set of facial data of the user satisfies a first set of facial expression criteria, wherein the first set of facial expression criteria include a requirement that the first set of facial data corresponds to the first specific facial expression in order for the first set of facial expression criteria to be met, outputting confirmation that the first set of facial data of the user corresponds to the first specific facial expression;

outputting a second facial expression prompt instructing the user to position one or more of the user's facial features in a second specific facial expression different from the first specific facial expression;

after outputting the second facial expression prompt, capturing a second set of facial data of the user via the one or more cameras; and

in accordance with a determination that the second set of facial data of the user satisfies a second set of facial expression criteria, wherein the second set of facial expression criteria include a requirement that the second set of facial data corresponds to the second specific facial expression that is different from the first facial expression in order for the second set of facial expression criteria to be met, outputting confirmation that the second set of facial data of the user corresponds to the second specific facial expression.

26. The method of claim 25 , wherein:

the first facial expression prompt is outputted in accordance with a determination that a first set of enrollment criteria is not met, and

the second facial expression prompt is outputted in accordance with a determination that the first set of enrollment criteria is met and a second set of enrollment criteria is not met.

27. The method of claim 25 , further comprising:

after capturing the first set of facial data of the user via the one or more cameras, ceasing display of the first facial expression prompt; and

after capturing the second set of facial data of the user via the one or more cameras, ceasing display of the second facial expression prompt.

28. The method of claim 25 , wherein the first or second specific facial expression is selected from a group consisting of a smile, a frown, a squint, and a surprised expression.

29. The method of claim 25 , wherein the second facial expression prompt includes a prompt for the user to speak a set of one or more words.

30. The method of claim 25 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a third prompt to change a position of the user's head.

31. The method of claim 30 , wherein the third prompt is outputted before at least one of the first facial expression prompt or the second facial expression prompt.

32. The method of claim 25 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fourth prompt to change a position of the one or more cameras relative to the user's head while keeping the user's head stationary.

33. The method of claim 25 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a fifth prompt to indicate a height of the user.

34. The method of claim 25 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a sixth prompt to remove a set of glasses from the user's face for at least a portion of the enrollment process.

35. The method of claim 25 , wherein:

an avatar is generated using at least a portion of the facial data captured during the enrollment process, and

the avatar is displayed using an external computer system that is different from the computer system.

36. The method of claim 25 , wherein displaying the enrollment interface for enrolling one or more features of the user further includes:

outputting a seventh prompt to capture a pose of a non-facial feature of the user.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2023
From: DARGAN, DORIAN D.
To: APPLE INC.
Reel/Frame 064594/0264 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2022
From: BURTON, RUPERT THOMAS GUY; LEMAY, STEPHEN O.; RICKWALD, JASON; SORRENTINO, WILLIAM A., III; YERKES, GIANCARLO; BAUERLY, KRISTI ELAINE SCHMIDT; BUTCHER, GARY IAN
To: APPLE INC.
Reel/Frame 061352/0157 →
Continuity (2)
Provisional Application 63149989 · Feb 16, 2021
Related Publication 20220262080A1 · Aug 18, 2022
Cited By (2)
US 12,455,949 US 12,699,461