Granted Patent B1

US 12,249,014 · App. 17/877,595 · Granted Mar 11, 2025

Inventors: Mokhtar Mohamed Khorshid (Kirkland, WA); Trang Tran (San Leandro, CA); Beau Dunn (Lake Forest, CA); Erica Feld (Hoboken, NJ)

Assignee: META PLATFORMS, INC.

G06T13/40G06T15/005G06T19/006

Abstract

Claims (48)

1. A method comprising, by one or more computing systems:

rendering, for one or more displays of a first extended-reality (XR) display device, a first output image of an XR assistant avatar within a first environment associated with a first XR application, wherein the XR assistant avatar is rendered for the one or more displays of the first XR display device to have a first form according to a first rendering specification associated with the first XR application, and wherein the XR assistant avatar is interactable by a first user of the first XR display device to access an assistant system;

receiving an indication that the first user is switching from the first XR application to a second XR application;

accessing a second rendering specification associated with the second XR application based on the indication; and

rendering, for the one or more displays of a second XR display device, a second output image of the XR assistant avatar within a second environment associated with the second XR application, wherein the XR assistant avatar is rendered for the one or more displays of the second XR display device to have a second form according to the second rendering specification associated with the second XR application, wherein the second form is less transparent than the first form.

2. The method of claim 1 , wherein the one or more computing systems comprise the first or second XR display devices,

wherein the indication includes that the first user interacts with the XR assistant avatar,

further wherein in the first form the XR assistant avatar is semi-transparent, and

further wherein in the second form the XR assistant avatar is opaque.

3. The method of claim 1 , wherein the one or more computing systems comprise the assistant system.

4. The method of claim 1 , wherein the first or second environment is a real-world environment, and wherein the XR assistant avatar is an augmented-reality (AR) rendering.

5. The method of claim 1 , wherein the first or second environment is a virtual-reality (VR) environment, and wherein the XR assistant avatar is a VR rendering.

6. The method of claim 1 , wherein the first or second form of the XR assistant avatar is based on one or more of its voice, speech, emotion, tone, pitch, appearance, size, shape, clothing, orientation, position, depth, movement, gesture, facial expression, color, shading, outline, brightness, or luminescence.

7. The method of claim 1 , further comprising:

determining the second form based on a personalization model, wherein the personalization model is a machine-learning model trained on a plurality of prior reactions associated with the first user with respect to a plurality of morphing of the XR assistant avatar.

8. The method of claim 1 , further comprising:

detecting one or more user behaviors associated with the first user;

determining, based on the one or more user behaviors, one or more user sentiments associated with the first user; and

determining the second form based on the one or more user sentiments.

9. The method of claim 1 , wherein the first or second rendering specification is specific to the first or second XR display device.

10. The method of claim 1 , wherein the XR assistant avatar further has a first pose according to the first rendering specification associated with the first XR application, and wherein the XR assistant avatar further has a second pose according to the second rendering specification associated with the second XR application.

11. The method of claim 1 , further comprising, responsive to receiving the indication that the first user is switching from the first XR application to the second XR application:

registering a first set of actions with the XR assistant avatar based on the switching from the first XR application to the second XR application, wherein each action in the first set of actions corresponds to a first voice-initiated action executable via the assistant system; and

unregistering a second set of actions with the XR assistant avatar based on the switching from the first XR application to the second XR application, wherein each action in the second set of actions corresponds to a second voice-initiated action executable via the assistant system.

12. The method of claim 1 , wherein rendering the first or second output image of the XR assistant avatar comprising rendering the XR assistant avatar as a human-like avatar.

13. The method of claim 1 , wherein rendering the first or second output image of the XR assistant avatar comprising rendering the XR assistant avatar as an animated object or icon.

14. The method of claim 1 , wherein the first or second rendering specification is based on instructions specified in a software development kit associated with the assistant system, and wherein the first or second rendering specification comprises one or more of an executable command for rendering the XR assistant avatar, an executable script for rendering the XR assistant avatar, a context for rendering the XR assistant avatar, a scene for rendering the XR assistant avatar, a perspective projection for rendering the XR assistant avatar, an object-attribute for rendering the XR assistant avatar, or a field of view for rendering the XR assistant avatar.

15. The method of claim 1 , further comprising:

generating, based on context associated with the first user, one or more proactive suggestions; and

providing, via the XR assistant avatar by the assistant system, one or more of the proactive suggestions to the first user, wherein the provided proactive suggestions are associated with the first or second output image of the XR assistant avatar.

16. The method of claim 1 , further comprising:

determining one or more natural-language responses for the XR assistant avatar associated with the rendering of the second output image of the XR assistant avatar, wherein the one or more natural-language responses are customized for the first user based on the second XR application.

17. The method of claim 1 , wherein the first XR display device and the second XR display are a same device, and wherein the first XR application and the second XR application are different applications.

18. The method of claim 1 , further comprising:

receiving an indication that the first user is switching from the first XR display device to the second XR display device, wherein the first XR display device and the second XR display device are different devices.

19. The method of claim 1 , wherein the first XR display device and the second XR display device are different devices, and wherein the first XR application and the second XR application are different applications.

20. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

render, for one or more displays of a first extended-reality (XR) display device, a first output image of an XR assistant avatar within a first environment associated with a first XR application, wherein the XR assistant avatar is rendered for the one or more displays of the first XR display device to have a first form according to a first rendering specification associated with the first XR application, and wherein the XR assistant avatar is interactable by a first user of the first XR display device to access an assistant system;

receive an indication that the first user is switching from the first XR application to a second XR application;

access a second rendering specification associated with the second XR application based on the indication; and

render, for the one or more displays of a second XR display device, a second output image of the XR assistant avatar within a second environment associated with the second XR application, wherein the XR assistant avatar is rendered for the one or more displays of the second XR display device to have a second form according to the second rendering specification associated with the second XR application, wherein the second form is less transparent than the first form.

21. A system comprising:

one or more processors; and

a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

render, for one or more displays of a first extended-reality (XR) display device, a first output image of an XR assistant avatar within a first environment associated with a first XR application, wherein the XR assistant avatar is rendered for the one or more displays of the first XR display device to have a first form according to a first rendering specification associated with the first XR application, and wherein the XR assistant avatar is interactable by a first user of the first XR display device to access an assistant system;

receive an indication that the first user is switching from the first XR application to a second XR application;

access a second rendering specification associated with the second XR application based on the indication; and

render, for the one or more displays of a second XR display device, a second output image of the XR assistant avatar within a second environment associated with the second XR application, wherein the XR assistant avatar is rendered for the one or more displays of the second XR display device to have a second form according to the second rendering specification associated with the second XR application, wherein the second form is less transparent than the first form.

Assignments (1)

References Cited (221)

Cited By (2)

US 12,620,168 US 12,657,401