Avatar creation from natural language description
In one embodiment, a method includes accessing, by a computing device, a natural-language input comprising a description of an avatar and generating, from the natural-language input and by a trained avatar-creation model, a set of avatar-attribute feature vectors. The method further includes determining, by a trained avatar-attribute classifier, one or more avatar attributes from the set of avatar-attribute feature vectors; and generating the avatar based on the determined one or more avatar attributes for presentation on a display of a computing device.
1 . A method comprising:
accessing, by a computing device, a natural-language input comprising a description of an avatar;
determining whether the accessed natural-language input identifies a required attribute for the avatar, and either:
in response to a determination that the accessed natural-language input does not identify the required attribute, then (1) submitting a request for a user to identify the required attribute for the avatar (2) receiving a user input identifying the required attribute and (3) providing the natural language input to a trained avatar-creation model; or
in response to a determination that the accessed natural-language input identifies the required attribute, then providing the natural-language input to a trained avatar-creation model;
generating, from the natural-language input and by the trained avatar-creation model, a set of avatar-attribute feature vectors;
determining, by a trained avatar-attribute classifier, one or more avatar attributes from the set of avatar-attribute feature vectors; and
generating the avatar based on the determined one or more avatar attributes for presentation on a display of a computing device.
2 . The method of claim 1 , further comprising converting the accessed natural-language input from a speech input to a text input.
3 . The method of claim 1 , further comprising converting the accessed natural-language input from a first language to a default language.
4 . The method of claim 1 , wherein the user input identifying the required attribute explicitly identifies the required attribute, and the user input explicitly identifying the required attribute is included in the natural-language input provided to the trained avatar-creation model.
5 . The method of claim 1 , wherein the required attribute comprises a gender of the avatar.
6 . The method of claim 1 , wherein the trained avatar creation model comprises an encoder of a transformer model.
7 . The method of claim 1 , further comprising:
determining, by a scenario detection model of the avatar-creation model, whether the natural-language input identifies one or more scenarios; and
in response to a determination that the natural-language input identifies one or more scenarios, then determining one or more avatar attributes corresponding to the identified one or more scenarios and generating the avatar based at least in part on the one or more avatar attributes corresponding to the identified one or more scenarios.
8 . The method of claim 7 , further comprising weighting the one or more avatar attributes corresponding to the identified one or more scenarios relatively higher than the one or more avatar attributes determined from the set of avatar-attribute feature vectors.
9 . The method of claim 1 , further comprising:
accessing user feedback made in response to display of the avatar; and
combining a description of the user feedback with the accessed natural-language input to generate one or more revised avatar attributes and a revised avatar for presentation on the display of the computing device.
10 . The method of claim 1 , wherein the trained avatar-attribute classifier comprises a classifier trained by:
selecting a set of predefined attributes;
generating, by a text-generation language model, a training dataset comprising one or more sentences, each sentence containing at least one attribute from the set of predefined attributes; and
training the classifier based on the training dataset and the set of predefined attributes.
11 . One or more non-transitory computer readable storage media storing software that is operable when executed by one or more processors to:
access a natural-language input comprising a description of an avatar;
determine whether the accessed natural-language input identifies a required attribute for the avatar, and:
in response to a determination that the accessed natural-language input does not identify the required attribute, then (1) submit a request for a user to identify the required attribute for the avatar (2) receive a user input identifying the required attribute and (3) provide the natural language input to a trained avatar-creation model; or
in response to a determination that the accessed natural-language input identifies the required attribute, then provide the natural-language input to a trained avatar-creation model;
generate from the natural-language input and by a trained avatar-creation model, a set of avatar-attribute feature vectors;
determine, by a trained avatar-attribute classifier, one or more avatar attributes from the set of avatar-attribute feature vectors; and
generate the avatar based on the determined one or more avatar attributes for presentation on a display of a computing device.
12 . The media of claim 11 , wherein the user input identifying the required attribute explicitly identifies the required attribute, and the user input explicitly identifying the required attribute is included in the natural-language input provided to the trained avatar-creation model.
13 . The media of claim 11 , wherein the software is further operable when executed by one or more processors to:
determine, by a scenario detection model of the avatar-creation model, whether the natural-language input identifies one or more scenarios; and
in response to a determination that the natural-language input identifies one or more scenarios, then determine one or more avatar attributes corresponding to the identified one or more scenarios and generate the avatar based at least in part on the one or more avatar attributes corresponding to the identified one or more scenarios.
14 . The media of claim 11 , wherein the software is further operable when executed by one or more processors to:
access user feedback made in response to display of the avatar; and
combine a description of the user feedback with the accessed natural-language input to generate one or more revised avatar attributes and a revised avatar for presentation on the display of the computing device.
15 . The media of claim 11 , wherein the trained avatar-attribute classifier comprises a classifier trained by:
selecting a set of predefined attributes;
generating, by a text-generation language model, a training dataset comprising one or more sentences, each sentence containing at least one attribute from the set of predefined attributes; and
training the classifier based on the training dataset and the set of predefined attributes.
16 . A system comprising one or more non-transitory computer readable storage media storing instructions; and one or more processors coupled to the non-transitory computer readable storage media, the one or more processors operable to execute the instructions to:
access a natural-language input comprising a description of an avatar;
determine whether the accessed natural-language input identifies a required attribute for the avatar, and:
in response to a determination that the accessed natural-language input does not identify the required attribute, then (1) submit a request for a user to identify the required attribute for the avatar (2) receive a user input identifying the required attribute and (3) provide the natural language input to a trained avatar-creation model; or
in response to a determination that the accessed natural-language input identifies the required attribute, then provide the natural-language input to a trained avatar-creation model;
generate from the natural-language input and by a trained avatar-creation model, a set of avatar-attribute feature vectors;
determine, by a trained avatar-attribute classifier, one or more avatar attributes from the set of avatar-attribute feature vectors; and
generate the avatar based at least in part on the determined one or more avatar attributes for presentation on a display of a computing device.
17 . The system of claim 16 , wherein the user input identifying the required attribute explicitly identifies the required attribute, and the user input explicitly identifying the required attribute is included in the natural-language input provided to the trained avatar-creation model.
18 . The system of claim 16 , wherein the one or more processors are further operable to execute the instructions to:
determine, by a scenario detection model of the avatar-creation model, whether the natural-language input identifies one or more scenarios; and
in response to a determination that the natural-language input identifies one or more scenarios, then determine one or more avatar attributes corresponding to the identified one or more scenarios and generate the avatar based on the one or more avatar attributes corresponding to the identified one or more scenarios.
19 . The system of claim 16 , wherein the one or more processors are further operable to execute the instructions to:
access user feedback made in response to display of the avatar; and
combine a description of the user feedback with the accessed natural-language input to generate one or more revised avatar attributes and a revised avatar for presentation on the display of the computing device.
20 . The system of claim 16 , wherein the trained avatar-attribute classifier comprises a classifier trained by:
selecting a set of predefined attributes;
generating, by a text-generation language model, a training dataset comprising one or more sentences, each sentence containing at least one attribute from the set of predefined attributes; and
training the classifier based on the training dataset and the set of predefined attributes.