Electronic messaging utilizing animatable 3D models

1. A method for creating a customized animatable 3D model for use in an electronic communication between at least two users, the method comprising:

receiving, via a camera associated with a mobile device, an input from a first user, the first user using the mobile device, the input being in the form of at least one of an audio stream and a visual stream, the visual stream including at least one image or video;

decomposing the input into a plurality of facial expressions of the first user;

selecting, using one or more control point matrices and a template geometry of an animatable 3D model and based on the plurality of facial expressions, one or more base facial expressions from a set of base expressions, the one or more base facial expressions mimicking the plurality of facial expressions of the first user;

based on the animatable 3D model and the input, automatically generating a dynamically customized animation of the animatable 3D model of a virtual character corresponding to the first user, the generating of the dynamically customized animation comprising:

deforming, based on the one or more base facial expressions, the template geometry into a custom geometry, the custom geometry having the one or more base facial expressions; and

performing dynamic conversion of the input, into an expression stream and corresponding time information, the expression stream including the one or more base facial expressions associated with the custom geometry;

generating a selectable link for transmission as part of an electronic message, the selectable link linking to the expression stream and the corresponding time information; and

causing display, to a second user, of the dynamically customized animation of the animatable 3D model of the virtual character corresponding to the first user.

2. The method of claim 1 , wherein the electronic message is an instant message comprising a text portion in addition to the selectable link.

3. The method of claim 1 , wherein the dynamic conversion includes performing normalization of lighting and illumination for at least some of the at least one image or video in the visual stream.

4. The method of claim 1 , wherein active movements derived from the input, in the form of the at least one of the audio stream and the visual stream, are used for the dynamic conversion into the expression stream and the corresponding time information.

5. The method of claim 1 , wherein, for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion, of the input from the first user into the expression stream and the corresponding time information, includes determining the one or more base facial expressions corresponding to the visual stream.

6. The method of claim 1 , wherein the generating of the dynamically customized animation comprising the performing dynamic conversion of the input is performed frame by frame.

7. The method of claim 5 , wherein, for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion further comprising dynamically generating the expression stream at least in part by dynamically determining combinations of two or more of the base expressions from at least one of the audio stream and the visual stream.

8. The method of claim 5 , wherein for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion further comprises dynamically generating the expression stream at least in part by dynamically determining combinations of three base expressions from the visual stream.

9. The method of claim 8 , wherein for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion further comprises determining certain movements to apply to the animatable 3D model based on a viseme determined based on the audio stream.

10. The method of claim 8 , wherein, for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion further comprises determining certain movements to apply to the animatable 3D model based on determining a direction, from the visual stream, at least one eye of the first user is looking.

11. The method of claim 8 , wherein, for the generating of the dynamically customized animation comprising the performing dynamic conversion of the input, the dynamic conversion further comprises determining motion cues based on idle behavior of the first user based on the visual stream.

12. The method of claim 1 , wherein the dynamically converting further comprises determining at least one microexpression, from a frame of the visual stream, comprising determining weights to changes detected in the frame of the visual stream from at least a prior frame of the visual stream, in the face of the first user.

13. The method of claim 1 , further comprising receiving the animatable 3D model of the virtual character corresponding to the first user, the animatable 3D model being customizable such that the customized animatable 3D model can be generated therefrom.

14. The method of claim 1 , wherein the animatable 3D model is customizable such that the customized animatable 3D model can be generated therefrom, and the animatable 3D model is:

downloaded, for customization processing, from a cloud-based system to the mobile device of the second user.

15. The method of claim 1 , wherein the animatable 3D model is customizable such that the customized animatable 3D model can be generated therefrom, and the animatable 3 D model is:

customized on a cloud-based system such that the animatable 3D model need not be downloaded or transmitted to the mobile device of the second user.

16. The method of claim 1 , wherein the input is further received via a microphone of the mobile device, the input causing generation of the visual stream and the audio stream, respectively, on the mobile device.

17. The method of claim 1 , wherein the visual stream comprises a video comprising a plurality of captured images.

18. The method of claim 1 , wherein a frame of the visual stream comprises a photo.

19. The method of claim 1 , wherein the input is in the form of descriptive text, the descriptive text being used for generation of the at least one of the audio stream and the visual stream.

20. The method of claim 1 , wherein the generating the selectable link is performed automatically or in response to an action from the first user.

21. The method of claim 1 , wherein the animatable 3D model is one of a plurality of animatable models that is created for the first user; the first user selecting which of the plurality of animatable models to be customized based on user input during the electronic messaging.

22. The method of claim 1 , wherein the 3D animatable model or the customized 3D animatable model making certain expressions is mapped to a keyboard so as to be selectable by the first user.

23. The method of claim 1 , wherein the causing display is performed automatically or in response to an action from the second user.

24. A system for creating a customized animatable 3D model for use in an electronic message, the system comprising:

a processor; and

a memory for storing executable instructions, the processor executing the instructions to:

receive, via a camera associated with a mobile device, an input from a first user, the input being in the form of at least one of an audio stream and a visual stream;

decompose the input into a plurality of facial expressions of the first user;

select, using one or more control point matrices and a template geometry of an animatable 3D model and based on the plurality of facial expressions, one or more base facial expressions from a set of base expressions, the one or more base facial expressions mimicking the plurality of facial expressions of the first user;

based on the animatable 3D model and the input, automatically generate a dynamically customized animation of the animatable 3D model of a virtual character corresponding to the first user, the generating comprising:

deforming, based on the one or more base facial expressions, the template geometry into a custom geometry, the custom geometry having the one or more base facial expressions; and

performing dynamic conversion of the input into an expression stream and corresponding time information, the expression stream including the one or more base facial expressions associated with the custom geometry;

generate a selectable link for transmission as part of an electronic message, the selectable link linking to the expression stream and the corresponding time information; and

cause display of the dynamically customized animatable 3D model to a second user.