Low-bandwidth avatar animation
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating avatars based on physical sensor measurements. One of the methods includes receiving, by a second user device from a video streaming platform system during a video streaming session between a first user device and the second user device, an updated physical sensor measurement of the first user device. An updated graphical representation of an avatar selected by a first user of the first user device is generated by the second user device based on the updated physical sensor measurement of the first user device. The updated graphical representation of the avatar is presented by the second user device on a display device of the second user device during a video streaming session between the first user device and the second user device.
1. A system comprising:
a first user device comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
generating a representation of an updated physical sensor measurement, wherein the representation comprises one or more values that represent an updated orientation of the first user device;
obtaining, by the first user device, a previously recorded orientation of the first user device;
computing, by the first user device, a difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
providing, by the first user device to a social messaging platform, message content and data representing the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
generating, by the social messaging platform based on the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device, an updated animation of an avatar for the first user device;
generating, by the social messaging platform, a message for display to other users of the social messaging platform, the message including the message content received from the first user device and the updated animation of the avatar generated from the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device; and
providing, to a second user device by the social messaging platform, the message including the updated animation of the avatar for display within the message on a display of the second user device.
2. The system of claim 1 , wherein:
the message includes image content; and
the updated animation of the avatar represents a first user's reaction to the image content, the first user corresponding to the first user device.
3. The system of claim 2 , wherein:
the image content comprises image content captured by the first user device; and
the representation of the updated physical sensor measurement comprises one or more values that represent an updated orientation of the first user device at a time when the image content was captured by the first user device.
4. The system of claim 1 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
receiving, by the first user device, an audio level of the first user device measured by one or more microphones of the first user device;
computing a degree of mouth openness for a mouth of the avatar based on the audio level; and
generating the mouth of the avatar based on the computed degree of mouth openness.
5. The system of claim 1 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation and the previously recorded orientation, that the first user device has moved more than a threshold number of degrees; and
responsive to determining that the first user device has moved more than a threshold number of degrees, generating the updated animation of the avatar.
6. The system of claim 1 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation, the previously recorded orientation, and a time associated with the previously recorded orientation, an orientation velocity; and
generating the updated animation of the avatar based on the orientation velocity.
7. The system of claim 1 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
animating the avatar to transition from a first animation of the avatar to a second animation of the avatar, the first animation of the avatar corresponding to the previously recorded orientation and the second animation of the avatar corresponding to the updated orientation.
8. A method comprising:
generating a representation of an updated physical sensor measurement, wherein the representation comprises one or more values that represent an updated orientation of a first user device;
obtaining, by the first user device, a previously recorded orientation of the first user device;
computing, by the first user device, a difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
providing, by the first user device to a social messaging platform, message content and data representing the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
generating, by the social messaging platform based on the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device, an updated animation of an avatar for the first user device;
generating, by the social messaging platform, a message for display to other users of the social messaging platform, the message including the message content received from the first user device and the updated animation of the avatar generated from the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device; and
providing, to a second user device by the social messaging platform, the message including the updated animation of the avatar for display within the message on a display of the second user device.
9. The method of claim 8 , wherein:
the message includes image content; and
the updated animation of the avatar represents a first user's reaction to the image content, the first user corresponding to the first user device.
10. The method of claim 9 , wherein:
the image content comprises image content captured by the first user device; and
the representation of the updated physical sensor measurement comprises one or more values that represent an updated orientation of the first user device at a time when the image content was captured by the first user device.
11. The method of claim 8 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
receiving, by the first user device, an audio level of the first user device measured by one or more microphones of the first user device;
computing a degree of mouth openness for a mouth of the avatar based on the audio level; and
generating the mouth of the avatar based on the computed degree of mouth openness.
12. The method of claim 8 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation and the previously recorded orientation, that the first user device has moved more than a threshold number of degrees; and
responsive to determining that the first user device has moved more than a threshold number of degrees, generating the updated animation of the avatar.
13. The method of claim 8 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation, the previously recorded orientation, and a time associated with the previously recorded orientation, an orientation velocity; and
generating the updated animation of the avatar based on the orientation velocity.
14. The method of claim 8 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
animating the avatar to transition from a first animation of the avatar to a second animation of the avatar, the first animation of the avatar corresponding to the previously recorded orientation and the second animation of the avatar corresponding to the updated orientation.
15. A non-transitory computer-readable storage medium comprising instructions that when executed cause a system of one or more computers to perform operations comprising:
generating a representation of an updated physical sensor measurement, wherein the representation comprises one or more values that represent an updated orientation of a first user device;
obtaining, by the first user device, a previously recorded orientation of the first user device;
computing, by the first user device, a difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
providing, by the first user device to a social messaging platform, message content and data representing the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device;
generating, by the social messaging platform based on the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device, an updated animation of an avatar for the first user device;
generating, by the social messaging platform, a message for display to other users of the social messaging platform, the message containing the message content received from the first user device and the updated animation of the avatar generated from the computed difference between the updated orientation of the first user device and the previously recorded orientation of the first user device; and
providing, to a second user device by the social messaging platform, the message including the updated animation of the avatar for display within the message on a display of the second user device.
16. The non-transitory computer-readable storage medium of claim 15 , wherein:
the message includes image content; and
the updated animation of the avatar represents a first user's reaction to the image content, the first user corresponding to the first user device.
17. The non-transitory computer-readable storage medium of claim 16 , wherein:
the image content comprises image content captured by the first user device; and
the representation of the updated physical sensor measurement comprises one or more values that represent an updated orientation of the first user device at a time when the image content was captured by the first user device.
18. The non-transitory computer-readable storage medium of claim 15 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
receiving, by the first user device, an audio level of the first user device measured by one or more microphones of the first user device;
computing a degree of mouth openness for a mouth of the avatar based on the audio level; and
generating the mouth of the avatar based on the computed degree of mouth openness.
19. The non-transitory computer-readable storage medium of claim 15 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation and the previously recorded orientation, that the first user device has moved more than a threshold number of degrees; and
responsive to determining that the first user device has moved more than a threshold number of degrees, generating the updated animation of the avatar.
20. The non-transitory computer-readable storage medium of claim 15 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
determining, based on the updated orientation, the previously recorded orientation, and a time associated with the previously recorded orientation, an orientation velocity; and
generating the updated animation of the avatar based on the orientation velocity.
21. The non-transitory computer-readable storage medium of claim 15 , wherein generating, by the social messaging platform, the updated animation of the avatar comprises:
animating the avatar to transition from a first animation of the avatar to a second animation of the avatar, the first animation of the avatar corresponding to the previously recorded orientation and the second animation of the avatar corresponding to the updated orientation.