IP Library Granted Patent US 9,747,495
Granted Patent B2
US 9,747,495 · App. 13/787,541 · Granted Aug 29, 2017

Systems and methods for creating and distributing modifiable animated video messages

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,747,495
App. No.
13/787,541
Granted
Aug 29, 2017
Kind
B2
Abstract

Systems and methods in accordance with embodiments of the invention enable collaborative creation, transmission, sharing, non-linear exploration, and modification of animated video messages. One embodiment includes a video camera, a processor, a network interface, and storage containing an animated message application, and a 3D character model. In addition, the animated message application configures the processor to: capture a video sequence using the video camera; detect a human face within a sequence of video frames; track changes in human facial expression of a human face detected within a sequence of video frames; map tracked changes in human facial expression to motion data, where the motion data is generated to animate the 3D character model; apply motion data to animate the 3D character model; render an animation of the 3D character model into a file as encoded video; and transmit the encoded video to a remote device via the network interface.

Claims (63)

1. A method for generating animated video messages comprising:

generating groupings of animation sequences, wherein:

each grouping of animation sequences corresponds to an action in an animated sequence; and

each animation within a grouping of animation sequences corresponds to an alternative way to perform the action corresponding to that grouping; receiving a user selection of an animation from each of the groupings of animation sequences; and

creating an interactive animated video message using the user selected animations.

2. The method as recited in claim 1 , wherein creating the interactive animated video message using the user selected animations comprises:

generating motion data based on the user selected animations; and

applying the motion data to a 3 D character model to cause the 3 D character model to perform the user selected animations.

3. The method as recited in claim 2 , further comprising arranging the user selected animations based on an order of the groupings of animation sequences.

4. The method as recited in claim 2 , further comprising randomizing an order of the user selected animations before generating the motion data.

5. The method as recited in claim 2 , further comprising extracting an animation sequence from the interactive animated video message for reuse in another animated video message.

6. The method as recited in claim 2 , further comprising transmitting the motion data to a remote device via a network interface.

7. The method as recited in claim 1 , further comprising obtaining an animation within a grouping of animation sequences corresponding to an action by:

receiving a sequence of video frames;

detecting a human face within the sequence of the video frames;

tracking, within the sequence of video frames, a facial expression change in the detected human face;

mapping the tracked facial expression changes to 3 D motion data; and

applying the mapped 3 D motion data to a 3 D character model to cause the 3 D character model to perform the animation.

8. The method as recited in claim 7 , further comprising matching the detected facial expressions to a stored facial expression model from a database of stored facial expressions.

9. The method as recited in claim 8 , further comprising obtaining, based on matching the detected facial expressions to the stored facial expression model, the mapped 3 D motion data for the stored facial expression model.

10. The method as recited in claim 9 , wherein the 3 D character model comprises data that describes an articulated skeleton, at least one mesh, and skinning weights that indicate how the articulated skeleton deforms the at least one mesh for a given motion.

11. The method as recited in claim 10 , wherein the articulated skeleton comprises a plurality of bones configured to rotate in one or more degrees of freedom around joints, and wherein the at least one mesh comprises a plurality of vertices defining a surface having a plurality of polygons.

12. The method as recited in claim 1 , further comprising obtaining a description of a space of human faces and principal component analysis facial expressions using a training data set containing multiple images of human faces registered to a template image of a human face and multiple images of human facial expressions registered to the same template image of the human face.

13. A system for generating animated video messages, the system comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:

generate groupings of animation sequences, wherein:

each grouping of animation sequences corresponds to an action in an animated sequence; and

each animation within a grouping of animation sequences corresponds to an alternative way to perform action corresponding to that grouping; receiving a user selection of an animation from each of the groupings of animation sequences;

create an interactive animated video message using the user selected animations; and

transmit the interactive animated video message to a remote device via a network interface.

14. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to extract an animation sequence from the interactive animated video message for reuse in another animated video message.

15. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to:

generate motion data based on the interactive animated video message; and

apply the motion data to a 3 D character model to cause the 3 D character model to perform the actions of the interactive animated video message.

16. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to playback the interactive animated video message via a display device.

17. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain an animation within a grouping of animation sequences corresponding to an action is by:

receiving a sequence of video frames;

detecting a human face within the sequence of the video frames;

tracking, within the sequence of video frames, a facial expression change in the detected human face;

mapping the tracked facial expression changes to 3 D motion data; and

applying the mapped 3 D motion data to a 3 D character model to cause the 3 D character model to the animation.

18. The system as recited in claim 17 , further comprising instructions that, when executed by the at least one processor, cause the system to match the detected facial expressions to a stored facial expression model from a database of stored facial expressions.

19. The system as recited in claim 18 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain, based on matching the detected facial expressions to the stored facial expression model, the mapped 3 D motion data for the stored facial expression model.

20. The system as recited in claim 19 , wherein the 3 D character model comprises data that describes an articulated skeleton, at least one mesh, and skinning weights that indicate how the articulated skeleton deforms the at least one mesh for a given motion.

21. The system as recited in claim 20 , wherein the articulated skeleton comprises a plurality of bones configured to rotate in one or more degrees of freedom around joints, and wherein the at least one mesh comprises a plurality of vertices defining a surface having a plurality of polygons.

22. The system as recited in claim 13 , further comprising instructions that, when executed by the at least one processor, cause the system to obtain a description of a space of human faces and principal component analysis facial expressions using a training data set containing multiple images of human faces registered to a template image of a human face and multiple images of human facial expressions registered to the same template image of the human face.

23. A method for generating animated videos comprising:

generating groupings of animation sequences, wherein:

each grouping of animation sequences corresponds to an action in a sequence of animations; and

each animation within a grouping of animation sequences corresponds to an alternative way to perform the action corresponding to that grouping;

receiving a user selection of an animation from each of two or more of the groupings of animation sequences;

creating an interactive animated video using the user selected animations;

generating motion data based on the interactive animated video and;

applying motion data to animate a 3 D character model to cause the 3 D character model to perform the user selected animations of the interactive animated video.

24. The method as recited in claim 23 , further comprising playing back, via a display device, the animated 3 D character model performing the actions of the interactive animated video.

25. The method as recited in claim 23 , further comprising transmitting the motion data to a remote device via a network interface.

26. The method as recited in claim 23 , further comprising arranging the user selected animations based on an order of the groupings of animation sequences before generating the motion data.

27. The method as recited in claim 23 , further comprising obtaining an animation within a grouping of animation sequences corresponding to an action by:

receiving a sequence of video frames;

detecting a human face within the sequence of the video frames;

tracking, within the sequence of video frames, a facial expression change in the detected human face; mapping the tracked facial expression changes to 3 D motion data; and

applying the mapped 3 D motion data to a 3 D character model to cause the 3 D character model to the animation.

Assignments (3)
CHANGE OF NAME Recorded Apr 8, 2019
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 048867/0882 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 15, 2015
From: MIXAMO, INC.
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 035910/0796 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2013
From: CORAZZA, STEFANO; BABCOCK, DANIEL; PINA, CHARLES; DROUIN, SYLVIO
To: MIXAMO, INC.
Reel/Frame 031341/0781 →