Generating target-character-animation sequences based on style-aware puppets patterned after source-character-animation sequences

1. A system comprising:

at least one processor; and

at least one non-transitory computer memory comprising a source-character-animation sequence of a two-dimensional animation character portrayed in source drawings corresponding to a source-skeletal-animation sequence of a source skeleton in source-skeletal poses, and instructions that, when executed by at least one processor, cause the system to:

generate style-aware puppets for the source drawings portraying the two-dimensional animation character, each style-aware puppet comprising a coarse-deformation model of a source drawing portraying the two-dimensional animation character from the source-character-animation sequence, a residual-deformation model of the source drawing portraying the two-dimensional animation character, a skeletal-difference map comprising differences between a source-skeletal pose and a stylized-skeletal pose corresponding to the source drawing portraying the two-dimensional animation character, and a visual-texture representation of the source drawing portraying the two-dimensional animation character;

identify a target-skeletal-animation sequence comprising a target skeleton in target-skeletal poses;

generate target-style-aware puppets corresponding to the target-skeletal-animation sequence by, for a target-skeletal pose of the target-skeletal poses:

comparing the target-skeletal pose with the source-skeletal-animation sequence to select a set of source-skeletal poses;

identifying a set of source drawings from the source-character-animation sequence from the source-character-animation sequence corresponding to the set of source-skeletal poses;

blending the style-aware puppets corresponding to the set of source drawings to create a target-style-aware puppet; and

based on the target-style-aware puppets, generate a target-character-animation sequence of the two-dimensional animation character portrayed in target drawings corresponding to the target-skeletal-animation sequence.

2. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to:

compare the target-skeletal pose with the source-skeletal-animation sequence to select the set of source-skeletal poses by:

identifying a set of target-skeletal poses adjacent to the target-skeletal pose in the target-skeletal-animation sequence; and

selecting the set of source-skeletal poses by comparing trajectories of joints across the set of target-skeletal poses with trajectories of joints across the set of source-skeletal poses; and

identify the set of source drawings by identifying source drawings corresponding to the set of source-skeletal poses in the source-character-animation sequence.

3. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to blend the style-aware puppets corresponding to the set of source drawings by:

interpolating coarse-deformation models for the style-aware puppets corresponding to the set of source drawings to create a modified-coarse-deformation model for the target-style-aware puppet;

linearly blending residual-deformation models for the style-aware puppets corresponding to the set of source drawings to create a modified-residual-deformation model for the target-style-aware puppet;

linearly blending skeletal-difference maps for the style-aware puppets to create a modified-skeletal-difference map for the target-style-aware puppet; and

linearly blending visual-texture representations for the style-aware puppets corresponding to the set of source drawings to create a modified-visual-texture representation for the target-style-aware puppet.

4. The system of claim 3 , further comprising instructions that, when executed by the at least one processor, cause the system to linearly blend the visual-texture representations for the style-aware puppets corresponding to the set of source drawings by:

rectifying the visual-texture representations for each style-aware puppet corresponding to the set of source drawings based on the coarse-deformation model for the style-aware puppet and the residual-deformation model for the style-aware puppet; and

linearly blending the rectified visual-texture representations to create the modified-visual-texture representation for the target-style-aware puppet.

5. The system of claim 4 , further comprising instructions that, when executed by the at least one processor, cause the system to, based on the target-style-aware puppet, generate the target-character-animation sequence of the two-dimensional animation character portrayed in the target drawings by, for the target-style-aware puppet:

applying the modified-skeletal-difference map to the target-skeletal pose to generate a stylized-target-skeletal pose; and

utilizing the modified-coarse-deformation model for the target-style-aware puppet, the modified-residual-deformation model for the target-style-aware puppet, and the stylized-target-skeletal pose to generate a target drawing portraying the two-dimensional animation character.

6. The system of claim 4 , further comprising instructions that, when executed by the at least one processor, cause the system to, based on the target-style-aware puppet, generate the target-character-animation sequence of the two-dimensional animation character portrayed in the target drawings by, for the target-style-aware puppet, utilize a segmentation-guidance channel and a temporal-appearance-guidance channel with the modified-visual-texture representation to generate the two-dimensional animation character in the target drawing.

7. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to determine the coarse-deformation model of the source drawing portraying the two-dimensional animation character by:

generating a layered as-rigid-as-possible (“ARAP”) mesh for a layered-template puppet, wherein the layered ARAP mesh comprises ARAP mesh layers corresponding to semantic layers of the layered-template puppet; and

adjusting the ARAP mesh layers of the layered ARAP mesh to conform to the semantic layers of the layered-template puppet positioned according to the source drawing.

8. The system of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the system to select the set of source drawings from the source-character-animation sequence by:

identifying a set of target-skeletal poses adjacent to the target-skeletal pose in the target-skeletal-animation sequence comprising the target skeleton in the target-skeletal poses;

selecting a set of source-skeletal poses analogous to the set of target-skeletal poses by comparing trajectories of joints across the set of target-skeletal poses with trajectories of joints across the set of source-skeletal poses; and

selecting the set of source drawings by identifying source drawings corresponding to the set of source-skeletal poses in the source-character-animation sequence.

9. The system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the system to blend the style-aware puppets corresponding to the set of source drawings to create the target-style-aware puppet by:

identifying additional sets of source drawings from the source-character-animation sequence, the additional sets of source drawings each comprising at least one source drawing overlapping with the set of source drawings in the source-character-animation sequence; and

blending style-aware puppets corresponding to the additional sets of source drawings with the style-aware puppets corresponding to the set of source drawings.

10. The system of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the system to blend the style-aware puppets corresponding to the additional sets of source drawings with the style-aware puppets corresponding to the set of source drawings by:

interpolating ARAP meshes for the style-aware puppets corresponding to both the set of source drawings and the additional sets of source drawings to create a modified ARAP mesh for the target-style-aware puppet;

linearly blending residual-deformation models for the style-aware puppets corresponding to both the set of source drawings and the additional sets of source drawings to create a modified-residual-deformation model for the target-style-aware puppet;

linearly blending skeletal-difference maps for the style-aware puppets corresponding to both the set of source drawings and the additional sets of source drawings to create a modified-skeletal-difference map for the target-style-aware puppet; and

linearly blending visual-texture representations for the style-aware puppets corresponding to both the set of source drawings and the additional sets of source drawings to create a modified-visual-texture representation for the target-style-aware puppet.

11. A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to:

identify a source-character-animation sequence of a two-dimensional animation character portrayed in source drawings corresponding to a source-skeletal-animation sequence of a source skeleton in source-skeletal poses;

generate style-aware puppets for the source drawings portraying the two-dimensional animation character, each style-aware puppet comprising a coarse-deformation model of a source drawing portraying the two-dimensional animation character from the source drawings, a residual-deformation model of the source drawing, a skeletal-difference map comprising differences between a source-skeletal pose and a stylized-skeletal pose corresponding to the source drawing, and a visual-texture representation of the source drawing;

identify a target-skeletal-animation sequence comprising a target skeleton in target-skeletal poses;

for a target-skeletal pose of the target-skeletal poses, select from the source-character-animation sequence a set of source drawings analogous to the target-skeletal pose; and

blend style-aware puppets corresponding to the set of source drawings to create a target-style-aware puppet corresponding to the target-skeletal pose; and

based on the target-style-aware puppets, generate a target-character-animation sequence of the two-dimensional animation character portrayed in target drawings.

12. The non-transitory computer readable storage medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to blend the style-aware puppets corresponding to the set of source drawings by:

interpolating coarse-deformation models for the style-aware puppets corresponding to the set of source drawings to create a modified-coarse-deformation model for the target-style-aware puppet;

linearly blending residual-deformation models for the style-aware puppets corresponding to the set of source drawings to create a modified-residual-deformation model for the target-style-aware puppet;

linearly blending skeletal-difference maps for the style-aware puppets to create a modified skeletal difference for the target-style-aware puppet; and

linearly blending visual-texture representations for the style-aware puppets corresponding to the set of source drawings to create a modified-visual-texture representation for the target-style-aware puppet.

13. The non-transitory computer readable storage medium of claim 12 , further comprising instructions that, when executed by the at least one processor, cause the computing device to linearly blend the visual-texture representations for the style-aware puppets corresponding to the set of source drawings by:

rectifying the visual-texture representations for each style-aware puppet corresponding to the set of source drawings based on the coarse-deformation model for the style-aware puppet and the residual-deformation model for the style-aware puppet; and

linearly blending the rectified visual-texture representations to create a modified-visual-texture representation for the target-style-aware puppet.

14. The non-transitory computer readable storage medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to select the set of source drawings analogous to the target-skeletal pose by:

identifying a set of target-skeletal poses adjacent to the target-skeletal pose in the target-skeletal-animation sequence comprising the target skeleton in the target-skeletal poses;

selecting a set of source-skeletal poses analogous to the set of target-skeletal poses by comparing trajectories of joints across the set of target-skeletal poses with trajectories of joints across the set of source-skeletal poses; and

selecting the set of source drawings by identifying source drawings corresponding to the set of source-skeletal poses in the source-character-animation sequence.

15. The non-transitory computer readable storage medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computing device to, based on the target-style-aware puppets, generate the target-character-animation sequence of the two-dimensional animation character portrayed in the target drawings by, for the target-style-aware puppet:

generating a segmentation-guidance channel comprising segment-color labels for semantic layers corresponding to a style-aware puppet of a source drawing and segment-color labels for semantic layers corresponding to the target-style-aware puppet;

generating a temporal-appearance-guidance channel comprising a smoothed visual-texture representation corresponding to the style-aware puppet of the source drawing and a smoothed modified-visual-texture representation for the target-style-aware puppet; and

synthesizing a target drawing for the target-character-animation sequence based on the segmentation-guidance channel and the temporal-appearance-guidance channel.

16. A method comprising:

identifying a source-character-animation sequence of a two-dimensional animation character portrayed in source drawings corresponding to a source-skeletal-animation sequence of a source skeleton in source-skeletal poses;

generating style-aware puppets for the source drawings portraying the two-dimensional animation character, each style-aware puppet comprising a coarse-deformation model of a source drawing portraying the two-dimensional animation character from the source drawings, a residual-deformation model of the source drawing, a skeletal-difference map comprising differences between a source-skeletal pose and a stylized-skeletal pose corresponding to the source drawing, and a visual-texture representation of the source drawing;

identifying a target-skeletal-animation sequence comprising a target skeleton in target-skeletal poses;

for a target-skeletal pose of the target-skeletal poses, selecting from the source-character-animation sequence a set of source drawings analogous to the target-skeletal pose; and

blending style-aware puppets corresponding to the set of source drawings to create a target-style-aware puppet corresponding to the target-skeletal pose; and

based on the target-style-aware puppets, generating a target-character-animation sequence of the two-dimensional animation character portrayed in target drawings.

17. The method of claim 16 , wherein the style-aware puppets correspond to a layered-template puppet comprising:

semantic layers corresponding to segments of the two-dimensional animation character in a model-source drawing;

junctions at which each of the semantic layers connect to one or more of the semantic layers; and

joints of the source skeleton corresponding to locations of the semantic layers.

18. The method of claim 17 , further comprising determining the coarse-deformation model of the source drawing by:

generating a layered mesh for the layered-template puppet, wherein the layered mesh comprises mesh layers corresponding to the semantic layers of the layered-template puppet; and

adjusting the mesh layers of the layered mesh to conform to the semantic layers of the layered-template puppet positioned according to the source drawing.

19. The method of claim 18 , further comprising determining the skeletal-difference map by:

determining joint positions of the layered-template puppet positioned according to the source drawing based on attachment points of the mesh layers on the semantic layers of the layered-template puppet;

inferring a root-joint position, bone lengths, and joint rotations based on the joint positions of the layered-template puppet to create the stylized-skeletal pose corresponding to the source drawing; and

determining differences between root-joint positions, bone lengths, and joint rotations of the source-skeletal pose and the stylized-skeletal pose.

20. The method of claim 16 , further comprising selecting the set of source drawings analogous to the target-skeletal pose by:

identifying a set of target-skeletal poses adjacent to the target-skeletal pose in the target-skeletal-animation sequence comprising the target skeleton in the target-skeletal poses;

selecting a set of source-skeletal poses analogous to the set of target-skeletal poses by comparing trajectories of joints across the set of target-skeletal poses with trajectories of joints across the set of source-skeletal poses; and

selecting the set of source drawings by identifying source drawings corresponding to the set of source-skeletal poses in the source-character-animation sequence.