Granted Patent B2

US 12,675,930 · App. 18/242,964 · Granted Jul 7, 2026

Inventors: Gurunandan Krishnan Gorumkonda (Kirkland, WA); Shree K. Nayar (New York, NY)

Assignee: Snap Inc.

G06T13/205G06T7/246G06T13/40G06T13/80

Abstract

Claims (85)

1 . A method comprising:

identifying, using one or more processors, a set of animation elements within a computer model, the set of animation elements being associated with a set of motion patterns and a set of speed harmonics;

generating, using the one or more processors, sets of motion data values, each set of motion data values comprising a state-space description of a subset of the set of motion patterns and a subset of the set of speed harmonics for a subset of the set of animation elements within the computer model, each state-space description associated with a respective tempo range;

receiving audio input;

selecting a set of motion data values comprising a state-space description from the sets of motion data values based on a tempo range corresponding with the audio input;

selecting a minimum audio energy threshold and a maximum audio energy threshold based on the audio input received;

assigning a probability to each value of the set of motion data values comprising the state-space description, wherein a zero probability is assigned to a subset of motion data values of the set of motion data values based on the subset of motion data values being associated with energy levels that are either less than the minimum audio energy threshold or greater than the maximum audio energy threshold; and

generating an animation for the audio input based on the set of motion data values, the animation comprising a default animation for a period when the audio input has an energy level either less than the minimum audio energy threshold or greater than the maximum audio energy threshold, the default animation corresponding with a first motion pattern not in the state-space description, and the animation comprising a transition animation between the default animation and a second motion pattern selected from the state-space description.

2 . The method of claim 1 , further comprising:

identifying a plurality of unwanted motion patterns that exceed an animation velocity threshold; and

suppressing the plurality of unwanted motion patterns.

3 . The method of claim 2 , wherein generating the animation comprises:

generating an output image from the computer model, and the set of animation elements; and

animating the output image using a skinned model and the set of motion data values including the probability for each value of the set of motion data values and the plurality of unwanted motion patterns.

4 . The method of claim 1 , further comprising:

processing the audio input to identify a set of audio characteristics, the set of audio characteristics comprising a tempo value and a danceability value.

5 . The method of claim 4 , wherein the danceability value comprises a beat consistency score and at least one music energy criteria.

6 . The method of claim 3 , wherein the output image is animated during a first time period when the audio input meets at least one music energy criteria and wherein the output image is animated using the default animation during a second time period when the audio input does not meet the at least one music energy criteria.

7 . The method of claim 1 , wherein identifying the set of animation elements comprises:

receiving, via a user interface input of an overlay management interface, a user selection of the set of motion patterns selected from a set of system motion patterns, wherein a subset of the set of system motion patterns is selected for each animation element of the set of animation elements; and

selecting a speed harmonic for each user selection of the set of motion patterns, such that the state-space description of the set of motion patterns comprises selected combinations of motion patterns and speed harmonics for each of the set of animation elements.

8 . The method of claim 7 , wherein assigning the probability to each value of the set of motion data values comprising the state-space description comprises:

displaying a first animation comprising a first animation element animated with a first combination of a third motion pattern and a first speed harmonic for a selected tempo; and

receiving a user input assigning a corresponding probability for the first animation element, the third motion pattern, and the first speed harmonic.

9 . The method of claim 7 , wherein assigning the probability to each value of the set of motion data values comprising the state-space description comprises:

for each animation element of the set of animation elements and each of the selected combinations of motion patterns and speed harmonics for each animation element:

displaying an output animation of the computer model with a corresponding combination of each motion pattern and speed harmonics for a corresponding animation element; and

receiving a corresponding user input assigning a corresponding probability to generate the probability to each value of the set of motion data values comprising the state-space description.

10 . The method of claim 1 , further comprising:

calculating a model motion energy for each value of the set of motion data values comprising the state-space description using a first motion model, wherein the model motion energy is determined according to:

KE=Σ j∈joints ½ I j ω j 2 ;

where:

KE is the corresponding model motion energy for a particular set of motion data values of the set of values;

j represents each of a number of joints of the set of animation elements;

I j =Σ K∈child ½ w ( K ) r jk 2 : Moment of Inertia;

w(k) is a moment of inertia associated with a child joint;

ω j is an angular velocity at a given joint;

k is each of a number of child joints attached to a parent joint within the computer model;

r is a radius of each joint from an energy reference point; and

the angular velocity is determined individually for each motion pattern and each speed harmonic of the set of motion patterns and the set of speed harmonics.

11 . A device comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the device to perform operations comprising:

identifying a set of animation elements within a computer model, the set of animation elements being associated with a set of motion patterns and a set of speed harmonics;

generating sets of motion data values, each set of motion data values comprising a state-space description of a subset of the set of motion patterns and a subset of the set of speed harmonics for a subset of the set of animation elements within the computer model, each state-space description associated with a respective tempo range;

receiving audio input;

selecting a set of motion data values comprising a state-space description from the sets of motion data values based on a tempo range corresponding with the audio input;

selecting a minimum audio energy threshold and a maximum audio energy threshold based on the audio input received;

assigning a probability to each value of the set of motion data values comprising the state-space description, wherein a zero probability is assigned to a subset of motion data values of the set of motion data values based on the subset of motion data values being associated with energy levels that are either less than the minimum audio energy threshold or greater than the maximum audio energy threshold; and

generating an animation for the audio input based on the set of motion data values, the animation comprising a default animation for a period when the audio input has an energy level either less than the minimum audio energy threshold or greater than the maximum audio energy threshold, the default animation corresponding with a first motion pattern not in the state-space description, and the animation comprising a transition animation between the default animation and a second motion pattern selected from the state-space description.

12 . The device of claim 11 , wherein the operations further comprise:

identifying a plurality of unwanted motion patterns that exceed an animation velocity threshold; and

suppressing the plurality of unwanted motion patterns.

13 . The device of claim 12 , wherein generating the animation comprises:

generating an output image from the computer model, and the set of animation elements; and

animating the output image using a skinned model and the set of motion data values including the probability for each value of the set of motion data values and the plurality of unwanted motion patterns.

14 . The device of claim 11 , the operations further comprising:

processing the audio input to identify a set of audio characteristics for audio data received at a microphone, the set of audio characteristics comprising a tempo value and a danceability value.

15 . The device of claim 14 , wherein the danceability value comprises a beat consistency score and at least one music energy criteria.

16 . The device of claim 13 , wherein the output image is animated during a first time period when the audio input meets at least one music energy criteria and wherein the output image is animated using the default animation during a second time period when the audio input does not meet the at least one music energy criteria.

17 . The device of claim 11 , wherein identifying the set of animation elements comprises:

receiving, via a user interface input of an overlay management interface, a user selection of the set of motion patterns selected from a set of system motion patterns, wherein a subset of the set of system motion patterns is selected for each animation element of the set of animation elements; and

selecting a speed harmonic for each user selection of the set of motion patterns, such that the state-space description of the set of motion patterns comprises selected combinations of motion patterns and speed harmonics for each of the set of animation elements.

18 . The device of claim 17 , wherein assigning the probability to each value of the set of motion data values comprising the state-space description comprises:

displaying a first animation comprising a first animation element animated with a first combination of a third motion pattern and a first speed harmonic for a selected tempo; and

receiving a user input assigning a corresponding probability for the first animation element, the third motion pattern, and the first speed harmonic.

19 . The device of claim 11 , further comprising:

calculating a model motion energy for each value of the set of motion data values comprising the state-space description using a first motion model, wherein the model motion energy is determined according to:

KE=Σ j∈joints ½ I j ω j 2 ;

where:

KE is the corresponding model motion energy for a particular set of motion data values of the set of values; p 2 j represents each of a number of joints of the set of animation elements;

I j =Σ K∈child ½ w ( K ) r jk 2 : Moment of Inertia;

w(k) is a moment of inertia associated with a child joint;

ω j is an angular velocity at a given joint;

k is each of a number of child joints attached to a parent joint within the computer model;

r is a radius of each joint from an energy reference point; and

the angular velocity is determined individually for each motion pattern and each speed harmonic of the set of motion patterns and the set of speed harmonics.

20 . A non-transitory computer readable medium comprising instructions that, when executed by processing circuitry of a device, cause the device to perform operations comprising:

identifying, using one or more processors, a set of animation elements within a computer model, the set of animation elements being associated with a set of motion patterns and a set of speed harmonics;

generating, using the one or more processors, sets of motion data values, each set of motion data values comprising a state-space description of a subset of the set of motion patterns and a subset of the set of speed harmonics for a subset of the set of animation elements within the computer model, each state-space description associated with a respective tempo range;

receiving audio input;

selecting a set of motion data values comprising a state-space description from the sets of motion data values based on a tempo range corresponding with the audio input;

selecting a minimum audio energy threshold and a maximum audio energy threshold based on the audio input received;

assigning a probability to each value of the set of motion data values comprising the state-space description, wherein a zero probability is assigned to a subset of motion data values of the set of motion data values based on the subset of motion data values being associated with energy levels that are either less than the minimum audio energy threshold or greater than the maximum audio energy threshold; and

generating an animation for the audio input based on the set of motion data values, the animation comprising a default animation for a period when the audio input has an energy level either less than the minimum audio energy threshold or greater than the maximum audio energy threshold, the default animation corresponding with a first motion pattern not in the state-space description, and the animation comprising a transition animation between the default animation and a second motion pattern selected from the state-space description.

Assignments (1)

Continuity (3)

References Cited (189)