Animating graphical objects using input video
A method and system for driving a graphical object based on a performance of an actor in a video stream. A plurality of key frames are selected in a calibration video stream, allowing animation control values to be defined, in accordance with artistic intent, for each of a set of control variables, corresponding to each of the key frames. Features of the actor in the calibration video stream are parameterized so as to obtain a vector of values of parameters for each frame of the calibration video stream, and, then, an array of distance vectors is derived, characterizing a distance between each pair of vectors of values of parameters among the video stream frames. The space of distance vectors is mapped into the set of control variables according to a mapping operation which is then applied to distance vectors derived from an actual performance video stream to obtain a time sequence of animation control values. Finally, an animation is created on the basis of applying the sequence of animation control values to the graphical object.
1. A method for driving a graphical object in computer-generated material in film, television and computer games based on a performance of an actor in an actual performance video stream, the method comprising:
a. selecting a plurality of key calibration frames in a calibration video stream to serve as a basis set of frames specific to the actor and the graphical object;
b. defining animation control values, in accordance with artistic intent, for each of a set of control variables, corresponding to each of the key calibration frames;
c. parameterizing features of the actor in the calibration video stream so as to obtain a vector of values of parameters for each frame of the calibration video stream;
d. deriving, for each key calibration frame, a vector containing a distance to all of the key calibration frames;
e. for each frame in the video stream, deriving a vector containing, as elements, distances to all of the basis set of key calibration frames;
f. mapping the space of distance vectors into the set of control variables according to a mapping operation;
g. applying the mapping operation to distance vectors derived from the actual performance video stream to obtain a time sequence of animation control values; and
h. creating the graphical object embodying a particular visual depiction of the actor based on applying the sequence of animation control values to the graphical object.
2. A method in accordance with claim 1 , wherein the mapping operation is a matrix of regression coefficients.
3. A method in accordance with claim 1 , wherein the actual performance video stream is identical to the calibration video stream.