Blendshape compression system
The systems and methods described herein can pre-process a blendshape matrix via a global clusterization process and a local clusterization process. The pre-processing can cause the blendshape matrix to be divided into multiple blocks. The techniques can further apply a matrix compression technique to each block of the blendshape matrix to generate a compression result. The matrix compression technique can comprise a matrix approximation step, an accuracy verification step, and a recursive compression step. The compression result for each block may be combined to generate a compressed blendshape matrix for rendering a virtual entity.
1. A computer-implemented method for rendering a three-dimensional virtual entity during runtime execution of a game application, comprising:
under control of one or more hardware computing devices configured with specific computer-executable instructions, the specific computer-executable instructions stored in an electronic hardware memory,
receiving user input associated with the virtual entity in a virtual environment of the game application, wherein the virtual entity comprises a plurality of blendshapes;
determining movement of a virtual entity based, at least in part, on the user input;
identifying a first pose of a virtual entity for rendering within the virtual environment based on the determined movement;
identifying a compressed blendshape matrix associated with rendering the virtual entity;
accessing the compressed blendshape matrix, wherein the compressed blendshape matrix includes approximated values for a plurality of sub-matrixes of an uncompressed blendshape matrix;
determining movements of a set of the plurality of blendshapes of the virtual entity based on the compressed blendshape matrix by performing, at least, a matrix-vector multiplication and an inverse permutation on individual sub-matrixes of the plurality of sub-matrixes; and
rendering the first pose based at least in part on the determined movements of the set of blendshapes.
2. The computer-implemented method of claim 1 , wherein a mesh of the virtual entity includes the plurality of blendshapes and a plurality of vertices.
3. The computer-implemented method of claim 2 , wherein determining movements of a set of the plurality of blendshapes of the virtual entity includes determining movement vectors of a set of the plurality of vertices associated with the set of blendshapes.
4. The computer-implemented method of claim 3 further comprising calculating a weight vector to achieve the first pose, wherein effects of movement of a plurality of movement vectors are controlled by the weight vector.
5. The computer-implemented method of claim 2 , wherein determining movements of a set of the plurality of blendshapes of the virtual entity includes calculating movements of the vertices based on approximated values in the compressed blendshape matrix.
6. The computer-implemented method of claim 5 , wherein the movements of the vertices are calculated using o=n+Bw, wherein o is a vector containing resulting vertex positions of target blendshapes; w is a vector storing a blending weight of the target blendshapes, n is a neutral shape vector, and B is the compressed blendshape matrix.
7. The computer-implemented method of claim 1 further comprising:
permutating a portion of a weight vector by multiplying with permutation matrixes for respective blocks of the compressed blendshape matrix;
for each block,
performing matrix-vector multiplication on each non-zero sub-matrix in a pre-processed matrix associated with the respective sub-matrix,
assembling the result of the matrix-vector multiplication for each non-zero sub-matrix of the block into a resulting displacement vector for the respective block; and
performing an inverse permutation on the resulting displacement vectors resulting in displacements associated with movements of the vertices.
8. The computer-implemented method of claim 1 , wherein each row of the compressed blendshape matrix is associated with a different blendshape and each column is associated with a pose of the virtual entity.
9. The computer-implemented method of claim 8 , where values of the compressed matrix correspond to displacements of respective blendshapes with respect to a neutral pose.
10. A computer system comprising:
a hardware processor in communication with the electronic data store, the hardware processor configured to execute specific computer-executable instructions to at least:
receive user input associated with the virtual entity in a virtual environment of the game application, wherein the virtual entity comprises a plurality of blendshapes;
determine movement of a virtual entity based, at least in part, on the user input;
identify a first pose of a virtual entity for rendering within the virtual environment based on the determined movement;
identify a compressed blendshape matrix associated with rendering the virtual entity;
access the compressed blendshape matrix, wherein the compressed blendshape matrix includes approximated values for a plurality of sub-matrixes of an uncompressed blendshape matrix;
determine movements of a set of the plurality of blendshapes of the virtual entity based on the compressed blendshape matrix by performing, at least, a matrix-vector multiplication and an inverse permutation on individual sub-matrixes of the plurality of sub-matrixes; and
render the first pose based at least in part on the determined movements of the set of blendshapes.
11. The system of claim 10 , wherein a mesh of the virtual entity includes the plurality of blendshapes and a plurality of vertices.
12. The system of claim 11 , wherein when determining movements of a set of the plurality of blendshapes of the virtual entity, the computer-executable instructions further configure the hardware processor to determine movement vectors of a set of the plurality of vertices associated with the set of blendshapes.
13. The system of claim 12 , wherein the computer-executable instructions further configure the hardware processor to calculate a weight vector to achieve the first pose, wherein effects of movement of a plurality of movement vectors are controlled by the weight vector.
14. The system of claim 11 , wherein when determining movements of a set of the plurality of blendshapes of the virtual entity the computer-executable instructions further configure the hardware processor to calculate movements of the vertices based on approximated values in the compressed blendshape matrix.
15. The system of claim 14 , wherein the movements of the vertices are calculated using o=n+Bw, wherein o is a vector containing resulting vertex positions of target blendshapes; w is a vector storing a blending weight of the target blendshapes, n is a neutral shape vector, and B is the compressed blendshape matrix.
16. The system of claim 10 , wherein the computer-executable instructions further configure the hardware processor to:
permutate a portion of a weight vector by multiplying with permutation matrixes for respective blocks of the compressed blendshape matrix;
for each block,
perform matrix-vector multiplication on each non-zero sub-matrix in a pre-processed matrix associated with the respective sub-matrix, and
assemble the result of the matrix-vector multiplication for each non-zero sub-matrix of the block into a resulting displacement vector for the respective block; and
perform an inverse permutation on the resulting displacement vectors resulting in displacements associated with movements of the vertices.
17. The system of claim 10 , wherein each row of the compressed blendshape matrix is associated with a different blendshape and each column is associated with a pose of the virtual entity.
18. The system of claim 17 , wherein values of the compressed matrix correspond to displacements of respective blendshapes with respect to a neutral pose.