Method and apparatus for generating virtual character, electronic device and readable storage medium
A method and apparatus for generating a virtual character, an electronic device and a computer readable storage medium are provided. The method includes: performing mesh simplification on an initial model of a virtual character to obtain a mesh-simplified model; obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.
1. A method for generating a virtual character, the method comprising:
performing mesh simplification on an initial model of the virtual character by using a standard mesh simplification algorithm with a mesh simplification mark, to obtain a mesh-simplified model; wherein the mesh simplification mark is used to indicate that a degree of mesh simplification on a facial area of the initial model is less than a degree of mesh simplification on other areas;
obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and
establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.
2. The method according to claim 1 , wherein the method further comprises:
synchronizing and updating coordinates of the mesh-simplified initial model in a dimension of a two-dimensional map, to obtain an updated model; and
performing protective processing on areas of five facial sense organs of the updated model by using skeleton skin information of the initial model.
3. The method according to claim 1 , wherein the method further comprises:
determining a device type of a current device; and
determining, in response to the device type being a mobile terminal, a parameter of a target mesh simplification degree matching a computing performance of the current mobile terminal.
4. The method according to claim 3 , wherein the performing mesh simplification on the initial model by using the standard mesh simplification algorithm with the mesh simplification mark, comprises:
performing mesh simplification on the initial model by using the standard mesh simplification algorithm with a facial boundary protection mark, wherein a parameter of a mesh simplification degree of the standard mesh simplification algorithm is the parameter of the target mesh simplification degree.
5. The method according to claim 1 , wherein the method further comprises:
performing a model alignment between the mesh-simplified model and the initial model, and updating coordinates of two-dimensional maps of the mesh-simplified model and the initial model.
6. The method according to claim 1 , wherein the obtaining the first target model by performing white model mapping rendering on the area of each material type on the mesh-simplified model, and obtaining the second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model, comprises:
performing area partitioning on the mesh-simplified model according to material type, to obtain a partition model;
establishing a mapping relationship between a two-dimensional map of an area of each material type and the white model mapping rendering by using a differentiable renderer and a white model mapping rendering technology, to obtain the first target model; and
establishing a mapping relationship between the two-dimensional map of an area of each material type and the hyper-realistic rendering by using the differentiable renderer and a hyper-realistic pipeline, to obtain the second target model.
7. The method according to claim 1 , wherein the establishing the bidirectional mapping between the first target model and the second target model, and obtaining the target virtual character through iterative updating of the bidirectional mapping, comprises:
establishing a first mapping relationship between the first target model and the second target model by using a recurrent neural network;
establishing a second mapping relationship between the second target model and the first target model by using the recurrent neural network;
performing iterative updating on the first mapping relationship and the second mapping relationship, based on a perceptual loss function and an adversarial loss function in the recurrent neural network, to obtain an updated first mapping relationship and an updated second mapping relationship; and
updating the second target model by using the updated first mapping relationship and the updated second mapping relationship, to obtain the target virtual character.
8. The method according to claim 1 , wherein the method further comprises:
acquiring a user image of a target user; and
generating a virtual character corresponding to the target user in a virtual data space, based on the user image and the target virtual character.
9. An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
performing mesh simplification on an initial model of the virtual character by using a standard mesh simplification algorithm with a mesh simplification mark, to obtain a mesh-simplified model; wherein the mesh simplification mark is used to indicate that a degree of mesh simplification on a facial area of the initial model is less than a degree of mesh simplification on other areas;
obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and
establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.
10. The electronic device according to claim 9 , wherein the operations further comprise:
synchronizing and updating coordinates of the mesh-simplified initial model in a dimension of a two-dimensional map, to obtain an updated model; and
performing protective processing on areas of five facial sense organs of the updated model by using skeleton skin information of the initial model.
11. The electronic device according to claim 9 , wherein the operations further comprise:
determining a device type of a current device; and
determining, in response to the device type being a mobile terminal, a parameter of a target mesh simplification degree matching a computing performance of the current mobile terminal.
12. The electronic device according to claim 11 , wherein the performing mesh simplification on the initial model by using the standard mesh simplification algorithm with the mesh simplification mark, comprises:
performing mesh simplification on the initial model by using the standard mesh simplification algorithm with a facial boundary protection mark, wherein a parameter of a mesh simplification degree of the standard mesh simplification algorithm is the parameter of the target mesh simplification degree.
13. The electronic device according to claim 9 , wherein the operations further comprise:
performing a model alignment between the mesh-simplified model and the initial model, and updating coordinates of two-dimensional maps of the mesh-simplified model and the initial model.
14. The electronic device according to claim 9 , wherein the obtaining the first target model by performing white model mapping rendering on the area of each material type on the mesh-simplified model, and obtaining the second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model, comprises:
performing area partitioning on the mesh-simplified model according to material type, to obtain a partition model;
establishing a mapping relationship between a two-dimensional map of an area of each material type and the white model mapping rendering by using a differentiable renderer and a white model mapping rendering technology, to obtain the first target model; and
establishing a mapping relationship between the two-dimensional map of an area of each material type and the hyper-realistic rendering by using the differentiable renderer and a hyper-realistic pipeline, to obtain the second target model.
15. The electronic device according to claim 9 , wherein the establishing the bidirectional mapping between the first target model and the second target model, and obtaining the target virtual character through iterative updating of the bidirectional mapping, comprises:
establishing a first mapping relationship between the first target model and the second target model by using a recurrent neural network;
establishing a second mapping relationship between the second target model and the first target model by using the recurrent neural network;
performing iterative updating on the first mapping relationship and the second mapping relationship, based on a perceptual loss function and an adversarial loss function in the recurrent neural network, to obtain an updated first mapping relationship and an updated second mapping relationship; and
updating the second target model by using the updated first mapping relationship and the updated second mapping relationship, to obtain the target virtual character.
16. The electronic device according to claim 9 , wherein the operations further comprise:
acquiring a user image of a target user; and
generating a virtual character corresponding to the target user in a virtual data space, based on the user image and the target virtual character.
17. A non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform operations comprising:
performing mesh simplification on an initial model of the virtual character by using a standard mesh simplification algorithm with a mesh simplification mark, to obtain a mesh-simplified model; wherein the mesh simplification mark is used to indicate that a degree of mesh simplification on a facial area of the initial model is less than a degree of mesh simplification on other areas;
obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and
establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.