Method for training SMPL parameter prediction model, computer device, and storage medium
A method for training an SMPL parameter prediction model, including: obtaining a sample picture; inputting the sample picture into a pose parameter prediction model to obtain a predicted pose parameter; inputting the sample picture into a shape parameter prediction model to obtain a predicted shape parameter; calculating model prediction losses according to an SMPL parameter prediction model and annotation information of the sample picture; and updating the pose parameter prediction model and the shape parameter prediction model according to the model prediction losses.
1. A method for training a skinned multi-person linear model (SMPL) parameter prediction model, performed by a computer device, the method comprising:
obtaining a sample picture, the sample picture containing a human body image;
inputting the sample picture into a pose parameter prediction model to obtain a predicted pose parameter, the predicted pose parameter being a parameter used for indicating a human body pose in the SMPL parameter prediction model;
inputting the sample picture into a shape parameter prediction model to obtain a predicted shape parameter, the predicted shape parameter being a parameter used for indicating a human body shape in the SMPL parameter prediction model;
constructing a three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter;
calculating model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and annotation information of the sample picture; and
reversely training the pose parameter prediction model and the shape parameter prediction model the according to the model prediction losses.
2. The method according to claim 1 , wherein the model prediction losses comprise a first model prediction loss; and the calculating model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and annotation information of the sample picture comprises:
calculating the first model prediction loss according to the three-dimensional human body model based on the SMPL parameter prediction model and annotated SMPL parameters in the annotation information, the annotated SMPL parameters comprising an annotated pose parameter and an annotated shape parameter.
3. The method according to claim 2 , wherein the calculating the first model prediction loss according to the three-dimensional human body model based on the SMPL parameter prediction model and annotated SMPL parameters in the annotation information comprises:
calculating a first Euclidean distance between the annotated pose parameter and the predicted pose parameter;
calculating a second Euclidean distance between the annotated shape parameter and the predicted shape parameter; and
determining the first model prediction loss according to the first Euclidean distance and the second Euclidean distance.
4. The method according to claim 1 , wherein the model prediction losses comprise a second model prediction loss; and the calculating the model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and the annotation information of the sample picture comprises:
calculating the second model prediction loss according to predicted joint coordinates of joints in the three-dimensional human body model and annotated joint coordinates of joints in the annotation information.
5. The method according to claim 4 , wherein the annotated joint coordinates comprise three-dimensional annotated joint coordinates and/or two-dimensional annotated joint coordinates; and
the calculating the second model prediction loss according to predicted joint coordinates of joints in the three-dimensional human body model and annotated joint coordinates of joints in the annotation information comprises:
calculating third Euclidean distances between three-dimensional predicted joint coordinates of the joints in the three-dimensional human body model and the three-dimensional annotated joint coordinates; and
calculating the second model prediction loss according to the third Euclidean distances.
6. The method according to claim 5 , wherein the calculating third Euclidean distances between three-dimensional predicted joint coordinates of the joints in the three-dimensional human body model and the three-dimensional annotated joint coordinates comprises:
determining the three-dimensional predicted joint coordinates of the joints in the three-dimensional human body model according to vertex coordinates of model vertices around the joints in the three-dimensional human body model; and
calculating the third Euclidean distances between the three-dimensional predicted joint coordinates and the three-dimensional annotated joint coordinates.
7. The method according to claim 5 , wherein the calculating the second model prediction loss according to the third Euclidean distances comprises:
calculating fourth Euclidean distances between two-dimensional predicted joint coordinates of the joints in the three-dimensional human body model and the two-dimensional annotated joint coordinates; and
calculating the second model prediction loss according to the fourth Euclidean distances.
8. The method according to claim 7 , wherein the pose parameter prediction model is further configured to output a projection parameter according to the inputted sample picture, the projection parameter being used for projecting points in a three-dimensional space into a two-dimensional space; and
the calculating fourth Euclidean distances between two-dimensional predicted joint coordinates of the joints in the three-dimensional human body model and the two-dimensional annotated joint coordinates comprises:
determining the three-dimensional predicted joint coordinates of the joints in the three-dimensional human body model according to vertex coordinates of model vertices around the joints in the three-dimensional human body model;
performing projection processing on the three-dimensional predicted joint coordinates according to the projection parameter to obtain the two-dimensional predicted joint coordinates; and
calculating the fourth Euclidean distances between the two-dimensional predicted joint coordinates and the two-dimensional annotated joint coordinates.
9. The method according to claim 1 , wherein the model prediction losses comprise a third model prediction loss; and the calculating the model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and the annotation information of the sample picture comprises:
calculating the third model prediction loss according to a predicted two-dimensional human body contour of the three-dimensional human body model and an annotated two-dimensional human body contour in the annotation information.
10. The method according to claim 9 , wherein the pose parameter prediction model is further configured to output a projection parameter according to the inputted sample picture, the projection parameter being used for projecting points in a three-dimensional space into a two-dimensional space; and
the calculating the third model prediction loss according to a predicted two-dimensional human body contour of the three-dimensional human body model and an annotated two-dimensional human body contour in the annotation information comprises:
projecting model vertices in the three-dimensional human body model into a two-dimensional space according to the projection parameter, and generating the predicted two-dimensional human body contour;
calculating a first contour loss and a second contour loss according to the predicted two-dimensional human body contour and the annotated two-dimensional human body contour; and
determining the third model prediction loss according to the first contour loss and the second contour loss.
11. The method according to claim 10 , wherein the calculating a first contour loss and a second contour loss according to the predicted two-dimensional human body contour and the annotated two-dimensional human body contour comprises:
calculating a first shortest distance from each contour point in the predicted two-dimensional human body contour to the annotated two-dimensional human body contour, and
calculating the first contour loss according to the first shortest distance corresponding to each contour point in the predicted two-dimensional human body contour; and
calculating a second shortest distance from each contour point in the annotated two-dimensional human body contour to the predicted two-dimensional human body contour, and calculating the second contour loss according to the second shortest distance corresponding to each contour point in the annotated two-dimensional human body contour.
12. The method according to claim 11 , wherein the calculating the first contour loss according to the first shortest distance corresponding to each contour point in the predicted two-dimensional human body contour comprises:
determining a first weight of the contour point corresponding to each model vertex according to visibility of the joint to which the each model vertex in the three-dimensional human body model belongs; and
calculating the first contour loss according to the first weight and the first shortest distance corresponding to each contour point in the predicted two-dimensional human body contour, wherein
in a case that the joint to which a model vertex belongs is visible, the first weight of the contour point corresponding to the model vertex is 1; and in a case that the joint to which a model vertex belongs is invisible, the first weight of the contour point corresponding to the model vertex is 0.
13. The method according to claim 11 , wherein the calculating the first contour loss according to the first shortest distance corresponding to each contour point in the predicted two-dimensional human body contour comprises:
determining the predicted joint coordinates of the joint to which each model vertex in the three-dimensional human body model belongs;
determining a second weight of the contour point corresponding to each model vertex according to a fifth Euclidean distance between the predicted joint coordinates and the annotated joint coordinates, the second weight and the fifth Euclidean distance having a negative correlation; and
calculating the first contour loss according to the second weight and the first shortest distance corresponding to each contour point in the predicted two-dimensional human body contour.
14. The method according to claim 1 , further comprising:
performing regularization on the predicted shape parameter to obtain a fourth model prediction loss; and
updating the pose parameter prediction model and the shape parameter prediction model according to the fourth model prediction loss.
15. The method according to claim 1 , comprising:
obtaining a target picture, the target picture containing a human body image;
inputting the target picture into the updated pose parameter prediction model to obtain a predicted pose parameter;
inputting the target picture into the updated shape parameter prediction model to obtain a predicted shape parameter; and
constructing a target three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter.
16. A computer device, comprising a processor and a memory, the memory storing a plurality of instructions that, when executed by the processor, cause the computer device to perform a plurality of operations including:
obtaining a sample picture, the sample picture containing a human body image;
inputting the sample picture into a pose parameter prediction model to obtain a predicted pose parameter, the predicted pose parameter being a parameter used for indicating a human body pose in the SMPL parameter prediction model;
inputting the sample picture into a shape parameter prediction model to obtain a predicted shape parameter, the predicted shape parameter being a parameter used for indicating a human body shape in the SMPL parameter prediction model;
constructing a three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter;
calculating model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and annotation information of the sample picture; and
reversely training the pose parameter prediction model and the shape parameter prediction model according to the model prediction losses.
17. The computer device according to claim 16 , wherein the plurality of operations further comprise:
obtaining a target picture, the target picture containing a human body image;
inputting the target picture into the updated pose parameter prediction model to obtain a predicted pose parameter;
inputting the target picture into the updated shape parameter prediction model to obtain a predicted shape parameter; and
constructing a target three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter.
18. A non-transitory computer-readable storage medium, storing a plurality of instructions that, when executed by a processor of a computer device, cause the computer device to perform a plurality of operations including:
obtaining a sample picture, the sample picture containing a human body image;
inputting the sample picture into a pose parameter prediction model to obtain a predicted pose parameter, the predicted pose parameter being a parameter used for indicating a human body pose in the SMPL parameter prediction model;
inputting the sample picture into a shape parameter prediction model to obtain a predicted shape parameter, the predicted shape parameter being a parameter used for indicating a human body shape in the SMPL parameter prediction model;
constructing a three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter;
calculating model prediction losses according to the three-dimensional human body model based on the SMPL parameter prediction model and annotation information of the sample picture; and
reversely training the pose parameter prediction model and the shape parameter prediction model according to the model prediction losses.
19. The non-transitory computer-readable storage medium according to claim 18 , wherein the plurality of operations further comprise:
obtaining a target picture, the target picture containing a human body image;
inputting the target picture into the updated pose parameter prediction model to obtain a predicted pose parameter;
inputting the target picture into the updated shape parameter prediction model to obtain a predicted shape parameter; and
constructing a target three-dimensional human body model according to the predicted pose parameter and the predicted shape parameter.