Inference model construction method, inference model construction device, recording medium, configuration device, and configuration method
An inference model construction method obtains a distribution of control points pertaining to a reference state and a distribution of the control points pertaining to a defined state with respect to a defined representation model. The method extracts a first feature value based on the distribution of the control points pertaining to the reference state. The method machine-learns the distribution of the control points pertaining to the defined state while using, as a label, the first feature value, and constructs an inference model based on a result of the leaning performed with respect to a plurality of the defined representation models.
1 . An inference model construction method for constructing an inference model for inferring deformation, in a defined state, of each part of a two-dimensional image of a target object, with respect to a representation model for realizing a rendering representation corresponding to a state different from a reference state of the target object by deforming the part of the two-dimensional image corresponding to the reference state, wherein the defined state differs from the reference state, wherein
the representation model is defined by defining deformation of each part of the two-dimensional image in at least one defined state, and is configured to realize a rendering representation corresponding to at least a state between the reference state and the defined state,
the deformation of each part in the representation model is controlled by a mode of a distribution of control points set for the part,
the inference model construction method comprises:
obtaining the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state with respect to a defined representation model, which is the representation model that has been defined;
extracting a first feature value based on the distribution of the control points pertaining to the reference state obtained in the obtaining; and
estimating a second feature value based on the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state that are obtained in the obtaining,
machine-learning the distribution of the control points pertaining to the defined state obtained in the obtaining while using, as a label, the first feature value extracted in the extracting and the second feature value estimated in the estimating, and constructing an inference model based on a result of the leaning performed with respect to a plurality of the defined representation models, wherein
the second feature value includes information indicating an amount of deformation of translational components and an amount of deformation of non-translational components regarding deformation from the reference state to the defined state,
the target object includes a head of a character,
the information indicating the amount of the deformation of the translational components in the second feature value is estimated based on an amount of movement of at least one control point set for a third-type part constituting the head of the character, between the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state, and
the information indicating the amount of the deformation of the non-translational components in the second feature value is estimated based on a difference between a plurality of control points set for a fourth-type part constituting the head of the character, in an amount of movement between the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state.
2 . The inference model construction method according to claim 1 , wherein
each part in the representation model includes a two-dimensional image of the part, a curved surface to which the two-dimensional image is to be applied, and the control points specifying a shape of the curved surface, and
the first feature value includes information indicating a center position and a size of the curved surface of the part in the distribution of the control points pertaining to the reference state.
3 . The inference model construction method according to claim 2 , wherein
the first feature value further includes information indicating a size of the two-dimensional image to be applied to the curved surface pertaining to each part in the reference state.
4 . The inference model construction method according to claim 1 , further comprising
normalizing the defined representation model,
wherein the representation model normalized in the normalizing is obtained in the obtaining.
5 . The inference model construction method according to claim 4 , wherein
the normalization includes:
normalizing a scale of the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state, based on a distance between two parts included in first-type parts constituting the head of the character; and
normalizing an amount of deformation from the reference state to the defined state, based on an amount of movement of control points set for a second-type part constituting the head of the character between the scale-normalized distribution of the control points pertaining to the reference state and the scale-normalized distribution of the control points pertaining to the defined state.
6 . The inference model construction method according to claim 1 , wherein
the fourth-type part is a part that represents unevenness in the head of the character, and
the amount of the deformation of the non-translational components is estimated based on a difference in an amount of movement between a protruding portion and a recessed portion of the character.
7 . The inference model construction method according to claim 1 , wherein
the third-type part is a part indicating a face of the character.
8 . A non-transitory computer-readable recording medium in which is stored a program for causing a computer to execute the inference model construction method according to claim 1 .
9 . A non-transitory computer-readable recording medium in which is stored a program for causing a computer to configure a representation model for realizing a rendering representation corresponding to a state different from a reference state of a target object by deforming each part of a two-dimensional image corresponding to the reference state of the target object, by using the inference model constructed with use of the inference model construction method according to claim 1 , wherein
the deformation of each part in the representation model is controlled by a mode of a distribution of control points set for the part,
the program causes the computer to execute:
input processing for obtaining the distribution of the control points pertaining to the reference state with respect to a configuration target object;
first determination processing for determining the first feature value based on information obtained through the input processing;
second determination processing for determining the second feature value;
inference processing for inferring, with use of the inference model, the distribution of the control points pertaining to the defined state of the configuration target object, based on the first feature value determined through the first determination processing and the second feature value determined through the second determination processing; and
output processing for configuring and outputting the representation model of the configuration target object, based on a result of the inference performed through the inference processing.
10 . The recording medium according to claim 9 , wherein the program further causes the computer to execute:
display control processing for causing a display unit to display a rendering representation corresponding to the defined state of the configuration target object, based on the representation model output through the output processing;
acceptance processing for accepting an adjustment of at least one of an amount of deformation of translational components or an amount of deformation of non-translational components, with respect to deformation in the rendering representation corresponding to the defined state from the reference state; and
change processing for, if the adjustment is accepted through the acceptance processing, changing the distribution of the control points pertaining to the defined state of the output representation model, based on the adjusted amount of the deformation of the translational components and the non-translational components,
wherein if the adjustment is accepted through the acceptance processing, the rendering representation corresponding to the defined state of the configuration target object is displayed in the display unit, based on the representation model after the changing through the change processing.
11 . The recording medium according to claim 10 , wherein the program further causes the computer to execute
separation processing for separating the distribution of the control points pertaining to the defined state inferred through the inference processing into a distribution of the translational components and a distribution of the non-translational components,
wherein in the change processing, the distribution of the control points pertaining to the defined state of the output representation model is changed to a distribution of the control points obtained by combining the distribution of the translational components and the distribution of the non-translational components that have been changed in accordance with the adjusted amount of the deformation of the translational components and the non-translational components.
12 . The recording medium according to claim 11 , wherein
a constraint condition regarding a placement relationship is defined for at least some parts of the configuration target object, and
in the change processing, placement positions of the at least some parts are changed so as to ensure the placement relationship of the at least some parts before and after the adjustment.
13 . A configuration device for configuring a representation model for realizing a rendering representation corresponding to a state different from a reference state of a target object by deforming each part of a two-dimensional image corresponding to the reference state of the target object, by using the inference model constructed with use of the inference model construction method according to claim 1 , wherein
the deformation of each part in the representation model is controlled by a mode of a distribution of control points set for the part,
the configuration device comprises:
at least one processor; and
a memory configured to store instructions that, when executed by the at least one processor, cause the at least one processor to function as:
an input unit configured to obtain the distribution of the control points pertaining to the reference state with respect to a configuration target object;
a first determination unit configured to determine the first feature value based on information obtained by the input unit;
a second determination unit configured to determine the second feature value;
an inference unit configured to infer, with use of the inference model, the distribution of the control points pertaining to the defined state of the configuration target object, based on the first feature value determined by the first determination unit and the second feature value determined by the second determination unit; and
an output unit configured to configure and output the representation model of the configuration target object, based on a result of the inference performed by the inference unit.
14 . A configuration method for configuring a representation model for realizing a rendering representation corresponding to a state different from a reference state of a target object by deforming each part of a two-dimensional image corresponding to the reference state of the target object, by using the inference model constructed with use of the inference model construction method according to claim 1 , wherein
the deformation of each part in the representation model is controlled by a mode of a distribution of control points set for the part,
the configuration method comprises:
obtaining the distribution of the control points pertaining to the reference state with respect to a configuration target object;
determining the first feature value based on information obtained in the obtaining;
determining the second feature value;
inferring, with use of the inference model, the distribution of the control points pertaining to the defined state of the configuration target object, based on the determined first feature value and the determined second feature value; and
configuring and outputting the representation model of the configuration target object, based on a result of the inference performed in the inferring.
15 . An inference model construction device for constructing an inference model for inferring deformation, in a defined state, of each part of a two-dimensional image of a target object, with respect to a representation model for realizing a rendering representation corresponding to a state different from a reference state of the target object by deforming the part of the two-dimensional image corresponding to the reference state, wherein the defined state differs from the reference state, wherein
the representation model is defined by defining deformation of each part of the two-dimensional image in at least one defined state, and is configured to realize a rendering representation corresponding to at least a state between the reference state and the defined state,
the deformation of each part in the representation model is controlled by a mode of a distribution of control points set for the part,
the inference model construction device comprises:
at least one processor; and
a memory configured to store instructions that, when executed by the at least one processor, cause the at least one processor to function as:
an obtaining unit configured to obtain the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state with respect to a defined representation model, which is the representation model that has been defined;
an extraction unit configured to extract a first feature value based on the distribution of the control points pertaining to the reference state obtained by the obtaining unit;
an estimation unit configured to estimate a second feature value based on the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state that are obtained by the obtaining unit; and
a learning unit configured to machine-learn the distribution of the control points pertaining to the defined state obtained by the obtaining unit while using, as a label, the first feature value extracted by the extraction unit and the second feature value estimated by the estimation unit, and constructing an inference model based on a result of the leaning performed with respect to a plurality of the defined representation models, wherein
the second feature value includes information indicating an amount of deformation of translational components and an amount of deformation of non-translational components regarding deformation from the reference state to the defined state,
the target object includes a head of a character,
the information indicating the amount of the deformation of the translational components in the second feature value is estimated based on an amount of movement of at least one control point set for a third-type part constituting the head of the character, between the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state, and
the information indicating the amount of the deformation of the non-translational components in the second feature value is estimated based on a difference between a plurality of control points set for a fourth-type part constituting the head of the character, in an amount of movement between the distribution of the control points pertaining to the reference state and the distribution of the control points pertaining to the defined state.