Image processing apparatus, image processing method, and storage medium

1 . An image processing apparatus comprising:

one or more memories storing instructions; and

one or more processors executing the instructions to perform:

obtaining a first model generated by deforming, based on the orientation of an object identified by a plurality of texture images representing a color of the object based on image capturing of a plurality of imaging devices, a base model stored in advance and representing the basic three-dimensional shape of the object;

obtaining a second model representing the three-dimensional shape of the object, which is generated based on a silhouette image based on image capturing of the plurality of imaging devices;

deleting one or more unnecessary elements among elements configuring the second model based on a difference between the first model and the second model; and

determining shape accuracy of the second model based on a difference between the first model and the second model,

wherein the deleting the one or more unnecessary elements is performed by taking a specific area in the second model as a target based on the determined shape accuracy,

wherein among elements configuring the base model, to an element corresponding to a specific region of the object, exclusion information indicating whether the element is not the target of the deleting the one or more unnecessary elements is appended in advance,

wherein the deformation of the base model to which the exclusion information is appended is performed based on the specified orientation of the object,

wherein in the determination, in a case where the exclusion information appended to the corresponding element in the first model among the elements configuring the second model indicates that the element is not the target of the deleting the one or more unnecessary elements, the element is determined without exception to have a high shape accuracy and in a case where the exclusion information appended to the corresponding element in the deformed base model indicates that the element is the target of the deleting the one or more unnecessary elements, an element whose distance to the element configuring the deformed base model is longer than or equal to a threshold value is determined to be an element whose shape accuracy is low, and

wherein the deleting the one or more unnecessary elements is performed by taking an area including the element determined to have a low shape accuracy of the second model as the specific area.

2 . The image processing apparatus according to claim 1 , wherein

the first model is obtained by identifying the orientation of the object and then deforming the base model based on the identified orientation.

3 . The image processing apparatus according to claim 1 , wherein

the second model is generated by the visual hull method using the silhouette image.

4 . The image processing apparatus according to claim 1 , wherein

a distance of the element configuring the second model to the element configuring the first model is calculated and

the deleting the one or more unnecessary elements is performed by taking an area including the element whose calculated distance is longer than or equal to a threshold value among the elements configuring the second model as the specific area.

5 . The image processing apparatus according to claim 1 , wherein

the deleting the one or more unnecessary elements is performed by taking a specific region of the second model as a target based on a size of the element configuring the second model and a distance between the element configuring the second model and the element configuring the first model.

6 . The image processing apparatus according to claim 1 , wherein

camera parameters of the plurality of imaging devices are obtained and

in processing to delete the one or more unnecessary elements:

an element of interest among the elements configuring the second model is projected onto a texture image of an imaging device from which the element of interest is viewed based on the camera parameters;

the pixel value of the texture image at the projection destination is obtained; and

the deleting the one or more unnecessary elements is performed by determining whether to delete the element of interest based on the variance of the obtained pixel value.

7 . The image processing apparatus according to claim 1 , wherein

distance information represents from the plurality of imaging devices to the object is obtained and

the deleting the one or more unnecessary elements is performed based on the distance information.

8 . The image processing apparatus according to claim 1 , wherein

the object is a person and

the orientation of the object is identified by obtaining coordinate data of feature points of the person from each of the plurality of texture images and estimating the three-dimensional orientation of the person by a set of parts simulating a human bone structure based on the obtained coordinate data of the feature points.

9 . The image processing apparatus according to claim 1 , wherein

the second model represents the three-dimensional shape of the object by a voxel set or the format of point cloud.

10 . The image processing apparatus according to claim 1 , wherein

the base model represents the basic three-dimensional shape of the object by a set of polygon meshes.

11 . An image processing method comprising the steps of:

obtaining a first model generated by deforming, based on the orientation of an object identified by a plurality of texture images representing a color of the object based on image capturing of a plurality of imaging devices, a base model stored in advance and representing the basic three-dimensional shape of the object;

obtaining a second model representing the three-dimensional shape of the object, which is generated based on a silhouette image based on image capturing of the plurality of imaging devices;

deleting one or more unnecessary elements among elements configuring the second model based on a difference between the first model and the second model; and

determining shape accuracy of the second model based on a difference between the first model and the second model,

wherein the deleting the one or more unnecessary elements is performed by taking a specific area in the second model as a target based on the determined shape accuracy,

wherein among elements configuring the base model, to an element corresponding to a specific region of the object, exclusion information indicating whether the element is not the target of the deleting the one or more unnecessary elements is appended in advance,

wherein the deformation of the base model to which the exclusion information is appended is performed based on the specified orientation of the object,

wherein in the determination, in a case where the exclusion information appended to the corresponding element in the first model among the elements configuring the second model indicates that the element is not the target of the deleting the one or more unnecessary elements, the element is determined without exception to have a high shape accuracy and in a case where the exclusion information appended to the corresponding element in the deformed base model indicates that the element is the target of the deleting the one or more unnecessary elements, an element whose distance to the element configuring the deformed base model is longer than or equal to a threshold value is determined to be an element whose shape accuracy is low, and

wherein the deleting the one or more unnecessary elements is performed by taking an area including the element determined to have a low shape accuracy of the second model as the specific area.

12 . A non-transitory computer readable storage medium storing a program for causing a computer to perform an image processing method comprising the steps of:

obtaining a first model generated by deforming, based on the orientation of an object identified by a plurality of texture images representing a color of the object based on image capturing of a plurality of imaging devices, a base model stored in advance and representing the basic three-dimensional shape of the object;

obtaining a second model representing the three-dimensional shape of the object, which is generated based on a silhouette image based on image capturing of the plurality of imaging devices;

deleting one or more unnecessary elements among elements configuring the second model based on a difference between the first model and the second model; and

determining shape accuracy of the second model based on a difference between the first model and the second model,

wherein the deleting the one or more unnecessary elements is performed by taking a specific area in the second model as a target based on the determined shape accuracy,

wherein among elements configuring the base model, to an element corresponding to a specific region of the object, exclusion information indicating whether the element is not the target of the deleting the one or more unnecessary elements is appended in advance,

wherein the deformation of the base model to which the exclusion information is appended is performed based on the specified orientation of the object,

wherein in the determination, in a case where the exclusion information appended to the corresponding element in the first model among the elements configuring the second model indicates that the element is not the target of the deleting the one or more unnecessary elements, the element is determined without exception to have a high shape accuracy and in a case where the exclusion information appended to the corresponding element in the deformed base model indicates that the element is the target of the deleting the one or more unnecessary elements, an element whose distance to the element configuring the deformed base model is longer than or equal to a threshold value is determined to be an element whose shape accuracy is low, and

wherein the deleting the one or more unnecessary elements is performed by taking an area including the element determined to have a low shape accuracy of the second model as the specific area.