Information processing apparatus, information processing method, and storage medium for associating defect regions of a 3D model with silhouette images
To make it possible to easily identify a silhouette image causing a hole or chip having occurred in a 3D model and an imaging device corresponding to the silhouette image. Defect region information identifying a defect region of a 3D model representing a three-dimensional shape of an object, which is generated based on a plurality of silhouette images, is set. Then, the defect region identified by the set defect region information is associated with the plurality of silhouette images and results of the association are displayed.
1 . An information processing apparatus comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to perform:
obtaining a plurality of silhouette images representing a silhouette of an object, wherein the plurality of silhouette images are captured by a plurality of imaging devices;
generating a 3D model representing a three-dimensional shape of the object based on the plurality of silhouette images;
setting defect region information identifying a defect region of the generated 3D model, the defect region corresponding to a missing portion of the three-dimensional shape of the object caused by a missing silhouette portion in at least one of the plurality of silhouette images;
associating the defect region identified by the set defect region information with the plurality of silhouette images; and
generating and displaying a highlighted image in which the defect region identified by the associated defect region information is highlighted for a silhouette image in which the defect region is included in a silhouette of the object among the plurality of silhouette images and in which the highlighted image indicates a region corresponding to the missing silhouette portion in the silhouette image.
2 . The information processing apparatus according to claim 1 , wherein
the one or more processors further execute the instructions to perform:
displaying a first UI screen for receiving user instructions to designate a defect region having occurred in the generated 3D model and
the defect region information is set based on the user instructions via the first UI screen.
3 . The information processing apparatus according to claim 2 , wherein
on the first UI screen, a UI element for a user to designate the generated 3D model and the defect region having occurred in the 3D model is displayed and
the user instructions are given by using the UI element.
4 . The information processing apparatus according to claim 3 , wherein
the UI element is a three-dimensional marker.
5 . The information processing apparatus according to claim 1 , wherein
the one or more processors further execute the instructions to perform:
detecting the defect region having occurred in the generated 3D model based on data relating to the object, which is stored in advance in a storage unit and
the defect region information is set based on results of the detection.
6 . The information processing apparatus according to claim 5 , wherein
the data relating to the object is a basic 3D model having no defect of the object and
the defect region having occurred in the 3D model is detected by performing processing to fit the basic 3D model to the generated 3D model.
7 . The information processing apparatus according to claim 6 , wherein
the object is a person and
the 3D model having no defect is a 3D model with bones.
8 . The information processing apparatus according to claim 5 , wherein
the data relating to the object is one or a plurality of pieces of two-dimensional shape data representing a basic shape of the object when being viewed from a specific viewpoint.
9 . The information processing apparatus according to claim 1 , wherein
the one or more processors further execute the instructions to perform:
displaying at least one highlighted image selectively in accordance with a ratio of an area occupied by a defect region highlighted in each highlighted image when a plurality of highlighted images is generated.
10 . The information processing apparatus according to claim 1 , wherein
the one or more processors further execute the instructions to perform:
modifying a silhouette image in which a defect region identified by the associated defect region information is included in a silhouette of the object so that the defect region is filled in the silhouette of the object indicated by the silhouette image.
11 . The information processing apparatus according to claim 10 , wherein
the one or more processors further execute the instructions to perform:
displaying a second UI screen for receiving user instructions to designate a portion to be modified and
the modifying is performed based on the user instructions via the second UI screen.
12 . The information processing apparatus according to claim 10 , wherein
the modifying is performed automatically based on results of the association.
13 . The information processing apparatus according to claim 10 , wherein
the one or more processors further execute the instructions to perform:
regenerating the 3D model of the object by using the modified silhouette image.
14 . The information processing apparatus according to claim 1 , wherein
if the obtained plurality of silhouette images is moving images including a plurality of frames, the generating, the setting, the associating, and the displaying are performed for each frame.
15 . The information processing apparatus according to claim 14 , wherein
defect region information identifying a defect region of the 3D model generated for a specific frame of the plurality of frames is set and
in the associating, processing to develop the defect region information in the specific frame in frames before and after the specific frame is performed.
16 . The information processing apparatus according to claim 1 , wherein
the one or more processors further execute the instructions to perform:
associating the defect region identified by the set defect region information with the plurality of silhouette images by projecting the defect region onto each of the plurality of silhouette images using imaging parameters of the plurality of imaging devices and determining, for each of the plurality of silhouette images, whether a projected region corresponding to the defect region protrudes from or is not included in a silhouette area in the silhouette image.
17 . An information processing method comprising the steps of:
obtaining a plurality of silhouette images representing a silhouette of an object, wherein the plurality of silhouette images are captured by a plurality of imaging devices;
generating a 3D model representing a three-dimensional shape of the object based on the plurality of silhouette images;
setting defect region information identifying a defect region of the generated 3D model, the defect region corresponding to a missing portion of the three-dimensional shape of the object caused by a missing silhouette portion in at least one of the plurality of silhouette images;
associating the defect region identified by the set defect region information with the plurality of silhouette images; and
generating and displaying a highlighted image in which the defect region identified by the associated defect region information is highlighted for a silhouette image in which the defect region is included in a silhouette of the object among the plurality of silhouette images and in which the highlighted image indicates a region corresponding to the missing silhouette portion in the silhouette image.
18 . A non-transitory computer readable storage medium storing a program for causing a computer to perform an information processing method comprising the steps of:
obtaining a plurality of silhouette images representing a silhouette of an object, wherein the plurality of silhouette images are captured by a plurality of imaging devices;
generating a 3D model representing a three-dimensional shape of the object based on the plurality of silhouette images;
setting defect region information identifying a defect region of the generated 3D model, the defect region corresponding to a missing portion of the three-dimensional shape of the object caused by a missing silhouette portion in at least one of the plurality of silhouette images;
associating the defect region identified by the set defect region information with the plurality of silhouette images; and
generating and displaying a highlighted image in which the defect region identified by the associated defect region information is highlighted for a silhouette image in which the defect region is included in a silhouette of the object among the plurality of silhouette images and in which the highlighted image indicates a region corresponding to the missing silhouette portion in the silhouette image.