Information processing apparatus and method of generating three-dimensional model
An information processing apparatus that generates a three-dimensional model based on a plurality of camera images obtained using a plurality of cameras, performs: transforming, into two-dimensional coordinate points on a camera image, a plurality of representative coordinate points specified from one processing unit voxel of a plurality of processing unit voxels that are obtained by dividing a target three-dimensional space serving as a target of three-dimensional model generation; determining, by using transformation results of the plurality of representative coordinate points, a coordinate point on the camera image corresponding to an internal coordinate point of the one processing unit voxel; and generating the three-dimensional model based on the determined coordinate point on the camera image corresponding to the internal coordinate point of the one processing unit voxel.
1. An information processing apparatus comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to:
(a) transform, into two-dimensional coordinates on a plurality of images obtained using a plurality of image capturing apparatuses, three-dimensional coordinates of a plurality of representative points in a three-dimensional space serving as a target of three-dimensional model generation;
(b) transform, into two-dimensional coordinates on the plurality of images, three-dimensional coordinates of a non-representative point that exists in the three-dimensional space and that is different from the plurality of representative points, based on (i) transformation results of the plurality of representative points, (ii) positions of the plurality of representative points on the three-dimensional space, and (iii) a position of the non-representative point on the three-dimensional space, wherein the three-dimensional coordinates of the non-representative point are transformed by interpolation of the transformation results of the plurality of representative points; and
(c) generate a three-dimensional model based on (i) the plurality of images, (ii) the transformation results of the plurality of representative points, and (iii) transformation results of the non-representative point.
2. The apparatus according to claim 1 , wherein the three-dimensional coordinates of the non-representative point are transformed based on (i) positional relationships of the plurality of representative points in the three-dimensional space and (ii) positional relationships of the plurality of representative points in the image.
3. The apparatus according to claim 1 , wherein the three-dimensional coordinates of the non-representative point are transformed by linear interpolation of the transformation results of the plurality of representative points.
4. The apparatus according to claim 1 , wherein the one or more processors further executes the instructions to:
(a) obtain a silhouette image to identify an object region corresponding to the three-dimensional model; and
(b) determine, based on (i) the silhouette image, (ii) the transformation results of the plurality of representative points, and (iii) the transformation results of the non-representative point, whether a model voxel including at least one of the plurality of representative points forms the object region of the three-dimensional model and whether a model voxel including the non-representative point forms the object region of the three-dimensional model, wherein the three-dimensional model is generated based on the determination result.
5. The apparatus according to claim 1 , wherein the three-dimensional coordinates of the plurality of representative points are transformed by matrix transformation and perspective projection transformation.
6. The apparatus according to claim 1 , wherein the three-dimensional coordinates of the non-representative point are transformed by using a transformation coefficient that is used to transform each of movement amounts of an x-coordinate, y-coordinate, and a z-coordinate in the three-dimensional space into movement amounts of a u-coordinate and a v-coordinate on the image.
7. The apparatus according to claim 6 , wherein the plurality of representative points include a pair of points in which only the x-coordinates differ, a pair of points in which only the y-coordinates differ, and a pair of points in which only the z-coordinates differ.
8. The apparatus according to claim 7 , wherein the plurality of representative coordinate points are formed from four points.
9. The apparatus according to claim 6 , wherein the transformation coefficient is calculated based on a change in a length of a line segment on the image in a case where the line segment of a predetermined length is moved parallel between the plurality of representative points in the three-dimensional space.
10. The apparatus according to claim 9 , wherein the plurality of representative points include a group of four coordinate points that have the same x-coordinate, a group of four points that have the same y-coordinate, and a group of four points that have the same z-coordinate.
11. The apparatus according to claim 10 , wherein the plurality of representative points are formed from six points.
12. The apparatus according to claim 1 , wherein the plurality of representative points are formed from points of a plurality of vertices of one processing unit voxel forming the three-dimensional space.
13. The apparatus according to claim 1 , wherein the three-dimensional coordinates of the non-representative point are transformed based on (i) the relationship between the representative point and the two-dimensional coordinates where the plurality of representative points are transformed, and (ii) the relationship between the positions of the plurality of representative points on the three-dimensional space and the position of the non-representative point on the three-dimensional space.
14. The apparatus according to claim 1 , wherein the three-dimensional model is generated based on (i) the plurality of images, (ii) the relationship between the plurality of representative points and the two-dimensional coordinates where the plurality of representative points are transformed, and (iii) the relationship between the non-representative point and the two-dimensional coordinates where the non-representative point is transformed.
15. The apparatus according to claim 1 , wherein the one or more processors further execute the instructions to select one of a first method and a second method, based on the relationship between (i) a size of a one processing unit voxel forming the three-dimensional space and (ii) a size of a model voxel forming the one processing unit voxel, wherein the first method comprises matrix transformation and perspective projection transformation, and wherein the second method comprises interpolation of the transformation results of the plurality of representative points, and
wherein the three-dimensional coordinates of the non-representative point are transformed based on the selected method.
16. The apparatus according to claim 15 , wherein the first method is selected in a case where a ratio of the size of the model voxel to the size of the one processing unit voxel exceeds an upper threshold or in a case where the ratio is lower than a lower threshold which is smaller than the upper threshold, and
wherein the second method is selected in other cases.
17. The apparatus according to claim 1 , wherein at least one of the plurality of representative points is within a voxel forming the three-dimensional space, and the non-representative point is within another voxel forming the three-dimensional space.
18. A generation method of generating a three-dimensional model, the method comprising:
transforming, into two-dimensional coordinates on a plurality of images obtained using a plurality of image capturing apparatuses, three-dimensional coordinates of a plurality of representative points in a three-dimensional space serving as a target of three-dimensional model generation;
transforming, into two-dimensional coordinates on the plurality of images, three-dimensional coordinates of a non-representative point that exists in the three-dimensional space and that is different from the plurality of representative points, based on (i) transformation results of the plurality of representative points, (ii) positions of the plurality of representative points on the three-dimensional space, and (iii) a position of the non-representative point on the three-dimensional space, wherein the three-dimensional coordinates of the non-representative point are transformed by interpolation of the transformation results of the plurality of representative points; and
generating the three-dimensional model based on (i) the plurality of images, (ii) the transformation results of the plurality of representative points, and (iii) transformation results of the non-representative point.
19. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a generation method of generating a three-dimensional model, the method comprising:
transforming, into two-dimensional coordinates on a plurality of images obtained using a plurality of image capturing apparatuses, three-dimensional coordinates of a plurality of representative points in a three-dimensional space serving as a target of three-dimensional model generation;
transforming, into two-dimensional coordinates on the plurality of images, three-dimensional coordinates of a non-representative point that exists in the three-dimensional space and that is different from the plurality of representative points, based on (i) transformation results of the plurality of representative points, (ii) a position of the plurality of representative points on the three-dimensional space, and (iii) a position of the non-representative point on the three-dimensional space, wherein the three-dimensional coordinates of the non-representative point are transformed by interpolation of the transformation results of the plurality of representative points; and
generating the three-dimensional model based on (i) the plurality of images, (ii) the transformation results of the plurality of representative points, and (iii) transformation results of the non-representative point.