Information processing apparatus and method, and storage medium
An information processing apparatus controls, based on a change with time passage in a three-dimensional model of an object that is generated based on a plurality of videos obtained from a plurality of image capture apparatuses, a frequency of the generation of the three-dimensional model of the object.
1 . An information processing apparatus, comprising:
one or more memories storing instructions; and
one or more processors executing the instructions to:
generate three-dimensional models of moving objects at a specific frequency based on a plurality of videos obtained from a plurality of image capture apparatuses;
identify a movement with time passage in the three-dimensional model of each object; and
change the specific frequency of each three-dimensional model based on the movement with time passage in the three-dimensional model of each object,
wherein the one or more processors execute the instructions to identify the movement with time passage in the three-dimensional model of the object based on a movement in a barycenter of the three-dimensional model of each object between different times,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined smaller than a first threshold, the one or more processors execute the instructions to suspend generation of the three-dimensional model of the object,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the first threshold and smaller than a second threshold, which is higher than the first threshold, the one or more processors execute the instructions to reduce the specific frequency of the three-dimensional model of the object compared to a normal model generation processing, and
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the second threshold, which is higher than the first threshold, the one or more processors execute the instructions to perform the normal model generation processing.
2 . The information processing apparatus according to claim 1 , wherein
the barycenter is a barycenter of a cuboid that circumscribes the three-dimensional model.
3 . The information processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
generate simplified three-dimensional models of each object that are coarser than a three-dimensional model used in generation of a virtual viewpoint video, and
identify the movement with time passage in the three-dimensional model of each object using the simplified three-dimensional models.
4 . The information processing apparatus according to claim 1 , wherein
the one or more processors execute the instructions to identify the movement with time passage in each three-dimensional model of each object at a time interval longer than a time interval of frames of the plurality of videos.
5 . The information processing apparatus according to claim 4 , wherein
the one or more processors execute the instructions to identify the movement with time passage in the three-dimensional model of each object at a time interval that is M times the time interval of the frames of the plurality of videos (where M is a natural number equal to or larger than two) based on a three-dimensional model obtained in a current frame and on a three-dimensional model obtained in a frame that is M frames ahead of the current frame.
6 . The information processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
store three-dimensional models that have been generated at times of respective frames,
wherein while generation of the three-dimensional model of the object is suspended, a pointer indicating a position of a latest three-dimensional model among three-dimensional models of the object that have already been stored is stored.
7 . The information processing apparatus according to claim 6 , wherein the one or more processors further execute the instructions to:
generate an image of the object viewed from a virtual viewpoint based on a three-dimensional model of the object, and generate a virtual viewpoint video using the image,
wherein the one or more processors execute the instructions to generate the image of the object by reading out the stored three-dimensional model of the object, and in a case where the pointer is stored, the one or more processors execute the instructions to read out the three-dimensional model from the position instructed by the pointer.
8 . The information processing apparatus according to claim 6 , wherein the one or more processors further execute the instructions to:
generate an image of the object viewed from a virtual viewpoint based on a three-dimensional model of the object, and generate a virtual viewpoint video using the image,
wherein the one or more processors execute the instructions to generate the image of the object by reading out the stored three-dimensional model of the object, and in a case where the pointer is stored, the one or more processors execute the instructions to use a three-dimensional model that has already been read out instead.
9 . The information processing apparatus according to claim 1 , wherein the one or more processors further execute the instructions to:
determine whether the object is a target of changing the specific frequency based on an attribute of the object or the three-dimensional model thereof, and
change the specific frequency with respect to the object of the determined target of changing.
10 . The information processing apparatus according to claim 9 , wherein
the one or more processors execute the instructions to determine that the object is the target of changing the specific frequency in a case where a size of the three-dimensional model of the object is larger than a threshold.
11 . The information processing apparatus according to claim 9 , wherein
the one or more processors execute the instructions to determine that the object is not the target of changing the specific frequency in a case where the three-dimensional model of the object has been determined to be a person.
12 . The information processing apparatus according to claim 10 , wherein the one or more processors further execute the instructions to:
set the threshold based on the attribute of the object or the three-dimensional model of the object.
13 . An information processing method, comprising:
generating three-dimensional models of moving objects at a specific frequency based on a plurality of videos obtained from a plurality of image capture apparatuses;
identifying a movement with time passage in the three-dimensional model of each object; and
changing the specific frequency of each three-dimensional model in the generating based on the movement with time passage in the three-dimensional model of each object,
wherein the movement with time passage in the three-dimensional model of the object is identified based on a movement in a barycenter of the three-dimensional model of each object between different times,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be smaller than a first threshold, generation of the three-dimensional model of the object is suspended,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the first threshold and smaller than a second threshold, which is higher than the first threshold, the specific frequency of the three-dimensional model of the object is reduced compared to a normal model generation processing, and
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the second threshold, the normal model generation processing is performed.
14 . A non-transitory computer readable storage medium storing a program for causing a computer to perform an information processing method, comprising:
generating three-dimensional models of moving objects at a specific frequency based on a plurality of videos obtained from a plurality of image capture apparatuses;
identifying a movement with time passage in the three-dimensional model of each object; and
changing the specific frequency of each three-dimensional model in the generating based on the movement with time passage in the three-dimensional model of each object,
wherein the movement with time passage in the three-dimensional model of the object is identified based on a movement in a barycenter of the three-dimensional model of each object between different times,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be smaller than a first threshold, generation of the three-dimensional model of the object is suspended,
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the first threshold and smaller than a second threshold, which is higher than the first threshold, the specific frequency of the three-dimensional model of the object is reduced compared to a normal model generation processing, and
in a case where the identified movement with time passage in the barycenter of the three-dimensional model of the object has been determined to be equal to or larger than the second threshold, the normal model generation processing is performed.