Information processing apparatus, information processing method and storage medium
The computation amount or memory usage in a case of estimating a radiance field with high accuracy regarding an target object is reduced. An information processing apparatus of the present disclosure obtains captured images data obtained by capturing objects in a space from viewpoints and camera parameters corresponding to the viewpoints, regarding an target object of the objects, obtains information indicating a likelihood that an image formed on each pixel in each captured image is the target object as a likelihood value corresponding to each pixel in each captured image for each target object, and estimates information on the space including color information corresponding to each position in the space and a likelihood value for each target object in each position in the space based on the captured images data, the camera parameters, and the likelihood value corresponding to the pixels in each captured image for each target object.
1 . An information processing apparatus comprising:
one or more hardware processors; and
one or more memories storing computer-executable instructions that are executable by the one or more hardware processors, the computer-executable instructions including instructions for:
obtaining data on a plurality of captured images obtained by capturing at least one object present in a predetermined space from a plurality of viewpoints and camera parameters corresponding to the plurality of viewpoints in a case of the capturing respectively;
regarding at least one target object of the at least one object, obtaining information indicating a likelihood that an image formed on each pixel in each of the plurality of captured images is an image of the target object as a likelihood value corresponding to each pixel in each of the plurality of captured images for each target object; and
estimating information on the predetermined space including color information corresponding to each position in the predetermined space and a likelihood value for each target object in each position in the predetermined space based on the data on the plurality of captured images, the camera parameters corresponding to the plurality of viewpoints respectively, and the likelihood value corresponding to each pixel in each of the plurality of captured images for each target object.
2 . The information processing apparatus according to claim 1 , wherein
the information on the predetermined space is information indicating a function outputting the color information corresponding to each position in the predetermined space and the likelihood value for each target object in each position in the predetermined space.
3 . The information processing apparatus according to claim 1 , wherein
the color information corresponding to each position in the predetermined space is color information on a combination of a position and an orientation in the predetermined space.
4 . The information processing apparatus according to claim 1 , wherein the computer-executable instructions further include instructions for:
estimating information on the predetermined space further including volume density corresponding to each position in the predetermined space.
5 . The information processing apparatus according to claim 4 , wherein
the information on the predetermined space is information indicating the function outputting the color information corresponding to each position in the predetermined space, the likelihood value for each target object in each position in the predetermined space, and the volume density corresponding to each position in the predetermined space.
6 . The information processing apparatus according to claim 4 , wherein the computer-executable instructions further include instructions for:
calculating an error about a color by using the data on the plurality of captured images, the camera parameters corresponding to the plurality of viewpoints respectively, the color information corresponding to each position in the predetermined space obtained by estimating, and volume density obtained by at least one of estimating and calculating,
calculating an error about the likelihood value by using the likelihood value corresponding to the pixels in each of the plurality of captured images for each target object, the likelihood value for each target object in each position in the predetermined space obtained by estimating, and the volume density obtained by at least one of estimating and the calculating; and
estimating the information on the predetermined space by minimizing the error about the color and the error about the likelihood value obtained by calculating.
7 . The information processing apparatus according to claim 1 , wherein the computer-executable instructions further include instructions for:
calculating volume density corresponding to each position in the predetermined space based on the likelihood value obtained by estimating each target object in each position in the predetermined space.
8 . The information processing apparatus according to claim 1 , wherein the computer-executable instructions further include instructions for:
generating an image visualizing an object in the predetermined space based on the information on the predetermined space.
9 . The information processing apparatus according to claim 8 , wherein the computer-executable instructions further include instructions for:
displaying and outputting the generated image on and to a display device.
10 . The information processing apparatus according to claim 8 , wherein the computer-executable instructions further include instructions for:
generating the image in such a way that a color of a representation corresponding to the target object in the image becomes more transparent as the likelihood value of the target object in the predetermined space is smaller.
11 . A method for processing information comprising the steps of:
obtaining data on a plurality of captured images obtained by capturing at least one object present in a predetermined space from a plurality of viewpoints and camera parameters corresponding to the plurality of viewpoints in a case of the capturing respectively;
regarding at least one target object of the at least one object, obtaining information indicating a likelihood that an image formed on each pixel in each of the plurality of captured images is the at least one target object as a likelihood value corresponding to each pixel in each of the plurality of captured images for each target object; and
estimating information on the predetermined space including color information corresponding to each position in the predetermined space and a likelihood value for each target object in each position in the predetermined space based on the data on the plurality of captured images, the camera parameters corresponding to the plurality of viewpoints respectively, and the likelihood value corresponding to each pixel in each of the plurality of captured images for each target object.
12 . A non-transitory computer-readable storage medium storing computer-executable instructions for causing a computer to perform a control method of an information processing apparatus, the control method comprising the steps of:
obtaining data on a plurality of captured images obtained by capturing at least one object present in a predetermined space from a plurality of viewpoints and camera parameters corresponding to the plurality of viewpoints in a case of the capturing respectively;
regarding at least one target object of the at least one object, obtaining information indicating a likelihood that an image formed on each pixel in each of the plurality of captured images is the at least one target object as a likelihood value corresponding to each pixel in each of the plurality of captured images for each target object; and
estimating information on the predetermined space including color information corresponding to each position in the predetermined space and a likelihood value for each target object in each position in the predetermined space based on the data on the plurality of captured images, the camera parameters corresponding to the plurality of viewpoints respectively, and the likelihood value corresponding to each pixel in each of the plurality of captured images for each target object.