Information processing apparatus, method for controlling information processing apparatus, and storage medium

1. An information processing apparatus comprising:

one or more hardware processors; and

one or more memories which store instructions executable by the one or more hardware processors to cause the information processing apparatus to perform at least:

holding a plurality of learning models, which are learned by using a captured image captured by an image capturing apparatus and depth information of the captured image as teacher data, for estimating depth information corresponding to an input image, in association with an imaging position of the captured image used as the teacher data;

selecting a learning model suitable for a scene shown in the input image from the plurality of learning models based on an evaluation result obtained from a degree of match between any one of imaging positions held in association with the learning models and an imaging position where the input image was captured; and

estimating first depth information using the input image and the selected learning model.

2. The information processing apparatus according to claim 1 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

obtaining position and orientation of the image capturing apparatus based on the first depth information.

3. The information processing apparatus according to claim 2 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

generating display information based on at least one of the input image, the first depth information, the depth information of the captured image held, the evaluation results, or the positions and orientation.

4. The information processing apparatus according to claim 3 ,

wherein in generating the display information, the instructions are executable by the one or more hardware processors to cause the information processing apparatus to perform:

generating the display information by combining, based on the first depth information, a CG image of a virtual object with the input image.

5. The information processing apparatus according to claim 3 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

estimating, for each of the learning models, third depth information based on the input image and the learning model,

wherein in generating the display information, the instructions are executable by the one or more hardware processors to cause the information processing apparatus to perform:

generating the display information by combining, based on the third depth information, a CG image of a virtual object with the input image.

6. The information processing apparatus according to claim 3 ,

wherein in generating the display information, the instructions are executable by the one or more hardware processors to cause the information processing apparatus to perform:

generating the display information by reproducing, based on the first depth information, a three-dimensional shape of a captured scene of the input image.

7. The information processing apparatus according to claim 3 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

estimating, for each of the learning models, third depth information based on the input image and the learning model, and

generating the display information by reproducing, based on the third depth information, a three-dimensional shape of a captured scene of the input image.

8. The information processing apparatus according to claim 3 , further comprising:

a display unit configured to display the display information.

9. The information processing apparatus according to claim 1 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

estimating second depth information from an input image, by stereo measurement method,

estimating, for each of the learning models, third depth information based on the input image and the learning model, and

calculating evaluation results of the learning models such that the higher a degree of match between the second depth information and the third depth information is, the higher the evaluation result is.

10. The information processing apparatus according to claim 9 , further comprising:

a sensor configured to measure a movement amount of the image capturing apparatus,

wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

calculating evaluation results of the learning models such that the higher the degree of match between at least one of sensor information measured by the sensor and depth information calculated based on the sensor information, and the third depth information is, the higher the evaluation result is.

11. The information processing apparatus according to claim 9 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

updating the second depth information and the third depth information, based on a second input image captured at a second time, which is different from a time at which the input image was captured by the image capturing apparatus, and

re-calculating evaluation results of the learning models such that the higher the degree of match between the updated second depth information and the updated third depth information is, the higher the evaluation result is.

12. The information processing apparatus according to claim 9 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

estimating a plurality of pieces of second depth information and a plurality of pieces of third depth information, based on a plurality of input images captured at a plurality of points in time by the image capturing apparatus, and

calculating evaluation values of the learning models such that the higher the degree of match between the plurality of pieces of second depth information and the plurality of pieces of third depth information is, the higher the evaluation result is.

13. The information processing apparatus according to claim 1 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

estimating, for each of the learning models, third depth information based on the input image and the learning model, and

calculating evaluation results of the learning models such that the more a size or shape of a known object detected from the input image conforms to the third depth information, the higher the evaluation result is.

14. The information processing apparatus according to claim 1 , wherein the instructions are executable by the one or more hardware processors to further cause the information processing apparatus to perform:

inputting a third input image and fourth depth information obtained by a second image capturing apparatus;

classifying the third input image and the fourth depth information according to a type of scene of the third input image or the fourth depth information;

holding, based on results of the classifying, the third input image and the fourth depth information according to the type of scene; and

generating the plurality of learning models for the respective types of scene by using the third input image and the fourth depth information, and storing the generated plurality of learning models.

15. A method for controlling an information processing apparatus, the method comprising:

holding a plurality of learning models, which are learned by using a captured image captured by an image capturing apparatus and depth information of the captured image as teacher data, for estimating depth information corresponding to an input image, in association with an imaging position of the captured image used as the teacher data;

selecting a learning model suitable for a scene shown in the input image from the plurality of learning models based on an evaluation result obtained from a degree of match between any one of imagining positions held in association with the learning models and an imaging position where the input image was captured; and

estimating first depth information using the input image and the selected learning model.

16. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute a method for controlling an information processing apparatus, the method comprising:

holding a plurality of learning models, which are learned by using a captured image captured by an image capturing apparatus and depth information of the captured image as teacher data, for estimating depth information corresponding to an input image, in association with an imaging position of the captured image used as the teacher data;

selecting a learning model suitable for a scene shown in the input image from the plurality of learning models based on an evaluation result obtained from a degree of match between any one of imagining positions held in association with the learning models and an imagining position where the input image was captured; and

estimating first depth information using the input image and the selected learning model.