Viewpoint recommendation apparatus, recommendation method thereof, and non-transitory computer readable medium

1. A viewpoint recommendation apparatus for estimating, from a first viewpoint of an object, a second viewpoint at which the object is to be observed next in order to estimate a pose of the object, the viewpoint recommendation apparatus comprising:

image acquisition means for acquiring an image of the object at the first viewpoint;

image feature extraction means for extracting an image feature of the image of the object from the image of the object acquired by the image acquisition means;

pose estimation means for calculating a first likelihood map indicating a relation between the estimated pose of the object estimated from the image of the object and a likelihood of this estimated pose based on the image feature of the object extracted by the image feature extraction means;

second storage means for storing a second likelihood map indicating a relation between the true first viewpoint in the estimated pose and a likelihood of this first viewpoint;

third storage means for storing a third likelihood map indicating a relation between the pose of the object when the object is observed at the first viewpoint and the second viewpoint and a likelihood of this pose; and

viewpoint estimation means for estimating the second viewpoint so that a value of an evaluation function becomes the maximum or minimum, the evaluation function using, as a parameter, a result of multiplying and integrating the first likelihood map calculated by the pose estimation means, the second likelihood map stored in the second storage means, and the third likelihood map stored in the third storage means.

2. The viewpoint recommendation apparatus according to claim 1 , wherein the viewpoint estimation means estimates the second viewpoint δ 2 (hat) using a formula below the first likelihood map p(ξ|I 1 ) estimated by the pose estimation means, the second likelihood map p(φ 1 |ξ) stored in the second storage means, and the third likelihood map p(θ|δ 2 , φ 1 ) stored in the third storage means:

{circumflex over (δ)} 2 =argmax δ 2 g (∫∫ P (θ|δ 2 ,Ø 1 ) p (Ø 1 |ξ) p (ξ| I 1 ) dξdØ 1 )

OR

{circumflex over (δ)} 2 =argmax δ 2 g (∫∫ p (θ|δ 2 ,Ø 1 ) p (Ø 1 |ξ) p (ξ| I 1 ) dξdØ 1 )  [Formula 5]

in this formula, ξ is the estimated pose of the object, I 1 is the image of the object acquired by the image acquisition means at the first viewpoint, φ 1 is the first viewpoint, θ is the pose of the object, and δ 2 is the second viewpoint.

3. The viewpoint recommendation apparatus according to claim 1 , further comprising first learning means for learning the image feature of each pose of the object, wherein

the pose estimation means compares the image feature of the object extracted by the image feature extraction means with the image feature of each pose of the object learned by the first learning means to calculate the first likelihood map.

4. The viewpoint recommendation apparatus according to claim 1 , wherein

the second storage means is second learning means which learns a relation between the true first viewpoint in the estimated pose and the likelihood of the true first viewpoint and stores it as the second likelihood map.

5. The viewpoint recommendation apparatus according to claim 1 , wherein

the third storage means is third learning means which learns a relation between the pose of the object when the object is observed at the first viewpoint and the second viewpoint and a likelihood of this pose and stores it as the third likelihood map.

6. The viewpoint recommendation apparatus according to claim 1 , wherein

the evaluation function is a function for calculating a variance of a distribution or a function for calculating an entropy of the distribution.

7. The viewpoint recommendation apparatus according to claim 1 , wherein

the viewpoint estimation means estimates at least one second viewpoint so that the value of the evaluation function becomes greater than or equal to a threshold or less than or equal to the threshold.

8. A recommendation method performed by a viewpoint recommendation apparatus for estimating, from a first viewpoint of an object, a second viewpoint at which the object is to be observed next in order to estimate a pose of the object, the recommendation method comprising:

acquiring an image of the object at the first viewpoint;

extracting an image feature of the image of the object from the acquired image of the object;

calculating a first likelihood map indicating a relation between the estimated pose of the object estimated from the image of the object and a likelihood of this estimated pose based on the extracted image feature of the object; and

estimating the second viewpoint so that a value of an evaluation function becomes the maximum or minimum, the evaluation function using, as a parameter, a result of multiplying and integrating the calculated first likelihood map, a second likelihood map indicating a relation between the true first viewpoint in the estimated pose and a likelihood of this first viewpoint, and a third likelihood map indicating a relation between the pose of the object when the object is observed at the first viewpoint and the second viewpoint and a likelihood of this pose.

9. A non-transitory computer readable medium storing a program of a viewpoint recommendation apparatus for estimating, from a first viewpoint of an object, a second viewpoint at which the object is to be observed next in order to estimate a pose of the object, the program causing a computer to execute:

a process of acquiring an image of the object at the first viewpoint;

a process of extracting an image feature of the image of the object from the acquired image of the object;

a process of calculating a first likelihood map indicating a relation between the estimated pose of the object estimated from the image of the object and a likelihood of this estimated pose based on the extracted image feature of the object; and

a process of estimating the second viewpoint so that a value of an evaluation function becomes the maximum or minimum, the evaluation function using, as a parameter, a result of multiplying and integrating the calculated first likelihood map, a second likelihood map indicating a relation between the true first viewpoint in the estimated pose and a likelihood of this first viewpoint, and a third likelihood map indicating a relation between the pose of the object when the object is observed at the first viewpoint and the second viewpoint and a likelihood of this pose.

10. A viewpoint recommendation apparatus for estimating, from a first viewpoint of an object, a second viewpoint at which the object is to be observed next in order to estimate a pose of the object, the viewpoint recommendation apparatus comprising:

a sensor configured to acquire an image of the object at the first viewpoint;

an image feature extractor configured to extract an image feature of the image of the object from the image of the object acquired by the sensor;

a pose estimation unit configured to calculate a first likelihood map indicating a relation between the estimated pose of the object estimated from the image of the object and a likelihood of this estimated pose based on the image feature of the object extracted by the image feature extractor;

a second storage configured to store a second likelihood map indicating a relation between the true first viewpoint in the estimated pose and a likelihood of this first viewpoint;

a third storage configured to store a third likelihood map indicating a relation between the pose of the object when the object is observed at the first viewpoint and the second viewpoint and a likelihood of this pose; and

a viewpoint estimation unit configured to estimate the second viewpoint so that a value of an evaluation function becomes the maximum or minimum, the evaluation function using, as a parameter, a result of multiplying and integrating the first likelihood map calculated by the pose estimation unit, the second likelihood map stored in the second storage, and the third likelihood map stored in the third storage.