Estimation device and method
According to one embodiment, an estimation device includes a processor. The processor is configured to acquire first and second images including a same subject continuously captured by a capture device installed in a mobile object, acquire a first distance to the subject included in the acquired first image and uncertainty of the first distance, extract a useful second distance from the acquired first distance based on the acquired uncertainty of the first distance, and estimate a position and orientation of the mobile object when the second image is captured by the capture device based on the acquired first and second images, and the extracted second distance.
1. An estimation device comprising:
a processor configured to:
acquire first and second images including a same subject continuously captured by a capture device installed in a mobile object;
acquire a first distance to the subject included in the acquired first image and uncertainty of the first distance;
extract a second distance from the acquired first distance based on the acquired uncertainty of the first distance; and
estimate a position and orientation of the mobile object when the second image is captured by the capture device based on the acquired first and second images, and the extracted second distance.
2. The estimation device according to claim 1 , wherein the processor is configured to:
acquire a bokeh value indicating bokeh occurring in the subject in the first image and uncertainty of the bokeh value as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the bokeh value into the first distance corresponding to the bokeh value; and
convert the uncertainty of the bokeh value into the uncertainty of the first distance corresponding to the bokeh value.
3. The estimation device according to claim 2 , wherein when the first image is input into a statistical model generated by learning bokeh that occurs in an image affected by an aberration of an optical system of the capture device and varies non-linearly in accordance with a distance to the subject included in the image, the bokeh value and the uncertainty of the bokeh value are output from the statistical model.
4. The estimation device according to claim 1 , wherein the processor is configured to:
acquire a reflected wave from the subject and intensity of the reflected wave as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the reflected wave into the first distance to the subject corresponding to the reflected wave; and
convert the intensity of the reflected wave into the uncertainty of the first distance converted from the reflected wave.
5. The estimation device according to claim 1 , wherein the processor is configured to:
acquire disparity in an image including the subject captured from a plurality of viewpoints by at least two capture devices and uncertainty of the disparity as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the disparity into the first distance corresponding to the disparity; and
convert the uncertainty of the disparity into the uncertainty of the first distance corresponding to the disparity.
6. The estimation device according to claim 2 , wherein the processor is configured to extract the second distance based on at least one of the uncertainty of the first distance, the first distance, the distance information, the uncertainty of the distance information, and luminance of the first image.
7. The estimation device according to claim 1 , wherein the processor is configured to extract, as the second distance, the first distance in which the uncertainty is less than a predetermined value.
8. The estimation device according to claim 1 , wherein the processor is configured to:
detect a plurality of first feature points from the first image and detect a plurality of second feature points from the second image;
acquire the first distance to the subject and the uncertainty of the first distance for each of the first feature points;
extract the second distance from the first distance acquired for each of the first feature points based on the acquired uncertainty of the first distance;
calculate a three-dimensional coordinate of the first feature point based on a real scale based on the extracted second distance; and
estimate the position and orientation of the mobile object based on the real scale by associating the first feature point for which the three-dimensional coordinate is calculated with the second feature point detected from the second image.
9. An estimation device comprising:
a processor configured to:
acquire first and second images including a same subject continuously captured by a capture device installed in a mobile object;
estimate a position and orientation of the mobile object with an indeterminate scale when the second image is captured by the capture device by detecting a plurality of feature points from the acquired first and second images and associating the feature points detected from the first and second images;
acquire a first distance to the subject included in the acquired second image and uncertainty of the first distance;
extract a second distance from the acquired first distance based on the acquired uncertainty of the first distance; and
correct, based on the extracted second distance, the estimated position and orientation of the mobile object with the indeterminate scale to the position and orientation of the mobile object based on a real scale.
10. The estimation device according to claim 9 , wherein the processor is configured to:
acquire a bokeh value indicating bokeh occurring in the subject in the second image and uncertainty of the bokeh value as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the bokeh value into the first distance corresponding to the bokeh value; and
convert the uncertainty of the bokeh value into the uncertainty of the first distance corresponding to the bokeh value.
11. The estimation device according to claim 10 , wherein when the second image is input into a statistical model generated by learning bokeh that occurs in an image affected by an aberration of an optical system of the capture device and varies non-linearly in accordance with a distance to the subject included in the image, the bokeh value and the uncertainty of the bokeh value are output from the statistical model.
12. The estimation device according to claim 9 , wherein the processor is configured to:
acquire a reflected wave from the subject and intensity of the reflected wave as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the reflected wave into the first distance to the subject corresponding to the reflected wave; and
convert the intensity of the reflected wave into the uncertainty of the first distance converted from the reflected wave.
13. The estimation device according to claim 9 , wherein the processor is configured to:
acquire disparity in an image including the subject captured from a plurality of viewpoints by at least two capture devices and uncertainty of the disparity as distance information regarding the first distance to the subject and uncertainty of the distance information;
convert the disparity into the first distance corresponding to the disparity; and
convert the uncertainty of the disparity into the uncertainty of the first distance corresponding to the disparity.
14. The estimation device according to claim 10 , wherein the processor is configured to extract the second distance based on at least one of the uncertainty of the first distance, the first distance, the distance information, the uncertainty of the distance information, and luminance of the first image.
15. The estimation device according to claim 10 , wherein the processor is configured to extract, as the second distance, the first distance in which the uncertainty is less than a predetermined value.
16. A method, executed by an estimation device, comprising:
acquiring first and second images including a same subject continuously captured by a capture device installed in a mobile object;
acquiring a first distance to the subject included in the acquired first image and uncertainty of the first distance;
extracting a second distance from the acquired first distance based on the acquired uncertainty of the first distance; and
estimating a position and orientation of the mobile object when the second image is captured by the capture device based on the acquired first and second images, and the extracted second distance.
17. The method according to claim 16 , wherein the acquiring of the first distance and the uncertainty of the first distance includes:
acquiring a bokeh value indicating bokeh occurring in the subject in the first image and uncertainty of the bokeh value as distance information regarding the first distance to the subject and uncertainty of the distance information;
converting the bokeh value into the first distance corresponding to the bokeh value; and
converting the uncertainty of the bokeh value into the uncertainty of the first distance corresponding to the bokeh value.
18. The method according to claim 17 , wherein when the first image is input into a statistical model generated by learning bokeh that occurs in an image affected by an aberration of an optical system of the capture device and varies non-linearly in accordance with a distance to the subject included in the image, the bokeh value and the uncertainty of the bokeh value are output from the statistical model.
19. The method according to claim 16 , wherein the acquiring of the first distance and the uncertainty of the first distance includes:
acquiring a reflected wave from the subject and intensity of the reflected wave as distance information regarding the first distance to the subject and uncertainty of the distance information;
converting the reflected wave into the first distance to the subject corresponding to the reflected wave; and
converting the intensity of the reflected wave into the uncertainty of the first distance converted from the reflected wave.
20. The method according to claim 16 , wherein the acquiring of the first distance and the uncertainty of the first distance includes:
acquiring disparity in an image including the subject captured from a plurality of viewpoints by at least two capture devices and uncertainty of the disparity as distance information regarding the first distance to the subject and uncertainty of the distance information;
converting the disparity into the first distance corresponding to the disparity; and
converting the uncertainty of the disparity into the uncertainty of the first distance corresponding to the disparity.