Information processing apparatus, three-dimensional position estimation method, and a non-transitory computer-readable medium
An information processing apparatus ( 10 ) includes: a detection unit ( 11 ) configured to detect, using an image and forks for supporting the transport pallet are captured, a shape of fork insertion slots into which the forks are inserted and a shape of the forks; an acquisition unit ( 12 ) configured to acquire a depth image showing depth information of the fork insertion slots and depth information of the forks in the image; a correction unit ( 13 ) configured to correct, using the shape of the fork insertion slots and the shape of the forks that are detected, the depth information so that the shape of the fork insertion slots and the shape of the forks can be recognized in the depth image; and a position estimation unit ( 14 ) configured to estimate three-dimensional positions of the fork insertion slots and three-dimensional positions of the forks based on the depth information.
1 . An information processing apparatus comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
detect, using an image in which a transport pallet and forks of a forklift for supporting the transport pallet are captured, a shape of fork insertion slots of the transport pallet into which the forks are inserted and a shape of the forks;
acquire a depth image showing depth information of the fork insertion slots and depth information of the forks in the image;
correct, using the shape of the fork insertion slots and the shape of the forks that are detected, the depth information so that the shape of the fork insertion slots and the shape of the forks can be recognized in the depth image; and
estimate three-dimensional positions of the fork insertion slots and three-dimensional positions of the forks based on the depth information,
wherein, in the case where measurement for depth information at positions representing the shape of the fork insertion slots and the shape of the forks in the depth image is not performed, the at least one processor is further configured to execute the instructions to estimate the depth information at the positions using depth information in the periphery of the positions, and
wherein, in the case where measurement for the depth information at positions representing the shape of the fork insertion slots is not performed, the at least one processor is further configured to execute the instructions to explore for depth information in an outward direction of the fork insertion slots from an arbitrary position inside the insertion slots and estimate, using the obtained depth information, depth information at positions where depth information has not been measured.
2 . The information processing apparatus according to claim 1 , wherein, in the case where measurement for the depth information at positions representing the shape of the forks is not performed, the at least one processor is further configured to execute the instructions to explore for depth information in an inward direction of the forks of the forklift from the positions where the depth information has not been measured and estimate, using the obtained depth information, the depth information at positions where depth information have not been measured.
3 . The information processing apparatus according to claim 1 , wherein the at least one processor is further configured to execute the instructions to detect the shape of the fork insertion slots of the transport pallet and the shape of the forks of the forklift by comparing feature points extracted from a peripheral image taken in advance using a camera that took the image and feature points extracted from the image.
4 . The information processing apparatus according to claim 3 , wherein the at least one processor is further configured to execute the instructions to detect the shape of the fork insertion slots and the shape of the forks by performing edge detection with hysteresis.
5 . The information processing apparatus according to- claim 1 , wherein the at least one processor is further configured to execute the instructions to:
specify the two dimensional coordinates of a plurality of feature points representing the shape of the fork insertion slots of the transportation pallet and the shape of the forks; and
correct depth information of the feature points whose depth information is unknown among the plurality of the feature points.
6 . The information processing apparatus according to- claim 1 , the at least one processor is further configured to execute the instructions to:
detect a marker affixed to the transport pallet in the image and estimate external parameters for the camera that took the image; and
estimate the three-dimensional positions of the fork insertion slots and the forks based on the depth information and the external parameters.
7 . The information processing apparatus according to claim 1 , wherein the depth image is contained in an RGB-D image taken using an RGB-D camera.
8 . A three-dimensional position estimation method performed by an information processing apparatus, the method comprising:
detecting, using an image in which a transport pallet and forks of a forklift for supporting the transport pallet are captured, a shape of fork insertion slots of the transport pallet into which the forks are inserted and a shape of the forks;
acquiring a depth image showing depth information of the fork insertion slots and the forks in the image;
correcting, using the shape of the fork insertion slots and the shape of the forks that are detected, the depth information so that the shape of the fork insertion slots and the shape of the forks can be recognized in the depth image;
estimating three-dimensional positions of the fork insertion slots and three-dimensional positions of the forks based on the depth information;
in the case where measurement for depth information at positions representing the shape of the fork insertion slots and the shape of the forks in the depth image is not performed, estimating the depth information at the positions using depth information in the periphery of the positions; and
in the case where measurement for the depth information at positions representing the shape of the fork insertion slots is not performed, exploring for depth information in an outward direction of the fork insertion slots from an arbitrary position inside the insertion slots and estimating, using the obtained depth information, depth information at positions where depth information has not been measured.
9 . A non-transitory computer readable-medium storing a program for causing a computer to execute operations comprising:
detecting, using an image in which a transport pallet and forks of a forklift for supporting the transport pallet are captured, a shape of fork insertion slots of the transport pallet into which the forks are inserted and a shape of the forks;
acquiring a depth image showing depth information of the fork insertion slots and the forks in the image;
correcting, using the shape of the fork insertion slots and the shape of the forks that are detected, the depth information so that the shape of the fork insertion slots and the shape of the forks can be recognized in the depth image; and
estimating three-dimensional positions of the fork insertion slots and three-dimensional positions of the forks based on the depth information;
in the case where measurement for depth information at positions representing the shape of the fork insertion slots and the shape of the forks in the depth image is not performed, estimating the depth information at the positions using depth information in the periphery of the positions; and
in the case where measurement for the depth information at positions representing the shape of the fork insertion slots is not performed, exploring for depth information in an outward direction of the fork insertion slots from an arbitrary position inside the insertion slots and estimating, using the obtained depth information, depth information at positions where depth information has not been measured.