Method and apparatus for determining position and orientation of bucket of excavator
Embodiments of the present disclosure provide a method and apparatus for determining position and orientation of a bucket of an excavator, an electronic device and a computer readable medium. The method may include: acquiring an image of a bucket of an excavator collected by a camera provided on an excavator body, the image of the bucket including a preset marker provided on the bucket; determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.
1. A method for determining position and orientation of a bucket of an excavator, comprising:
acquiring an image of the bucket of the excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.
2. The method according to claim 1 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset marker in the image of the bucket; and
estimating the position and orientation information of the camera relative to the bucket on the basis of the three-dimensional coordinate of the feature point of the preset marker, the pixel coordinate of the feature point of the preset marker in the image of the bucket, and a pre-acquired internal parameter of the camera.
3. The method according to claim 2 , wherein the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker further comprises:
matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker.
4. The method according to claim 3 , wherein the matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker comprises:
acquiring angle sensing data sensed by angle sensors provided on an excavating arm of the excavator and the bucket;
determining a target position range of the preset marker in the image of the bucket on the basis of the angle sensing data; and
detecting marker information within the target position range, and matching in the preset marker information library to identify the preset marker.
5. The method according to claim 2 , wherein the preset marker comprises a preset QR code label, and the feature point of the preset marker comprises an angular point of the preset QR code label.
6. An apparatus for determining position and orientation of a bucket of an excavator, comprising:
at least one processor; and
a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring an image of the bucket of the excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.
7. The apparatus according to claim 6 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset marker in the image of the bucket; and
estimating the position and orientation information of the camera relative to the bucket on the basis of the three-dimensional coordinate of the feature point of the preset marker, the pixel coordinate of the feature point of the preset marker in the image of the bucket, and a pre-acquired internal parameter of the camera.
8. The apparatus according to claim 7 , wherein the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker further comprises:
matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker.
9. The apparatus according to claim 8 , wherein the matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker comprises:
acquiring angle sensing data sensed by angle sensors provided on an excavating arm of the excavator and the bucket;
determining a target position range of the preset marker in the image of the bucket on the basis of the angle sensing data; and
detecting marker information within the target position range, and matching in the preset marker information library to identify the preset marker.
10. The apparatus according to claim 7 , wherein the preset marker comprises a preset QR code label, and the feature point of the preset marker comprises an angular point of the preset QR code label.
11. A non-transitory computer readable medium, storing a computer program, wherein the program, when executed by a processor, causes the processor to perform operations, the operations comprising:
acquiring an image of a bucket of an excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.
12. The non-transitory computer readable medium according to claim 11 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset marker in the image of the bucket; and
estimating the position and orientation information of the camera relative to the bucket on the basis of the three-dimensional coordinate of the feature point of the preset marker, the pixel coordinate of the feature point of the preset marker in the image of the bucket, and a pre-acquired internal parameter of the camera.
13. The non-transitory computer readable medium according to claim 11 , wherein the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker further comprises:
matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker.
14. The non-transitory computer readable medium according to claim 13 , wherein the matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker comprises:
acquiring angle sensing data sensed by angle sensors provided on an excavating arm of the excavator and the bucket;
determining a target position range of the preset marker in the image of the bucket on the basis of the angle sensing data; and
detecting marker information within the target position range, and matching in the preset marker information library to identify the preset marker.
15. The non-transitory computer readable medium according to claim 11 , wherein the preset marker comprises a preset QR code label, and the feature point of the preset marker comprises an angular point of the preset QR code label.