Three-dimensional posture estimating method and apparatus, device and computer storage medium

1. A three-dimensional posture estimating method, wherein the method comprises:

obtaining two-dimensional posture information of an object in an image and three-dimensional size information of the object, wherein the three-dimensional size information comprises length, width and height of the object;

determining coordinates of key points of the object in an object coordinate system according to the three-dimensional size information of the object, the coordinates of key points of the object are represented according to the length, width and height of the object; and

determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object,

wherein the two-dimensional posture information of the object in the image comprises: a minimum two-dimensional rectangular box in the image that can enclose the object;

the method further comprises: obtaining a rotation angle yaw of the object about the camera coordinate system based on a deep learning method, and

wherein the determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object comprises:

determining a rotation matrix R from the object coordinate system to the camera coordinate system according to the yaw;

determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, and

wherein the determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, comprises:

with respect to sides of the minimum two-dimensional rectangular box, using the R and internal parameters of the camera to form four groups of equations, each group of equation reflecting a positional relationship between key points falling at the minimum two-dimensional rectangular box and the two-dimensional rectangular box;

using a least square method to solve the four groups of equations, and determining the t.

2. The method according to claim 1 , wherein the method further comprises:

determining coordinates of the key points of the object in the camera coordinate system, according to coordinates of the key points of the object in the object coordinate system and the transformation relationship.

3. The method according to claim 2 , wherein the key points comprise: apexes of a minimum three-dimensional rectangular box that encloses the object.

4. The method according to claim 1 , wherein the two-dimensional posture information of the object in the image and the three-dimensional size information of the object are obtained based on a deep learning method.

5. The method according to claim 1 , wherein the two-dimensional posture information of the object in the image comprises: projection coordinates of the key points of the object on the image.

6. The method according to claim 5 , wherein the determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object comprises:

enabling the coordinates of the key points of the object in the object coordinate system and the projection coordinates on the image to respectively constitute 3D-2D coordinate pairs of respective key points;

using a geometrical correspondence relationship of 3D-2D coordinate pairs of at least partial key points to determine a rotation matrix R and a position translation vector t from the object coordinate system to the camera coordinate system.

7. The method according to claim 6 , wherein the using a geometrical correspondence relationship of 3D-2D coordinate pairs of at least partial key points to determine a rotation matrix R and a position translation vector t from the object coordinate system to the camera coordinate system comprises:

using the geometrical correspondence relationship of 3D-2D coordinate pairs of m key points to form m groups of equations, m being a positive integer;

using a PnP algorithm to solve said m groups of equations, to obtain the rotation matrix R and the position translation vector t from the object coordinate system to the camera coordinate system, wherein the object is represented with n points in a three-dimensional space and m≥n.

8. The method according to claim 7 , wherein during use of the PnP algorithm to solve said m groups of equations, a Random Sample Consensus RANSAC algorithm is employed to select a solution with a maximum interior point rate.

9. The method according to claim 1 , wherein the object comprises: an obstacle.

10. A device, wherein the device comprises:

one or more processors,

a storage for storing one or more programs,

the one or more programs, when executed by said one or more processors, enable said one or more processors to implement a three-dimensional posture estimating method, wherein the method comprises:

obtaining two-dimensional posture information of an object in an image and three-dimensional size information of the object, wherein the three-dimensional size information comprises length, width and height of the object;

determining coordinates of key points of the object in an object coordinate system according to the three-dimensional size information of the object, the coordinates of key points of the object are represented according to the length, width and height of the object; and

determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object,

wherein the two-dimensional posture information of the object in the image comprises: a minimum two-dimensional rectangular box in the image that can enclose the object;

the method further comprises: obtaining a rotation angle yaw of the object about the camera coordinate system based on a deep learning method, and

wherein the determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object comprises:

determining a rotation matrix R from the object coordinate system to the camera coordinate system according to the yaw;

determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, and

wherein the determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, comprises:

with respect to sides of the minimum two-dimensional rectangular box, using the R and internal parameters of the camera to form four groups of equations, each group of equation reflecting a positional relationship between key points falling at the minimum two-dimensional rectangular box and the two-dimensional rectangular box;

using a least square method to solve the four groups of equations, and determining the t.

11. The device according to claim 10 , wherein the method further comprises:

determining coordinates of the key points of the object in the camera coordinate system, according to coordinates of the key points of the object in the object coordinate system and the transformation relationship.

12. The device according to claim 10 , wherein the two-dimensional posture information of the object in the image and the three-dimensional size information of the object are obtained based on a deep learning method.

13. The device according to claim 10 , wherein the two-dimensional posture information of the object in the image comprises: projection coordinates of the key points of the object on the image.

14. The device according to claim 13 , wherein the determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object comprises:

enabling the coordinates of the key points of the object in the object coordinate system and the projection coordinates on the image to respectively constitute 3D-2D coordinate pairs of respective key points;

using a geometrical correspondence relationship of 3D-2D coordinate pairs of at least partial key points to determine a rotation matrix R and a position translation vector t from the object coordinate system to the camera coordinate system.

15. The device according to claim 14 , wherein the using a geometrical correspondence relationship of 3D-2D coordinate pairs of at least partial key points to determine a rotation matrix R and a position translation vector t from the object coordinate system to the camera coordinate system comprises:

using the geometrical correspondence relationship of 3D-2D coordinate pairs of m key points to form m groups of equations, m being a positive integer;

using a PnP algorithm to solve said m groups of equations, to obtain the rotation matrix R and the position translation vector t from the object coordinate system to the camera coordinate system, wherein the object is represented with n points in a three-dimensional space and m≥n.

16. The device according to claim 15 , wherein during use of the PnP algorithm to solve said m groups of equations, a Random Sample Consensus RANSAC algorithm is employed to select a solution with a maximum interior point rate.

17. A non-transitory storage medium including computer-executable instructions, the computer-executable instructions, when executed by a computer processor, being used to execute a three-dimensional posture estimating method, wherein the method comprises:

obtaining two-dimensional posture information of an object in an image and three-dimensional size information of the object, wherein the three-dimensional size information comprises length, width and height of the object;

determining coordinates of key points of the object in an object coordinate system according to the three-dimensional size information of the object, the coordinates of key points of the object are represented according to the length, width and height of the object; and

determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object,

wherein the two-dimensional posture information of the object in the image comprises: a minimum two-dimensional rectangular box in the image that can enclose the object;

the method further comprises: obtaining a rotation angle yaw of the object about the camera coordinate system based on a deep learning method, and

wherein the determining a transformation relationship between a camera coordinate system and the object coordinate system according to a geometrical relationship between coordinates of key points of the object in the object coordinate system and the two-dimensional posture information of the object comprises:

determining a rotation matrix R from the object coordinate system to the camera coordinate system according to the yaw;

determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, and

wherein the determining a position translation vector t from the object coordinate system to the camera coordinate system, according to a coordinate relationship between the minimum two-dimensional rectangular box and key points of the object falling at the minimum two-dimensional rectangular box, and the R, comprises:

with respect to sides of the minimum two-dimensional rectangular box, using the R and internal parameters of the camera to form four groups of equations, each group of equation reflecting a positional relationship between key points falling at the minimum two-dimensional rectangular box and the two-dimensional rectangular box;

using a least square method to solve the four groups of equations, and determining the t.