Method and device for acquiring 3D information of object

1. A method for acquiring three-dimensional (3D) information of an object, comprising:

acquiring an image comprising an object to be recognized, extracting two-dimensional (2D) key points of the object based on the image, and determining a candidate 3D model set matching the image;

for each candidate 3D model in the candidate 3D model set, determining 3D key points matching the 2D key points, and determining a plurality of first reference attitudes and positions of each candidate 3D model according to the 3D key points and the 2D key points;

acquiring a plurality of reprojection error values between each candidate 3D model and the object at the plurality of first reference attitudes and positions, and determining a first reprojection error value set corresponding to the candidate 3D model set;

determining that the 2D key points satisfy a preset sufficient constraint, comprising: determining that the 2D key points are not on a same face of the object;

determining a first target attitude and position and a first target 3D model corresponding to a minimum reprojection error value in the first reprojection error value set;

acquiring the 3D information of the object based on the first target attitude and position and the first target 3D model;

extracting a reference image of a reference object in the image according to a preset extraction strategy when determining that the 2D key points do not satisfy the preset sufficient constraint;

determining ground constraint information of the object according to the reference image, and determining a plurality of second reference attitudes and positions of each candidate 3D model according to the ground constraint information, the 2D key points and the 3D key points of each candidate 3D model;

acquiring a plurality of reprojection error values between each candidate 3D model and the object at the plurality of second reference attitudes and positions;

determining a second reprojection error value set corresponding to the candidate 3D model set, and determining a second target attitude and position and a second target 3D model corresponding to a minimum reprojection error value in the second reprojection error value set; and

acquiring the 3D information of the object according to the second target attitude and position and the second target 3D model.

2. The method according to claim 1 , wherein extracting the reference image of the reference object in the image according to the preset extraction strategy when determining that the 2D key points do not satisfy the preset sufficient constraint comprises:

recognizing a candidate reference image of each of a plurality of candidate reference objects in the image when the 2D key points are on the same face of the object;

recognizing 2D key points of each of the plurality of candidate reference images; and

determining a candidate reference object whose 2D key points are not on the same face of the candidate reference object as the reference object.

3. The method according to claim 2 , wherein when there are a plurality of reference objects, extracting the reference image of the reference object in the image comprises:

calculating an average value of 2D key points of each of the plurality of reference objects, to obtain a reference center point of each of the plurality of reference objects;

calculating an average value of 2D key points of the object, to obtain a center point of the object;

calculating an absolute value of a distance between the center point of the object and the reference center point of each of the plurality of reference objects, to obtain a plurality of absolute values;

comparing the plurality of absolute values, and determining a target reference object corresponding to a minimum absolute value, and

extracting the reference image corresponding to the target reference object.

4. The method according to claim 1 , wherein determining the ground constraint information of the object according to the reference image, and determining the plurality of second reference attitudes and positions of each candidate 3D model according to the ground constraint information, the 2D key points and the 3D key points of each candidate 3D model comprises:

acquiring height data of the reference image in a vertical direction and horizontal attitude data of the reference image in a horizontal direction;

determining a plurality of horizontal attitude estimation data of the object according to a preset estimation algorithm and the horizontal attitude data;

determining a normal rotation angle corresponding to each of the plurality of horizontal attitude estimation data according to a correspondence between the 2D key points and the 3D key points of each candidate 3D model; and

determining the plurality of second reference attitudes and positions of each candidate 3D model according to the plurality of horizontal attitude estimation data and the normal rotation angle corresponding to each horizontal attitude estimation data.

5. The method according to claim 4 , wherein determining the plurality of horizontal attitude estimation data of the object according to the preset estimation algorithm and the horizontal attitude data comprises:

acquiring first location information of the reference image and second location information of the image, and determining a search direction according to the first location information and the second location information; and

determining the plurality of horizontal attitude estimation data by taking the horizontal attitude data as an initial value, and performing an estimation along the search direction according to a least square algorithm.

6. The method according to claim 4 , wherein acquiring the plurality of reprojection error values between each candidate 3D model and the object at the plurality of second reference attitudes and positions comprises:

determining the normal rotation angle of each candidate 3D model at each of the plurality of second reference attitudes and positions; and

acquiring the reprojection error value between each candidate 3D model and the object at the normal rotation angle.

7. A device for acquiring 3D information of an object, comprising:

a processor; and

a memory, configured to store a computer program executable by the processor;

wherein when the computer program is executed by the processor, the processor is caused to:

acquire an image comprising an object to be recognized, and extract 2D key points of the object based on the image;

determine a candidate 3D model set matching the image;

determine 3D key points matching the 2D key points for each candidate 3D model in the candidate 3D model set;

determine a plurality of first reference attitudes and positions of each candidate 3D model according to the 3D key points and the 2D key points;

acquire a plurality of reprojection error values between each candidate 3D model and the object at the plurality of first reference attitudes and positions, and determine a first reprojection error value set corresponding to the candidate 3D model set;

determine that the 2D key points satisfy a preset sufficient constraint, comprising: determining that the 2D key points are not on a same face of the object;

determine a first target attitude and position and a first target 3D model corresponding to a minimum reprojection error value in the first reprojection error value set;

acquire the 3D information of the object based on the first target attitude and position and the first target 3D model;

extract a reference image of a reference object in the image according to a preset extraction strategy when determining that the 2D key points do not satisfy the preset sufficient constraint;

determine ground constraint information of the object according to the reference image, and determine a plurality of second reference attitudes and positions of each candidate 3D model according to the ground constraint information, the 2D key points and the 3D key points of each candidate 3D model;

acquire a plurality of reprojection error values between each candidate 3D model and the object at the plurality of second reference attitudes and positions;

determine a second reprojection error value set corresponding to the candidate 3D model set, and determine a second target attitude and position and a second target 3D model corresponding to a minimum reprojection error value in the second reprojection error value set; and

acquire the 3D information of the object according to the second target attitude and position and the second target 3D model.

8. The device according to claim 7 , wherein the processor is further configured to:

recognize a candidate reference image of each of a plurality of candidate reference objects in the image when the 2D key points are on the same face of the object;

recognize 2D key points of each of the plurality of candidate reference images; and

determine a candidate reference object whose 2D key points are not on the same face of the candidate reference object as the reference object.

9. The device according to claim 8 , wherein the processor is further configured to:

calculate an average value of 2D key points of each of the plurality of reference objects, to obtain a reference center point of each of the plurality of reference objects;

calculate an average value of 2D key points of the object, to obtain a center point of the object;

calculate an absolute value of a distance between the center point of the object and the reference center point of each of the plurality of reference objects, to obtain a plurality of absolute values;

compare the plurality of absolute values, and determine a target reference object corresponding to a minimum absolute value, and

extract the reference image corresponding to the target reference object.

10. The device according to claim 7 , wherein the processor is further configured to:

acquire height data of the reference image in a vertical direction and horizontal attitude data of the reference image in a horizontal direction;

determine a plurality of horizontal attitude estimation data of the object according to a preset estimation algorithm and the horizontal attitude data;

determine a normal rotation angle corresponding to each of the plurality of horizontal attitude estimation data according to a correspondence between the 2D key points and the 3D key points of each candidate 3D model; and

determine the plurality of second reference attitudes and positions of each candidate 3D model according to the plurality of horizontal attitude estimation data and the normal rotation angle corresponding to each horizontal attitude estimation data.

11. The device according to claim 10 , wherein the processor is further configured to:

acquire first location information of the reference image and second location information of the image, and determine a search direction according to the first location information and the second location information; and

determine the plurality of horizontal attitude estimation data by taking the horizontal attitude data as an initial value, and perform an estimation along the search direction according to a least square algorithm.

12. The device according to claim 10 , wherein the processor is further configured to:

determine the normal rotation angle of each candidate 3D model at each of the plurality of second reference attitudes and positions; and

acquire the reprojection error value between each candidate 3D model and the object at the normal rotation angle.

13. A non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, a method for acquiring 3D information of an object is implemented, the method comprising:

acquiring an image comprising an object to be recognized, extracting two-dimensional (2D) key points of the object based on the image, and determining a candidate 3D model set matching the image;

for each candidate 3D model in the candidate 3D model set, determining 3D key points matching the 2D key points, and determining a plurality of first reference attitudes and positions of each candidate 3D model according to the 3D key points and the 2D key points;

acquiring a plurality of reprojection error values between each candidate 3D model and the object at the plurality of first reference attitudes and positions, and determining a first reprojection error value set corresponding to the candidate 3D model set;

determining that the 2D key points satisfy a preset sufficient constraint, comprising: determining that the 2D key points are not on a same face of the object;

determining a first target attitude and position and a first target 3D model corresponding to a minimum reprojection error value in the first reprojection error value set;

acquiring the 3D information of the object based on the first target attitude and position and the first target 3D model;

extracting a reference image of a reference object in the image according to a preset extraction strategy when determining that the 2D key points do not satisfy the preset sufficient constraint;

determining ground constraint information of the object according to the reference image, and determining a plurality of second reference attitudes and positions of each candidate 3D model according to the ground constraint information, the 2D key points and the 3D key points of each candidate 3D model;

acquiring a plurality of reprojection error values between each candidate 3D model and the object at the plurality of second reference attitudes and positions;

determining a second reprojection error value set corresponding to the candidate 3D model set, and determining a second target attitude and position and a second target 3D model corresponding to a minimum reprojection error value in the second reprojection error value set; and

acquiring the 3D information of the object according to the second target attitude and position and the second target 3D model.