Granted Patent B2

US 11,037,305 · App. 16/235,668 · Granted Jun 15, 2021

Inventors: Yifei Zhan (Beijing, CN); Miao Yan (Beijing, CN); Wang Zhou (Beijing, CN); Xiong Duan (Beijing, CN); Xianpeng Lang (Beijing, CN); Changjie Ma (Beijing, CN)

Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.

G06T7/246G01S7/4802G01S7/4808G01S17/58G01S17/89G01S17/931G06K9/00765G06K9/6218G01S17/86G06T2207/10028G06T2207/30241

Abstract

Claims (91)

1. A method for processing point cloud data, the method comprising:

acquiring a first frame and a second frame respectively from the point cloud data;

from a plurality of objects in the first and second frames, extracting a first candidate object in the first frame and a second candidate object corresponding to the first candidate object in the second frame, respectively;

determining a first location of the first candidate object and a second location of the second candidate object in a coordinate system of the point cloud data, respectively;

identifying any one of the first candidate object and the second candidate object as a moving object in response to determining an offset between the first location and the second location;

removing the moving object from the point cloud data; and

generating a three-dimensional map based on the point cloud data from which the moving object is removed,

wherein extracting the first candidate object and the second candidate object comprises:

detecting a first ground object in the first frame and a second ground object in the second frame, respectively;

comparing a first height of a first object in the first frame with a first ground object height of the first ground object, and comparing a second height of a second object in the second frame with a second ground object height of the second ground object;

determining objects connected to one of the first ground object in the first frame and the second ground object in the second frame based on the comparing;

forming a first set of candidate objects and a second set of candidate objects respectively based on the objects connected to one of the first ground object in the first frame and the second ground object in the second frame; and

selecting the first candidate object from the first set of candidate objects, and selecting the second candidate object corresponding to the first candidate object from the second set of candidate objects,

wherein the extracting reduces computation associated with identification of the moving object from the plurality of objects in the first and second frames, and

wherein the method is performed by at least one processor.

2. The method according to claim 1 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

selecting the second candidate object corresponding to the first candidate object based on a size ratio of the first candidate object to the second candidate object.

3. The method according to claim 1 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

acquiring a first image associated with the first frame and a second image associated with the second frame, respectively; and

selecting the second candidate object based on determining that the second candidate object in the second image matches the first candidate object in the first image.

4. The method according to claim 1 , wherein the identifying the first candidate object as the moving object comprises:

acquiring a third frame from the point cloud data;

extracting a third candidate object corresponding to the first candidate object in the third frame; and

identifying any one of the first candidate object, the second candidate object, and the third candidate object as the moving object in response to determining that a third location of the third candidate object in the coordinate system is associated with the offset.

5. The method according to claim 4 , wherein the identifying any one of the first candidate object, the second candidate object, and the third candidate object as the moving object comprises:

determining a predicted location of the third candidate object in the coordinate system; and

identifying any one of the first candidate object, the second candidate object, and the third candidate object as the moving object in response to the predicted location being consistent with the third location.

6. The method according to claim 5 , wherein the determining the predicted location of the third candidate object in the third frame comprises:

determining an object speed of the first candidate object based on the offset; and

determining the predicted location based on the object speed.

7. The method according to claim 1 , further comprising:

determining, based on the first candidate object and the second candidate object, a set of points associated with the moving object in the point cloud data; and

removing the set of points from the point cloud data.

8. The method according to claim 7 , wherein the determining the set of points associated with the moving object further comprises:

determining, in a plurality of frames of the point cloud data, the set of frames associated with the moving object;

determining a motion trajectory of the moving object based on locations of the moving object in the set of frames; and

determining the set of points associated with the moving object based on the motion trajectory.

9. An apparatus for processing point cloud data, the apparatus comprising:

at least one processor; and

a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring a first frame and a second frame respectively from the point cloud data;

from a plurality of objects in the first and second frames, extracting a first candidate object in the first frame and a second candidate object corresponding to the first candidate object in the second frame, respectively;

determining a first location of the first candidate object and a second location of the second candidate object in a coordinate system of the point cloud data, respectively;

identifying any one of the first candidate object and the second candidate object as a moving object in response to determining an offset between the first location and the second location;

removing the moving object from the point cloud data; and

generating a three-dimensional map based on the point cloud data from which the moving object is removed,

wherein extracting the first candidate object and the second candidate object comprises:

detecting a first ground object in the first frame and a second ground object in the second frame, respectively;

comparing a first height of a first object in the first frame with a first ground object height of the first ground object, and comparing a second height of a second object in the second frame with a second ground object height of the second ground object;

determining objects connected to one of the first ground object in the first frame and the second ground object in the second frame based on the comparing;

forming a first set of candidate objects and a second set of candidate objects respectively based on the objects connected to one of the first ground object in the first frame and the second ground object in the second frame; and

selecting the first candidate object from the first set of candidate objects, and selecting the second candidate object corresponding to the first candidate object from the second set of candidate objects,

wherein the extracting reduces computation associated with identification of the moving object from the plurality of objects in the first and second frames.

10. The apparatus according to claim 9 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

selecting the second candidate object corresponding to the first candidate object based on a size ratio of the first candidate object to the second candidate object.

11. The apparatus according to claim 9 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

acquiring a first image associated with the first frame and a second image associated with the second frame, respectively; and

selecting the second candidate object based on determining that the second candidate object in the second image matches the first candidate object in the first image.

12. The apparatus according to claim 9 , wherein the identifying the first candidate object as the moving object comprises:

acquiring a third frame from the point cloud data;

extracting a third candidate object corresponding to the first candidate object in the third frame; and

identifying the first candidate object as the moving object in response to determining that a third location of the third candidate object in the coordinate system is associated with the offset.

13. The apparatus according to claim 12 , wherein the identifying any one of the first candidate object, the second candidate object, and the third candidate object as the moving object comprises:

determining a predicted location of the third candidate object in the coordinate system; and

identifying any one of the first candidate object, the second candidate object, and the third candidate object as the moving object in response to the predicted location being consistent with the third location.

14. The apparatus according to claim 13 , wherein the determining the predicted location of the third candidate object in the third frame comprises:

determining an object speed of the first candidate object based on the offset; and

determining the predicted location based on the object speed.

15. The apparatus according to claim 9 , the operations further comprise:

determining, based on the first candidate object and the second candidate object, a set of points associated with the moving object in the point cloud data; and

removing the set of points from the point cloud data.

16. A non-transitory computer storage medium, storing a computer program thereon, the computer program, when executed by a processor, causes the processor to perform operations, the operations comprising:

acquiring a first frame and a second frame respectively from point cloud data;

from a plurality of objects in the first and second frames, extracting a first candidate object in the first frame and a second candidate object corresponding to the first candidate object in the second frame, respectively;

determining a first location of the first candidate object and a second location of the second candidate object in a coordinate system of the point cloud data, respectively;

identifying any one of the first candidate object and the second candidate object as a moving object in response to determining an offset between the first location and the second location;

removing the moving object from the point cloud data; and

generating a three-dimensional map based on the point cloud data from which the moving object is removed,

wherein extracting the first candidate object and the second candidate object comprises:

detecting a first ground object in the first frame and a second ground object in the second frame, respectively;

comparing a first height of a first object in the first frame with a first ground object height of the first ground object, and comparing a second height of a second object in the second frame with a second ground object height of the second ground object;

determining objects connected to one of the first ground object in the first frame and the second ground object in the second frame based on the comparing;

forming a first set of candidate objects and a second set of candidate objects respectively based on the objects connected to one of the first ground object in the first frame and the second ground object in the second frame; and

selecting the first candidate object from the first set of candidate objects, and selecting the second candidate object corresponding to the first candidate object from the second set of candidate objects,

wherein the extracting reduces computation associated with identification of the moving object from the plurality of objects in the first and second frames.

17. The method according to claim 1 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

selecting the second candidate object corresponding to the first candidate object based on a size ratio of the first candidate object to the second candidate object, a smoothness degree, a reflection intensity, a location and a point cloud density.

18. The apparatus according to claim 9 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

selecting the second candidate object corresponding to the first candidate object based on a size ratio of the first candidate object to the second candidate object, a smoothness degree, a reflection intensity, a location and a point cloud density.

19. The non-transitory computer storage medium according to claim 16 , wherein the selecting the second candidate object from the second set of candidate objects comprises:

selecting the second candidate object corresponding to the first candidate object based on a size ratio of the first candidate object to the second candidate object, a smoothness degree, a reflection intensity, a location and a point cloud density.

Assignments (9)

Priority Claims (1)

Continuity (1)