IP Library Granted Patent US 11,080,919
Granted Patent B2
US 11,080,919 · App. 16/353,717 · Granted Aug 3, 2021

Method, apparatus, device and computer readable storage medium for reconstructing three-dimensional scene

Inventors: Li Yu (Beijing, CN); Shiyu Song (Beijing, CN)
Assignee: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
G06T15/04G06T2200/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,080,919
App. No.
16/353,717
Granted
Aug 3, 2021
Kind
B2
Abstract

Embodiments of the present disclosure relate to a method, apparatus, device and computer readable storage medium for reconstructing a three-dimensional scene. The method for reconstructing a three-dimensional scene includes acquiring a point cloud data frame set for a three-dimensional scene, point cloud data frames in the point cloud data frame set respectively having a pose parameter. The method further comprises determining a subset corresponding to a part of the three-dimensional scene from the point cloud data frame set. The method further comprises adjusting a pose parameter of a point cloud data frame in the subset to obtain an adjusted subset, the adjusted subset including at least two point cloud data frames having matching overlapping parts. The method further comprises updating the point cloud data frame set using the adjusted subset. In this way, distributed processing on a large amount of point cloud data may be realized.

Claims (71)

1. A method for reconstructing a three-dimensional scene, comprising:

acquiring a point cloud data frame set for the three-dimensional scene, point cloud data frames in the point cloud data frame set respectively having a pose parameter;

determining a subset corresponding to a part of the three-dimensional scene from the point cloud data frame set, comprising:

determining, based on trajectories corresponding to the point cloud data frame set, a position of an intersection in the three-dimensional scene, comprising: performing dilation on a trajectory image comprising the trajectories to determine a line representing a road by dilating trajectories corresponding to the road; determining, based on the line, at least one corner point at the intersection; and determining the position of the intersection based on the at least one corner point at the intersection;

forming, based on the position of the intersection, a connection diagram connecting the intersection; and

determining, based on the connection diagram, a closed path as the part of the three-dimensional scene;

updating the subset by removing data points associated with a dynamic object from point cloud data frames in the subset;

adjusting a pose parameter of a point cloud data frame in the updated subset to obtain an adjusted subset, the adjusted subset including at least two point cloud data frames having matching overlapping parts; and

updating the point cloud data frame set using the adjusted subset.

2. The method according to claim 1 , wherein the removing data points comprises:

removing data points representing a ground from the point cloud data frames;

recognizing the dynamic object in the point cloud data frames;

removing data points representing the dynamic object from the point cloud data frames; and

adding the data points representing the ground into the point cloud data frames.

3. The method according to claim 2 , wherein the recognizing the dynamic object comprises:

clustering point cloud data in the point cloud data frames to determine a plurality of clusters in the point cloud data frames; and

determining a cluster matching an attribute of the dynamic object from the plurality of clusters.

4. The method according to claim 1 , wherein the adjusting a pose parameter comprises:

extracting data points representing a static object from point cloud data frames in the subset, to update the subset; and

adjusting a pose parameter of a point cloud data frame in the updated subset.

5. The method according to claim 1 , wherein the at least two point cloud data frames include a first point cloud data frame and a second point cloud data frame, and the adjusting a pose parameter comprises:

determining a characteristic distance between corresponding objects in the first point cloud data frame and the second point cloud data frame; and

adjusting the pose parameter based on the characteristic distance.

6. The method according to claim 1 , wherein the updating the point cloud data frame set comprises:

determining another part of the three-dimensional scene, the another part overlapping the part of the three-dimensional scene;

determining another subset corresponding to the another part of the three-dimensional scene from the point cloud data frame set; and

adjusting a pose parameter of a point cloud data frame in the subset and a pose parameter of a point cloud data frame in the another subset, to cause a third point cloud data frame in the subset to overlap a fourth point cloud data frame in the another subset.

7. The method according to claim 1 , where the updating the point cloud data frame set comprises:

determining a pose parameter of a fifth point cloud data frame in the part through a global navigation satellite system; and

updating the point cloud data frame set based on the pose parameter of the fifth point cloud data frame.

8. An apparatus for reconstructing a three-dimensional scene, comprising:

at least one processor; and

a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring a point cloud data frame set for the three-dimensional scene, point cloud data frames in the point cloud data frame set respectively having a pose parameter;

determining a subset corresponding to a part of the three-dimensional scene from the point cloud data frame set, comprising:

determining, based on trajectories corresponding to the point cloud data frame set, a position of an intersection in the three-dimensional scene, comprising: performing dilation on a trajectory image comprising the trajectories to determine a line representing a road by dilating trajectories corresponding to the road; determining, based on the line, at least one corner point at the intersection; and determining the position of the intersection based on the at least one corner point at the intersection;

forming, based on the position of the intersection, a connection diagram connecting the intersection; and

determining, based on the connection diagram, a closed path as the part of the three-dimensional scene;

updating the subset by removing data points associated with a dynamic object from point cloud data frames in the subset;

adjusting a pose parameter of a point cloud data frame in the updated subset to obtain an adjusted subset, the adjusted subset including at least two point cloud data frames having matching overlapping parts; and

updating the point cloud data frame set using the adjusted subset.

9. The apparatus according to claim 8 , wherein the removing data points comprises:

removing data points representing a ground from the point cloud data frames;

recognizing the dynamic object in the point cloud data frames;

removing data points representing the dynamic object from the point cloud data frames; and

adding the data points representing the ground into the point cloud data frames,

wherein the recognizing the dynamic object comprises:

clustering point cloud data in the point cloud data frames to determine a plurality of clusters in the point cloud data frames; and

determining a cluster matching an attribute of the dynamic object from the plurality of clusters.

10. The apparatus according to claim 8 , wherein the adjusting a pose parameter comprises:

extracting data points representing a static object from point cloud data frames in the subset, to update the subset; and

adjusting a pose parameter of a point cloud data frame in the updated subset.

11. The apparatus according to claim 8 , wherein the at least two point cloud data frames include a first point cloud data frame and a second point cloud data frame, and the pose adjusting a pose parameter comprises:

determining a characteristic distance between corresponding objects in the first point cloud data frame and the second point cloud data frame; and

adjusting the pose parameter based on the characteristic distance.

12. The apparatus according to claim 8 , wherein the updating the point cloud data frame set comprises:

determining another part of the three-dimensional scene, the another part overlapping the part of the three-dimensional scene;

determining another subset corresponding to the another part of the three-dimensional scene from the point cloud data frame set; and

adjusting a pose parameter of a point cloud data frame in the subset and a pose parameter of a point cloud data frame in the another subset, to cause a third point cloud data frame in the subset to overlap a fourth point cloud data frame in the another subset.

13. The apparatus according to claim 8 , where the updating the point cloud data frame set comprises:

determining a pose parameter of a fifth point cloud data frame in the part through a global navigation satellite system; and

updating the point cloud data frame set based on the pose parameter of the fifth point cloud data frame.

14. A non-transitory computer readable storage medium, storing a computer program, wherein the program, when executed by a processor, causes the processor to perform operations, the operations comprising:

acquiring a point cloud data frame set for the three-dimensional scene, point cloud data frames in the point cloud data frame set respectively having a pose parameter;

determining a subset corresponding to a part of the three-dimensional scene from the point cloud data frame set, comprising:

determining, based on trajectories corresponding to the point cloud data frame set, a position of an intersection in the three-dimensional scene, comprising: performing dilation on a trajectory image comprising the trajectories to determine a line representing a road by dilating trajectories corresponding to the road; determining, based on the line, at least one corner point at the intersection; and determining the position of the intersection based on the at least one corner point at the intersection;

forming, based on the position of the intersection, a connection diagram connecting the intersection; and

determining, based on the connection diagram, a closed path as the part of the three-dimensional scene;

updating the subset by removing data points associated with a dynamic object from point cloud data frames in the subset;

adjusting a pose parameter of a point cloud data frame in the updated subset to obtain an adjusted subset, the adjusted subset including at least two point cloud data frames having matching overlapping parts; and

updating the point cloud data frame set using the adjusted subset.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 16, 2021
From: YU, LI; SONG, SHIYU
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 056877/0592 →
Priority Claims (1)
CN 201810565152.4 · Jun 4, 2018 · national
Continuity (1)
Related Publication 20190371044A1 · Dec 5, 2019