Sparse image point correspondences generation and correspondences refinement system for ground truth static scene sparse flow generation

1. A system for generating a ground truth dataset for motion planning, the system comprising:

an internet server, comprising:

an I/O port, configured to transmit and receive electrical signals to and from a client device;

a memory;

one or more processing units; and

one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for:

a static point extracting module configured to generate a LiDAR static-scene point cloud based on undistorted LiDAR scans, wherein the static point extracting module is configured to execute the following steps in order to generate a LiDAR static-scene point cloud:

a) identify points belonging to a static object in an environment for a pair of undistorted LiDAR scans;

b) align close points based on GNSS-inertial estimates;

c) transform a reference scan that is close in time to a target undistorted LiDAR scan so as to align the reference scan with the target undistorted LiDAR scan;

d) determine that a distance between a point in the target undistorted LiDAR scan and its closest point in the aligned reference scan is smaller than a threshold; and

e) extract the point from the target undistorted LiDAR scan in order to generate the LiDAR static-scene point cloud;

a corresponding module configured to correspond, for each pair of images, a first image of the pair to the LiDAR static-scene point cloud for the motion planning of a vehicle; and

a computing module configured to compute a camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud.

2. The system according to claim 1 , wherein the computing module is configured to compute the camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud as extrinsic parameters are sent from GNSS-insertion estimates.

3. The system according to claim 1 further comprising:

a first generating module configured to generate, for each 3D point in the LiDAR static-scene point cloud, two image points in sub-pixel accuracy.

4. The system according to claim 3 , wherein the first generating module is configured to generate the two image points in sub-pixel accuracy by projecting the 3D point onto the pair of images.

5. The system according to claim 3 further comprising:

an establishing module configured to establish a correspondence between the two image points.

6. The system according to claim 5 further comprising:

a second generating module configured to generate sparse image point correspondences in the pair of images.

7. The system according to claim 6 further comprising:

a comparing module configured to compare local image patches near correspondence image points.

8. The system according to claim 7 further comprising:

a rejecting module configured to reject correspondences that have a significant difference in the local image patches.

9. The system according to claim 8 further comprising:

a storing module configured to store sparse image point correspondences as a ground truth for a static-scene dataset for motion planning.

10. A system having a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions which, when executed by the one or more processors of a computing device, cause the computing device to:

generate a LiDAR static-scene point cloud based on undistorted LiDAR scans, wherein following steps are included:

a) identify points belonging to a static object in an environment for a pair of undistorted LiDAR scans;

b) align close points based on GNSS-inertial estimates;

c) transform a reference scan that is close in time to a target undistorted LiDAR scan so as to align the reference scan with the target undistorted LiDAR scan;

d) determine that a distance between a point in the target undistorted LiDAR scan and its closest point in the aligned reference scan is smaller than a threshold; and

e) extract the point from the target undistorted LiDAR scan in order to generate the LiDAR static-scene point cloud;

correspond, for each pair of images, a first image of the pair to the LiDAR static-scene point cloud for a motion planning of a vehicle; and

compute a camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud and align close points based on pose estimates.

11. The system according to claim 10 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

compute the camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud as extrinsic parameters are sent from GNSS-insertion estimates.

12. The system according to claim 10 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate, for each 3D point in the LiDAR static-scene point cloud, two image points in sub-pixel accuracy.

13. The system according to claim 12 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate the two image points in sub-pixel accuracy by projecting the 3D point onto the pair of images.

14. The system according to claim 12 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

establish a correspondence between the two image points.

15. The system according to claim 14 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate sparse image point correspondences in the pair of images.

16. The system according to claim 15 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

compare local image patches near correspondence image points.

17. The system according to claim 16 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

reject correspondences that have a significant difference in the local image patches.

18. The system according to claim 17 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

store sparse image point correspondences as a ground truth for a static-scene dataset for motion planning.

19. A method of generating a ground truth dataset for motion planning for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions which, when executed by the one or more processors of a computing device, cause the computing device to:

generate a LiDAR static-scene point cloud based on undistorted LiDAR scans, wherein following steps are included:

a) identify points belonging to a static object in an environment for a pair of undistorted LiDAR scans;

b) align close points based on GNSS-inertial estimates;

c) transform a reference scan that is close in time to a target undistorted LiDAR scan so as to align the reference scan with the target undistorted LiDAR scan;

d) determine that a distance between a point in the target undistorted LiDAR scan and its closest point in the aligned reference scan is smaller than a threshold; and

e) extract the point from the target undistorted LiDAR scan in order to generate the LiDAR static-scene point cloud;

correspond, for each pair of images, a first image of the pair to the LiDAR static-scene point cloud for a motion planning of a vehicle; and

compute a camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud and align close points based on pose estimates.

20. The method according to claim 19 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

compute the camera pose associated with the pair of images in the coordinate of the LiDAR static-scene point cloud as extrinsic parameters are sent from GNSS-insertion estimates.

21. The method according to claim 19 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate, for each 3D point in the LiDAR static-scene point cloud, two image points in sub-pixel accuracy.

22. The system according to claim 21 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate the two image points in sub-pixel accuracy by projecting the 3D point onto the pair of images.

23. The method according to claim 21 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

establish a correspondence between the two image points.

24. The method according to claim 23 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

generate sparse image point correspondences in the pair of images.

25. The method according to claim 24 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

compare local image patches near correspondence image points.

26. The method according to claim 25 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

reject correspondences that have a significant difference in the local image patches.

27. The method according to claim 26 , further comprising instructions which, when executed by the one or more processors, cause the computing device to:

store sparse image point correspondences as a ground truth for a static-scene dataset for motion planning.