Automated graph local constellation (GLC) method of correspondence search for registration of 2-D and 3-D data

1. A method for registration of two and three-dimensional (3-D) images, the method comprising:

capturing a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;

transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image;

extracting 2-D shape properties from the 2-D object shape morphology;

scanning the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle to produce a 3-dimensional point cloud comprising 3-D cloud image points thereof;

generating a height slice of the 3-dimensional point cloud comprising 3-D coordinate and intensity measurements of the geographical region;

extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;

extracting slice shape properties from the slice object shape morphology;

constellation matching the 2-D image to the height slice based on the 2-D shape properties and the slice shape properties;

determining matching points between the 2-D image points and the 3-D cloud image points;

estimating a pose position of the camera attached to the first vehicle based on the matching points to provide an estimated pose position;

determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle; and

based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

2. The method of claim 1 , wherein the 3-D coordinate and intensity measurements comprise Light Detection and Ranging (LIDAR) measurements.

3. The method of claim 1 , further comprising fusing the 2-D image points and the 3-D cloud image points based on the estimated pose position.

4. The method of claim 1 , further comprising:

finding a plurality of shapes for a plurality of objects;

identifying the shapes for further extraction to produce identified shapes;

extracting properties of the identified shapes comprising shape corners and shape dimensions; and

matching a constellation of objects via matching LIDAR object to EO objects based on the properties of the identified shapes.

5. A system for registration of two and three-dimensional images, the system comprising:

a first vehicle system configured to image a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;

a second vehicle system configured to scan a 3-dimensional point cloud comprising 3-D cloud image points of the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle;

a processor;

memory storing instructions executable by the processor to perform functions comprising:

transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image;

extracting 2-D shape properties from the 2-D object shape morphology;

generating a height slice of the 3-dimensional (3-D) point cloud comprising LIDAR measurements of the geographical region;

extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;

extracting slice shape properties from the slice object shape morphology;

constellation matching the 2-D image to the height slice based on the 2-D shape properties and the slice shape properties;

matching points of the 2-D image and the 3-dimensional point cloud based on the 2-D shape properties and the slice shape properties to produce a plurality of matched points;

estimating a pose position of the camera based on the matched points to provide an estimated pose position;

determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle; and

based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

6. A non-transitory computer readable storage medium comprising computer-executable instructions for registration of two and three-dimensional (3-D) images, the computer-executable instructions comprising:

capturing a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;

transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image;

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image;

extracting 2-D shape properties from the 2-D object shape morphology;

scanning the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle to produce a 3-dimensional point cloud comprising 3-D cloud image points thereof;

generating a height slice of the 3-dimensional point cloud comprising LIDAR measurements of the geographical region;

extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;

extracting slice shape properties from the slice object shape morphology;

constellation matching the 2-dimensional (2-D) image to the height slice based on the 2-D shape properties and the slice shape properties;

determining matching points between 2-D image points and 3-D cloud image points to produce a plurality of matched points;

estimating a pose position of a camera based on the matched points to provide an estimated pose position;

determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle; and

based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

7. The non-transitory computer readable storage medium of claim 6 , further comprising computer-executable instructions for: fusing the 2-D image points and the 3-D cloud image points based on the estimated pose position.

8. The non-transitory computer readable storage medium of claim 6 , further comprising computer-executable instructions for:

finding a plurality of shapes for a plurality of objects;

identifying the shapes for further extraction to produce identified shapes;

extracting properties of the identified shapes comprising shape corners and shape dimensions; and

matching a constellation of objects via matching LIDAR object to EO objects based on the properties of the identified shapes.

9. The method of claim 1 , wherein the 3-dimensional point cloud is calibrated into world coordinates, and the method further comprises:

determining the geo-location of the first vehicle relative to the world coordinates of the matching points of the 3-D cloud image points of the 3-dimensional point cloud and the 2-D image points.

10. The system of claim 5 , wherein the 3-dimensional point cloud is calibrated into world coordinates, and the functions further comprise:

determining the geo-location of the first vehicle relative to the world coordinates of the matching points of the 3-D cloud image points of the 3-dimensional point cloud and the 2-D image points.

11. The non-transitory computer readable storage medium of claim 6 , wherein the 3-dimensional point cloud is calibrated into world coordinates, and further comprising computer-executable instructions for:

determining the geo-location of the first vehicle relative to the world coordinates of the matching points of the 3-D cloud image points of the 3-dimensional point cloud and the 2-D image points.

12. A method for registration of two and three-dimensional (3-D) images, the method comprising:

capturing a 2-dimensional (2-D) image comprising 2-D image points of a geographical region by a camera attached to a first vehicle;

scanning the geographical region by a Light Detection and Ranging (LIDAR) system attached to a second vehicle to produce a 3-dimensional point cloud comprising 3-D cloud image points thereof;

determining matching points between the 2-D image points and the 3-D cloud image points by;

finding a plurality of shapes for a plurality of objects;

identifying the shapes for further extraction to produce identified shapes;

extracting properties of the identified shapes comprising shape corners and shape dimensions; and

matching a constellation of objects via matching LIDAR object to 2D objects based on the properties of the identified shapes;

estimating a pose position of the camera attached to the first vehicle based on the matching points to provide an estimated pose position;

determining a geo-location of the first vehicle based on the estimated pose position of the camera attached to the first vehicle; and

based on the matching points, generating a colorized 3-dimensional point cloud wherein at least some of the 3-D cloud image points of the 3-dimensional point cloud are given a color of an object taken from the 2-D image.

13. The method of claim 12 , wherein the 3-dimensional point cloud is calibrated into world coordinates, and the method further comprises:

determining the geo-location of the first vehicle relative to the world coordinates of the matching points of the 3-D cloud image points of the 3-dimensional point cloud and the 2-D image points.

14. The method of claim 12 , further comprising fusing the 2-D image points and the 3-D cloud image points based on the estimated pose position.

15. The method of claim 12 , wherein determining matching points between the 2-D image points and the 3-D cloud image points comprises:

transforming the 2-dimensional (2-D) image of the geographical region via a regional maxima transform or an edge segmenting and boundary filling (ESBF) transform to produce a filtered 2-D image.

16. The method of claim 15 , wherein determining matching points between the 2-D image points and the 3-D cloud image points further comprises:

iteratively eroding and opening the filtered 2-D image to produce a processed electro-optical (EO) 2-D image;

extracting 2-D object shape morphology from the processed EO 2-D image; and

extracting 2-D shape properties from the 2-D object shape morphology.

17. The method of claim 12 , further comprising:

generating a height slice of the 3-dimensional point cloud comprising 3-D coordinate and intensity measurements of the geographical region;

extracting slice object shape morphology from the height slice resulting in labeled shapes found in the 2-D slice image;

extracting slice shape properties from the slice object shape morphology; and

constellation matching the 2-D image to the height slice.

18. The method of claim 17 , wherein the 3-D coordinate and intensity measurements comprise Light Detection and Ranging (LIDAR) measurements.

19. The method of claim 12 , wherein the 3-D cloud image points have an intensity corresponding to a reflectivity of an object represented by the 3-D cloud image points, and wherein generating the colorized 3-dimensional point cloud comprises:

generating an enhanced point cloud with true colors.