Trans-spectral feature detection for volumetric image alignment and colorization
View Patent ↗Aligning and coloring a volumetric image, including: capturing intensity data using at least one scanner; generating an intensity image using the intensity data, wherein the intensity image includes at least one feature in a scene, the at least one feature including a sample feature; capturing image data using at least one camera, wherein the image data includes color information; generating a camera image using the image data, wherein the camera image includes the sample feature; matching the sample feature in the intensity image with the sample feature in the camera image to align the intensity image and the camera image; and generating a color image by applying the color information to the aligned intensity image.
1. A non-transitory computer-readable storage medium storing a computer program to align and color a volumetric image, the computer program comprising executable instructions that cause a computer to:
receive a 2-D color image and a 2-D intensity image;
convert the 2-D color image to a 2-D grayscale image;
perform a shape detection on the 2-D grayscale image and the 2-D intensity image to produce detected shapes;
perform a feature detection on the detected shapes to find matching shape points in the 2-D grayscale image and the 2-D intensity image;
calculate a phase correlation between the 2-D grayscale image and the 2-D intensity image;
transform the 2-D color image using the phase correlation to produce a correlated 2-D color image which has same orientation as the 2-D intensity image;
perform an initial image alignment including pre-processing for edges to produce a result;
perform finer feature detection based on the initial image alignment including feeding the result into a line detection processing to produce matched features;
extract matching feature points from the matched features;
calculate a final image transform using the matching shape points, the phase correlation, and the extracted matching feature points; and
align the 2-D color image to the 2-D intensity image using the final image transform.
2. The non-transitory computer-readable storage medium of claim 1 , further comprising executable instructions that cause the computer to
perform exposure stacking and color corrections on the 2-D color image.
3. The non-transitory computer-readable storage medium of claim 2 , further comprising executable instructions that cause the computer to:
generate one or more control points in the 2-D intensity image during the alignment of the volumetric image.
4. The non-transitory computer-readable storage medium of claim 1 , further comprising executable instructions that cause the computer to
capture and reproduce a 3-D intensity image of a scene from multiple locations using at least one scanner.
5. The non-transitory computer-readable storage medium of claim 4 , wherein the at least one scanner includes at least one Light Detection and Ranging (LIDAR) scanner.
6. The non-transitory computer-readable storage medium of claim 4 , further comprising executable instructions that cause the computer to
apply color information on the 3-D intensity image to produce the 2-D intensity image.
7. The non-transitory computer-readable storage medium of claim 4 , further comprising executable instructions that cause the computer to
capture the scene from multiple locations with multiple exposures using at least one camera to generate the 2-D color image.
8. The non-transitory computer-readable storage medium of claim 7 , wherein the at least one camera includes at least one high-dynamic-range (HDR) camera.
9. The non-transitory computer-readable storage medium of claim 1 , further comprising executable instructions that cause the computer to
perform correction of lens distortions in the 2-D color image during the image alignment.
10. The non-transitory computer-readable storage medium of claim 1 , further comprising executable instructions that cause the computer to:
send the 2-D color image and the 2-D intensity image to a cloud cluster, and
wherein the shape detection, the feature detection, the phase correlation, the 2-D color image transformation, the initial image alignment, the finer feature detection, the matching feature points extraction, and the final image transformation are performed in the cloud cluster to produce an output;
receive the output from the cloud cluster.