IP Library › Granted Patent US 10,930,062
Granted Patent B2
US 10,930,062 · App. 16/516,155 · Granted Feb 23, 2021

3D view model generation of an object utilizing geometrically diverse image clusters

Inventors: Jacob Wesely Gallaway (Melissa, TX); Jeremy Jens Gerhart (Richardson, TX); Stephen J. Raif (Sachse, TX); Jody Dale Verret (Rockwall, TX)
Assignee: Raytheon Company
G06T17/05G06T7/0002G06T7/32G06T7/60G06T7/97G06T15/06G06T15/20G06T2207/10028G06T2207/20221G06T2207/30168
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,930,062
App. No.
16/516,155
Filed
Jul 18, 2019
Granted
Feb 23, 2021
Kind
B2
Examiner
YANG, YI
Art Unit
2616
USPC
345/420
Abstract

A computer vision method, executed by one or more processors, for generating a single 3D model view of a geographic scene includes: receiving image data for the scene from a plurality of sensors located at different angles with respect to the geographic scene; dividing the image data into a plurality of image spatial regions; correlating the image data in each image spatial region to obtain a score for each image data in each image spatial region; grouping the image data in each image spatial region into two or more image clusters, based on the scores for each image; performing a multi-ray intersection within each image cluster to obtain a 3D reference point for each region; for each region, combining the one or more clusters, based on the 3D reference point for the region; and registering the combined clusters for each region to obtain a single 3D model view of the scene.

Claims (45)

1. A computer vision method, executed by one or more processors, for generating a single three-dimensional (3D) model view of a geographic scene, the method comprising:

receiving image data for the scene from a plurality of sensors located at different angles with respect to the geographic scene;

dividing the image data into a plurality of image spatial regions;

correlating the image data in each image spatial region to obtain a score for each image data in each image spatial region;

grouping the image data in each image spatial region into two or more image clusters, based on the scores for each image;

performing a multi-ray intersection within each image cluster to obtain a 3D reference point for each region;

for each region, combining the one or more clusters, based on the 3D reference point for the region; and

registering the combined clusters for each region to obtain a single 3D model view of the scene, wherein registering the combined clusters comprises:

rasterizing patches in two or more point clouds derived from the two or more image clusters;

determining 3D tie-points between the two or more point clouds via cross correlation processing;

performing a 3D bundle adjustment to determine misregistration error model parameters between the two or more point clouds;

applying the misregistration error model parameters to original point clouds, resulting in registered point clouds; and

merging the registered point clouds, resulting in a single, dense point cloud comprising of point clouds from each cluster.

2. The method of claim 1 , wherein the plurality of sensors are located on one or more platforms.

3. The method of claim 1 , wherein the plurality of image spatial regions are obtained by grouping the image data into coordinal and inter-coordinal map locations.

4. The method of claim 1 , wherein the image data is divided into the plurality of image spatial regions by selecting a reference image, finding correlation sets with higher strength that have diversity of convergence angles and ensuring image spatial regions have the largest convergence angles from all other groups.

5. The method of claim 1 , wherein correlating the image data further comprises geometrically analyzing each image spatial region and photogrammetrically analyzing each image spatial region to obtain a score for each image data in each image spatial region.

6. The method of claim 1 , wherein the score indicates a predicted quality of a point cloud generated from a corresponding image spatial region.

7. The method of claim 1 , wherein the multi-ray intersection utilizes a triangulation method.

8. The method of claim 1 , wherein the multi-ray intersection generates dense tie-points as a set of 1-to-N tie-points that resolve to an XYZ location in a 3D space to generate a point cloud from a reference image's perspective with a resolution half as dense as an original reference image.

9. The method of claim 1 , wherein combining the two or more clusters comprises merging multiple point clouds from each cluster perspective into a single point cloud representing the full scene.

10. The method of claim 1 , wherein correlating the image data utilizes a normalized-cross correlation score.

11. A computer vision system for generating a single three-dimensional (3D) model view of a geographic scene comprising:

one or more processors;

storage medium coupled to the one or more processors; and

I/O circuits, wherein the one or more processors:

receive image data for the scene from a plurality of sensors located at different angles with respect to the geographic scene;

divide the image data into a plurality of image spatial regions;

correlate the image data in each image spatial region to obtain a score for each image data in each image spatial region;

group the image data in each image spatial region into two or more image clusters, based on the scores for each image;

perform a multi-ray intersection within each image cluster to obtain a 3D reference point for each region;

for each region, combine the one or more clusters, based on the 3D reference point for the region; and

register the combined clusters for each region to obtain a single 3D model view of the scene, wherein registering the combined clusters comprises:

rasterizing patches in two or more point clouds derived from the two or more image clusters;

determining 3D tie-points between the two or more point clouds via cross correlation processing;

performing a 3D bundle adjustment to determine misregistration error model parameters between the two or more point clouds;

applying the misregistration error model parameters to original point clouds, resulting in registered point clouds; and

merging the registered point clouds, resulting in a single, dense point cloud comprising of point clouds from each cluster.

12. The system of claim 11 , wherein the plurality of sensors are located on one or more platforms.

13. The system of claim 11 , wherein the plurality of image spatial regions are obtained by grouping the image data into coordinal and inter-coordinal map locations.

14. The system of claim 11 , wherein the processor divides the image data into the plurality of image spatial regions by selecting a reference image, finding correlation sets with higher strength that have diversity of convergence angles and ensuring image spatial regions have the largest convergence angles from all other groups.

15. The system of claim 11 , wherein the processor correlates the image data by geometrically analyzing each image spatial region and photogrammetrically analyzing each image spatial region to obtain a score for each image data in each image spatial region.

16. The system of claim 11 , wherein the score indicates a predicted quality of a point cloud generated from a corresponding image spatial region.

17. The system of claim 11 , wherein the processor performs the multi-ray intersection to generate dense tie-points as a set of 1-to-N tie-points that resolve to an XYZ location in a 3D space to generate a point cloud from a reference image's perspective with a resolution half as dense as an original reference image.

18. The system of claim 11 , wherein the processor combines the two or more clusters by merging multiple point clouds from each cluster perspective into a single point cloud representing the full scene.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 18, 2019
From: GALLAWAY, JACOB WESELY; GERHART, JEREMY JENS; RAIF, STEPHEN J.; VERRET, JODY DALE
To: RAYTHEON COMPANY
Reel/Frame 049796/0395 →
Continuity (1)
Related Publication 20210019937A1 · Jan 21, 2021