IP Library Granted Patent US 10,157,479
Granted Patent B2
US 10,157,479 · App. 15/709,252 · Granted Dec 18, 2018

Synthesizing training data for broad area geospatial object detection

Inventors: Adam Estrada (Bethesda, MD); Christopher Burd (Washington, DC); Andrew Jenkins (Waterford, VA); Joseph Newbrough (Springfield, VA); Scott Szoko (Odenton, MD); Melanie Vinton (Fairfax Station, VA)
Assignee: DigitalGlobe, Inc.
G06T7/75G06K9/00637G06K9/6256G06K9/6267G06N3/04G06N3/0454G06N3/08G06T5/009G06T7/73G06T7/90G06T15/50G06T17/05G06T2207/10032G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,157,479
App. No.
15/709,252
Granted
Dec 18, 2018
Kind
B2
Abstract

A system for broad area geospatial object recognition, identification, classification, location and quantification, comprising an image manipulation module to create synthetically-generated images to imitate and augment an existing quantity of orthorectified geospatial images; together with a deep learning module and a convolutional neural network serving as an image analysis module, to analyze a large corpus of orthorectified geospatial images, identify and demarcate a searched object of interest from within the corpus, locate and quantify the identified or classified objects from the corpus of geospatial imagery available to the system. The system reports results in a requestor's preferred format.

Claims (21)

1. A system for broad area geospatial object detection using synthetically-generated training images for improved training of a deep learning model comprising a computing device comprising a processor, a memory, and a plurality of programming instructions stored in the memory and operable on the processor, wherein the plurality of programming instructions:

(a) retrieves a digital model of an object of interest from a data store;

(b) creates a flattened image from the retrieved digital model;

(c) compares the flattened image to a real geospatial image comprising an instance of the object of interest and associated background;

(d) scales the flattened image to align with the real geospatial image of the instance of the object of interest and separates the flattened image from the background of the real image;

(e) applies a plurality of environmental effects to replicate seasonal, time of day, associated brightness, and environmental factors consistent with a geographic location of the real background image to create a plurality of modified synthetic images;

(f) creates a plurality of shadowed, modified flattened images for the object of interest as if it were physically located and oriented where it would be affected by real-time and real-world shadowing;

(g) adjusts the shadowed, modified flattened images to resemble the real image;

(h) identifies and demarcates a footprint associated with each of the shadowed, modified flattened images;

(i) overlays the demarcated footprint onto a real image;

(j) generates a labeled corpus of manipulated synthetic training data comprising at least a portion of the modified flattened images; and

(k) trains a deep learning model comprising a convolutional neural network to recognize objects of the same type as the object of interest in geospatial images.

2. The system of claim 1 , further comprising an image analysis server comprising a second processor, a second memory, and a second plurality of programming instructions stored in the second memory and operable on the second processor, wherein the second plurality of programming instructions:

(a) uses the deep learning model to automatically identify and label all objects of interest in a received data set comprising a plurality of unanalyzed orthorectified geospatial imagery; and

(b) outputs the locations of the identified objects of interest.

3. A method for identifying objects of interest in geospatial images using a deep learning model and synthetically-generated training images the method comprising the steps of:

(a) automatically generating, using an image manipulation computer, a flattened image of an object of interest from a digital model of the object of interest;

(b) manipulating the flattened image to create a plurality of synthetic images comprising at least one of the object of interest, placing the object of interest in the plurality of synthetic images in a plurality of locations, environments, orientations, scales, exposures, and foci in order to create a large corpus of synthetically-generated training images;

(c) training, using the large corpus of synthetically-generated training images, a deep learning model comprising a convolutional neural network to recognize objects of the same type as the object of interest in a plurality of unlabeled geospatial images;

(d) analyzing the plurality of unlabeled geospatial images using the deep learning model to identify objects of interest; and

(e) generating an output file comprising location, classification, and quantity of the object of interest in each of the plurality of geospatial images.

Assignments (16)
RELEASE OF SECURITY INTEREST Recorded Mar 3, 2026
From: SIXTH STREET LENDING PARTNERS, ACTING IN ITS CAPACITY AS AGENT
To: AURORA INSIGHT INC.; VANTOR INC. (F/K/A MAXAR INTELLIGENCE INC.); VANTOR SERVICES INC. (F/K/A MAXAR MISSION SOLUTIONS INC.); LANTERIS SPACE LLC (F/K/A MAXAR SPACE LLC); SPATIAL ENERGY, LLC; LANTERIS SPACE ROBOTICS LLC (F/K/A MAXAR SPACE ROBOTICS LLC); VANTOR HOLDINGS INC. (F/K/A MAXAR TECHNOLOGIES HOLDINGS INC.)
Reel/Frame 075021/0624 →
CERTIFICATE OF AMENDMENT Recorded Jan 7, 2026
From: MAXAR INTELLIGENCE INC.
To: VANTOR INC.
Reel/Frame 074270/0330 →
CHANGE OF NAME Recorded Nov 4, 2025
From: MAXAR INTELLIGENCE INC.
To: VANTOR INC.
Reel/Frame 073461/0767 →
RELEASE (REEL 060389/FRAME 0720) Recorded May 12, 2023
From: ROYAL BANK OF CANADA
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063633/0431 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 5, 2023
From: MAXAR INTELLIGENCE INC. (F/K/A DIGITALGLOBE, INC.); AURORA INSIGHT INC.; MAXAR MISSION SOLUTIONS INC. ((F/K/A RADIANT MISSION SOLUTIONS INC. (F/K/A THE RADIANT GROUP, INC.)); MAXAR SPACE LLC (F/K/A SPACE SYSTEMS/LORAL, LLC); SPATIAL ENERGY, LLC; MAXAR SPACE ROBOTICS LLC ((F/K/A SSL ROBOTICS LLC) (F/K/A MDA US SYSTEMS LLC)); MAXAR TECHNOLOGIES HOLDINGS INC.
To: SIXTH STREET LENDING PARTNERS, AS ADMINISTRATIVE AGENT
Reel/Frame 063660/0138 →
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT - RELEASE OF REEL/FRAME 060389/0782 Recorded May 4, 2023
From: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063544/0074 →
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT - RELEASE OF REEL/FRAME 053866/0412 Recorded May 4, 2023
From: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063544/0011 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS AND TRADEMARKS - RELEASE OF REEL/FRAME 051258/0465 Recorded May 4, 2023
From: ROYAL BANK OF CANADA, AS AGENT
To: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
Reel/Frame 063542/0300 →
CHANGE OF NAME Recorded Feb 15, 2023
From: DIGITALGLOBE, INC.
To: MAXAR INTELLIGENCE INC.
Reel/Frame 062760/0832 →
RELEASE OF SECURITY INTEREST Recorded Jun 21, 2022
From: WILMINGTON TRUST, NATIONAL ASSOCIATION
To: DIGITALGLOBE, INC.; SPACE SYSTEMS/LORAL, LLC; RADIANT GEOSPATIAL SOLUTIONS LLC
Reel/Frame 060390/0282 →
SECURITY AGREEMENT Recorded Jun 17, 2022
From: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
To: WILMINGTON TRUST, NATIONAL ASSOCIATION
Reel/Frame 060389/0782 →
SECURITY AGREEMENT Recorded Jun 16, 2022
From: MAXAR INTELLIGENCE INC.; MAXAR SPACE LLC
To: ROYAL BANK OF CANADA
Reel/Frame 060389/0720 →
PATENT SECURITY AGREEMENT Recorded Sep 23, 2020
From: DIGITALGLOBE, INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS NOTES COLLATERAL AGENT
Reel/Frame 053866/0412 →
SECURITY AGREEMENT (NOTES) Recorded Dec 12, 2019
From: DIGITALGLOBE, INC.; RADIANT GEOSPATIAL SOLUTIONS LLC; SPACE SYSTEMS/LORAL, LLC (F/K/A SPACE SYSTEMS/LORAL INC.)
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, - AS NOTES COLLATERAL AGENT
Reel/Frame 051262/0824 →
AMENDED AND RESTATED U.S. PATENT AND TRADEMARK SECURITY AGREEMENT Recorded Dec 11, 2019
From: DIGITALGLOBE, INC.
To: ROYAL BANK OF CANADA, AS COLLATERAL AGENT
Reel/Frame 051258/0465 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2017
From: ESTRADA, ADAM; BURD, CHRISTOPHER; JENKINS, ANDREW; NEWBROUGH, JOSEPH; SZOKO, SCOTT; VINTON, MELANIE
To: DIGITALGLOBE, INC.
Reel/Frame 043864/0964 →
Continuity (4)
Continuation 15194541 · Jun 27, 2016
Continuation In Part 14835736 · Aug 26, 2015
Provisional Application 62301554 · Feb 29, 2016
Related Publication 20180158210A1 · Jun 7, 2018
Cited By (1)
US 12,205,031