IP Library Granted Patent US 11,836,223
Granted Patent B2
US 11,836,223 · App. 17/350,594 · Granted Dec 5, 2023

Systems and methods for automated detection of building footprints

Inventors: Li Chen (Fremont, CA); Purvi Goel (Fremont, CA); Ilknur Kaynar Kabul (Mountain View, CA); David Dongzhe Yang (Brighton, MA)
Assignee: Meta Platforms, Inc.
G06F18/2185G06F18/2113G06F18/2148G06N3/045G06N3/08G06V10/243G06V10/30G06V20/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,836,223
App. No.
17/350,594
Granted
Dec 5, 2023
Kind
B2
Abstract

The disclosed computer-implemented method may include collecting a set of labels that label polygons within a training set of images as architectural structures. The method may also include creating a set of noisy labels with a predetermined degree of noise by distorting boundaries of a number of the polygons within the training set of images. Additionally, the method may include simultaneously training two neural networks by applying a co-teaching method to learn from the set of noisy labels. The method may also include extracting a preferential list of training data based on the two trained neural networks. Furthermore, the method may include training a machine learning model with the preferential list of training data. Finally, the method may include identifying one or more building footprints in a target image using the trained machine learning model. Various other methods, systems, and computer-readable media are also disclosed.

Claims (83)

1. A computer-implemented method comprising:

collecting a set of labels that label polygons within a training set of images as architectural structures;

creating a set of noisy labels with a predetermined degree of noise by distorting boundaries of a number of the polygons within the training set of images;

simultaneously training two neural networks by applying a co-teaching method to learn from the set of noisy labels;

extracting a preferential list of training data based on the two trained neural networks;

training a machine learning model with the preferential list of training data; and

identifying at least one building footprint in a target image using the trained machine learning model.

2. The method of claim 1 , wherein collecting the set of labels comprises:

identifying the training set of images; and

collecting crowdsourced labels for images in the training set of images.

3. The method of claim 1 , wherein the predetermined degree of noise comprises at least one of:

a calculated degree of noise based on crowdsourced labels for a geographic region; or

a manually adjusted degree of noise.

4. The method of claim 1 , wherein distorting the boundaries of a polygon comprises at least one of:

adjusting a scale of the polygon;

rotating the polygon;

shifting a location of the polygon; or

skewing a boundary of the polygon.

5. The method of claim 1 , wherein creating the set of noisy labels comprises at least one of:

applying the set of labels to the distorted polygons;

removing a portion of the polygons from the training set of images; or

adjusting a distribution of types of polygons within the training set of images.

6. The method of claim 1 , wherein training the two neural networks comprises:

training the two neural networks on embeddings of the training set of images;

updating a first neural network using training data selected by a second neural network; and

updating the second neural network using training data selected by the first neural network.

7. The method of claim 1 , wherein extracting the preferential list of training data comprises:

evaluating an accuracy of labels for the training set of images; and

swapping a low accuracy label with a higher accuracy label.

8. The method of claim 1 , wherein extracting the preferential list of training data comprises:

ranking training data based on a loss function of the co-teaching method; and

adding training data to the preferential list of training data based on the ranking.

9. The method of claim 1 , further comprising:

evaluating a predictive accuracy of the two trained neural networks; and

modifying the two neural networks to improve the predictive accuracy.

10. The method of claim 9 , wherein evaluating the predictive accuracy of the two trained neural networks comprises calculating at least one of:

an intersection over union metric calculated for image pixels;

a precision score;

a recall score; or

an F-score.

11. A system comprising:

a collection module, stored in memory, that collects a set of labels that label polygons within a training set of images as architectural structures;

a creation module, stored in memory, that creates a set of noisy labels with a predetermined degree of noise by distorting boundaries of a number of the polygons within the training set of images;

a co-teaching module, stored in memory, that simultaneously trains two neural networks by applying a co-teaching method to learn from the set of noisy labels;

an extraction module, stored in memory, that extracts a preferential list of training data based on the two trained neural networks;

a training module, stored in memory, that trains a machine learning model with the preferential list of training data;

an identification module, stored in memory, that identifies at least one building footprint in a target image using the trained machine learning model; and

at least one processor that executes the collection module, the creation module, the co-teaching module, the extraction module, the training module, and the identification module.

12. The system of claim 11 , wherein the collection module collects the set of labels by:

identifying the training set of images; and

collecting crowdsourced labels for images in the training set of images.

13. The system of claim 11 , wherein the predetermined degree of noise comprises at least one of:

a calculated degree of noise based on crowdsourced labels for a geographic region; or

a manually adjusted degree of noise.

14. The system of claim 11 , wherein the creation module distorts the boundaries of a polygon by at least one of:

adjusting a scale of the polygon;

rotating the polygon;

shifting a location of the polygon; or

skewing a boundary of the polygon.

15. The system of claim 11 , wherein the creation module creates the set of noisy labels by at least one of:

applying the set of labels to the distorted polygons;

removing a portion of the polygons from the training set of images; or

adjusting a distribution of types of polygons within the training set of images.

16. The system of claim 11 , wherein the co-teaching module trains the two neural networks by:

training the two neural networks on embeddings of the training set of images;

updating a first neural network using training data selected by a second neural network; and

updating the second neural network using training data selected by the first neural network.

17. The system of claim 11 , wherein the extraction module extracts the preferential list of training data by:

evaluating an accuracy of labels for the training set of images; and

swapping a low accuracy label with a higher accuracy label.

18. The system of claim 11 , wherein the extraction module extracts the preferential list of training data by:

ranking training data based on a loss function of the co-teaching method; and

adding training data to the preferential list of training data based on the ranking.

19. The system of claim 11 , further comprising an evaluation module, stored in memory, that:

evaluates a predictive accuracy of the two trained neural networks; and

modifies the two neural networks to improve the predictive accuracy.

20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

collect a set of labels that label polygons within a training set of images as architectural structures;

create a set of noisy labels with a predetermined degree of noise by distorting boundaries of a number of the polygons within the training set of images;

simultaneously train two neural networks by applying a co-teaching method to learn from the set of noisy labels;

extract a preferential list of training data based on the two trained neural networks;

train a machine learning model with the preferential list of training data; and

identify at least one building footprint in a target image using the trained machine learning model.

Assignments (2)
CHANGE OF NAME Recorded Jan 11, 2022
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058685/0901 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: CHEN, LI; GOEL, PURVI; KABUL, ILKNUR KAYNAR; YANG, DAVID DONGZHE
To: FACEBOOK, INC.
Reel/Frame 058489/0963 →
Continuity (2)
Provisional Application 63113805 · Nov 13, 2020
Related Publication 20220156526A1 · May 19, 2022