IP Library Granted Patent US 11,227,191
Granted Patent B2
US 11,227,191 · App. 16/275,186 · Granted Jan 18, 2022

Conditional loss function modification in a neural network

Inventors: David Stuart Godwin, IV (Leander, TX); Thomas Scott Ashman (Long Beach, CA); Spencer Ryan Romo (Austin, TX); Melanie Stricklan (Long Beach, CA); Carrie Inez Hernandez (Long Beach, CA)
Assignee: Slingshot Aerospace, Inc.
G06K9/6262G06K9/00657G06K9/6202G06K9/6256G06N3/04G06N3/0436G06N3/0454G06N3/08G06N3/084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,227,191
App. No.
16/275,186
Granted
Jan 18, 2022
Kind
B2
Abstract

Method, electronic device, and computer readable medium embodiments are disclosed. In one embodiment, a method includes training a neural network using a first image dataset and a first truth dataset, then using the trained neural network to analyze a second image dataset. The training includes modifying a loss function of the neural network to forego penalizing the neural network when a feature is predicted with higher than a first confidence level by the neural network, and the first truth dataset has no feature corresponding to the predicted feature.

Claims (66)

1. A method comprising:

training a neural network to analyze remote sensed image data, said training comprising:

analyzing a first remote sensed image dataset using the neural network to predict features at respective locations within the first remote sensed image dataset;

comparing the predicted features from the neural network analysis with a first truth dataset corresponding to the first remote sensed image dataset; and

modifying a loss function of the neural network to forego penalizing the neural network when a feature is predicted at a given location within the first remote sensed image dataset with higher than a first confidence level by the neural network, and the first truth dataset has no feature at the given location corresponding to the predicted feature; and then

subsequent to the training, analyzing a second remote sensed image dataset using the trained neural network to detect features within the second remote sensed image dataset.

2. The method of claim 1 , wherein:

the second remote sensed image dataset has a corresponding second truth dataset; and

the method further comprises supplementing the second truth dataset with the detected features.

3. The method of claim 1 , wherein the method further comprises:

applying the loss function of the neural network when a feature in the first truth dataset has no corresponding feature predicted by the neural network.

4. The method of claim 1 , wherein:

said training further comprises analyzing the first remote sensed image dataset using a canonical approach to identify features at respective locations within the first remote sensed image dataset; and

said modifying the loss function of the neural network to forego penalizing the neural network is performed when the feature predicted at the given location within the first remote sensed image dataset with higher than the first confidence level by the neural network, is also identified with higher than a second confidence level by the canonical approach, and has no corresponding feature at the given location in the first truth dataset.

5. The method of claim 1 , wherein said modifying the loss function of the neural network to forego penalizing the neural network comprises:

modifying an output of the neural network to remove the predicted feature; and

applying the loss function to the neural network using the modified output and the first truth dataset, to thereby forego penalizing the neural network for predicting the feature.

6. The method of claim 1 , wherein said training further comprises:

combining the loss function with feedback from another analysis of the first remote sensed image dataset; and

applying the combined loss function when a feature in the first truth dataset is not predicted by the neural network.

7. The method of claim 6 , wherein:

said another analysis of the first remote sensed image dataset comprises a normalized differential vegetative index (NDVI) analysis; and

an object identified by the NDVI analysis overlaps at least partially with the feature in the first truth dataset that is not predicted by the neural network.

8. The method of claim 1 , wherein:

the loss function comprises a categorical cross-contributing loss function.

9. The method of claim 1 , wherein:

the first and second remote sensed image datasets comprise respective sets of aerial images, and the first and second truth datasets comprise respective sets of map data.

10. The method of claim 9 , wherein:

the first and second remote sensed image datasets respectively comprise at least one of color, at least one radio frequency band, a resolution, or a histogram.

11. The method of claim 10 , wherein:

the first confidence level comprises at least 90% confidence.

12. An electronic device comprising:

an I/O interface configured to receive image data; and

a processor operably coupled to the I/O interface,

wherein the processor is configured to train a neural network to analyze remote sensed image data, which includes being configured to:

analyze a first remote sensed image dataset using the neural network to predict features at respective locations within the first remote sensed image dataset;

compare the predicted features from the neural network analysis with a first truth dataset corresponding to the first remote sensed image dataset; and

modify a loss function of the neural network to forego penalizing the neural network when a feature is predicted at a given location within the first remote sensed image dataset with higher than a first confidence level by the neural network, and the first truth dataset has no feature at the given location corresponding to the predicted feature, and

wherein the processor is further configured to analyze a second remote sensed image dataset using the trained neural network to detect features within the second remote sensed image dataset.

13. The electronic device of claim 12 , wherein:

the second remote sensed image dataset has a corresponding second truth dataset; and

the processor is further configured to supplement the second truth dataset with the detected features.

14. The electronic device of claim 12 , wherein:

to train the neural network, the processor is further configured to analyze the first remote sensed image dataset using a canonical approach to identify features at respective locations within the first remote sensed image dataset; and

the processor is further configured to modify the loss function of the neural network to forego penalizing the neural network when the feature predicted at the given location within the first remote sensed image dataset with higher than the first confidence level by the neural network, is also identified with higher than a second confidence level by the canonical approach, and has no corresponding feature at the given location in the first truth dataset.

15. The electronic device of claim 12 , wherein to modify the loss function of the neural network to forego penalizing the neural network, the processor is further configured to:

modify an output of the neural network to remove the predicted feature; and

apply the loss function to the neural network using the modified output and the first truth dataset, to thereby forego penalizing the neural network for predicting the feature.

16. The electronic device of claim 12 , wherein the processor is further configured to:

combine the loss function with feedback from another analysis of the first remote sensed image dataset; and

apply the combined loss function when a feature in the first truth dataset is not predicted by the neural network.

17. A non-transitory computer-readable storage medium embodying a computer program, the computer program comprising computer readable program code that when executed by a processor of an electronic device causes the processor to:

train a neural network to analyze remote sensed image data, which includes causing the processor to:

analyze a first remote sensed image dataset using the neural network to predict features at respective locations within the first remote sensed image dataset;

compare the predicted features from the neural network analysis with a first truth dataset corresponding to the first remote sensed image dataset; and

modify a loss function of the neural network to forego penalizing the neural network when a feature is predicted at a given location within the first remote sensed image dataset with higher than a first confidence level by the neural network, and the first truth dataset has no feature at the given location corresponding to the predicted feature; and

analyze a second remote sensed image dataset using the trained neural network to detect features within the second remote sensed image dataset.

18. The non-transitory computer-readable storage medium of claim 17 , wherein:

the second remote sensed image dataset has a corresponding second truth dataset; and

further comprising program code that, that when executed by the processor, causes the processor to supplement the second truth dataset with the detected features.

19. The non-transitory computer-readable storage medium of claim 17 , further comprising program code that, that when executed by the processor, causes the processor to:

analyze the first remote sensed image dataset using a canonical approach to identify features at respective locations within the first remote sensed image dataset; and

modify the loss function of the neural network to forego penalizing the neural network when the feature predicted at the given location within the first remote sensed image dataset with higher than the first confidence level by the neural network, is also identified with higher than a second confidence level by the canonical approach, and has no corresponding feature at the given location in the first truth dataset.

20. The non-transitory computer-readable storage medium of claim 17 , wherein to modify the loss function of the neural network to forego penalizing the neural network, further comprises program code that, that when executed by the processor, causes the processor to:

modify an output of the neural network to remove the predicted feature; and

apply the loss function to the neural network using the modified output and the first truth dataset, to thereby forego penalizing the neural network for predicting the feature.

Assignments (2)
SECURITY INTEREST Recorded Jul 14, 2024
From: SLINGSHOT AEROSPACE, INC.
To: TRINITY CAPITAL INC., AS COLLATERAL AGENT
Reel/Frame 067983/0353 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 29, 2019
From: GODWIN, DAVID STUART; ROMO, SPENCER RYAN; HERNANDEZ, CARRIE INEZ; ASHMAN, THOMAS SCOTT; STRICKLAN, MELANIE
To: SLINGSHOT AEROSPACE, INC.
Reel/Frame 050215/0335 →
Continuity (3)
Provisional Application 62630097 · Feb 13, 2018
Provisional Application 62640404 · Mar 8, 2018
Related Publication 20190251398A1 · Aug 15, 2019