IP Library › Granted Patent US 10,891,483
Granted Patent B2
US 10,891,483 · App. 16/377,956 · Granted Jan 12, 2021

Texture classification of digital images in aerial inspection

Inventor: Yan Yang (Brisbane, AU)
Assignee: The Boeing Company
G06K9/00657B64C39/024G06K9/3233G06K9/6256G06T11/001B64C2201/123B64C2201/127G06T2207/10032G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,891,483
App. No.
16/377,956
Granted
Jan 12, 2021
Kind
B2
Abstract

Unmanned aerial inspection systems and associated methods. In one embodiment, an aerial platform (e.g., an Unmanned Aerial Vehicle (UAV)) navigates to a location of a geographic region, and captures a digital image of the geographic region with an imaging device. The aerial platform segments the digital image into superpixels, selects a region of interest from the digital image to define one or more patches associated with the superpixels, assigns terrain texture categories to the patches, and assigns the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image. The aerial platform determines whether a site contamination is present at the geographic region based on the texture classified representation of the digital image, and reports an alert upon identifying that the site contamination is present.

Claims (103)

1. An unmanned aerial inspection system, comprising:

an aerial platform comprising an imaging device, and at least one processor and at least one memory that:

navigate the aerial platform to a location of a geographic region;

capture a digital image of the geographic region with the imaging device while the aerial platform is airborne;

segment the digital image into superpixels;

select a region of interest from the digital image to define one or more patches associated with the superpixels;

assign terrain texture categories to the patches;

assign the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image;

determine whether a site contamination is present at the geographic region based on the texture classified representation of the digital image; and

report an alert upon determining that the site contamination is present.

2. The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:

designate one or more of the terrain texture categories as a site contamination category;

identify a percentage of the superpixels in the texture classified representation that are assigned the site contamination category; and

determine that the site contamination is present at the geographic region when the percentage exceeds a threshold.

3. The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:

designate one or more of the terrain texture categories as a site contamination category;

identify a total number of the superpixels in the texture classified representation that are assigned the site contamination category; and

determine that the site contamination is present at the geographic region when the total number exceeds a threshold.

4. The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory assign the terrain texture categories to the superpixels by:

for each individual superpixel of the superpixels,

identifying pixels in the individual superpixel that belong to at least one of the patches;

identifying one or more of the terrain texture categories assigned to each of the pixels; and

assigning one of the terrain texture categories that is assigned to a majority of the pixels as a terrain texture category for the individual superpixel.

5. The unmanned aerial inspection system of claim 1 wherein:

the at least one processor and at least one memory assign the terrain texture categories to the patches based on a patch classification model; and

the at least one processor and at least one memory:

present one or more test images to a user;

receive input from the user indicating areas in the test images as test patches;

receive input from the user assigning one of the terrain texture categories to the test patches; and

train the patch classification model based on the test patches.

6. The unmanned aerial inspection system of claim 5 wherein:

the site contamination comprises vegetation encroachment, and the terrain texture categories include at least a high vegetation category and a vegetation-free category;

the at least one processor and at least one memory receive input from the user indicating first areas of concentrated vegetation within the test images as first test patches, and receive input from the user assigning the high vegetation category to the first test patches; and

the at least one processor and at least one memory receive input from the user indicating second areas of non-vegetation within the test images as second test patches, and receive input from the user assigning the vegetation-free category to the second test patches.

7. The unmanned aerial inspection system of claim 5 wherein:

the site contamination comprises erosion, and the terrain texture categories include at least a high erosion category and an erosion-free category;

the at least one processor and at least one memory receive input from the user indicating first areas of concentrated erosion within the test images as first test patches, and receive input from the user assigning the high erosion category to the first test patches; and

the at least one processor and at least one memory receive input from the user indicating second areas of non-erosion within the test images as second test patches, and to receive input from the user assigning the erosion-free category to the second test patches.

8. The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:

send an alert message via wireless signals while the aerial platform is airborne when the site contamination is present.

9. The unmanned aerial inspection system of claim 1 wherein the at least one processor and at least one memory:

send an alert message via wireless signals while the aerial platform is airborne that a high vegetation contamination is present at the geographic region such that a vegetation removal service can be directed to the location of the geographic region.

10. The unmanned aerial inspection system of claim 1 wherein the aerial platform is one of a rotary-wing Unmanned Aerial Vehicle (UAV) and a fixed-wing UAV.

11. A method of performing a site inspection, the method comprising:

navigating an aerial platform to a location of a geographic region;

capturing a digital image of the geographic region with an imaging device onboard the aerial platform while the aerial platform is airborne;

segmenting the digital image into superpixels at the aerial platform;

selecting a region of interest from the digital image to define one or more patches associated with the superpixels;

assigning terrain texture categories to the patches;

assigning the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image;

determining whether a site contamination is present at the geographic region based on the texture classified representation of the digital image; and

reporting an alert upon determining that the site contamination is present.

12. The method of claim 11 wherein determining whether a site contamination is present comprises:

designating one or more of the terrain texture categories as a site contamination category;

identifying a percentage of the superpixels in the texture classified representation that are assigned the site contamination category; and

determining that the site contamination is present at the geographic region when the percentage exceeds a threshold.

13. The method of claim 11 wherein determining whether a site contamination is present comprises:

designating one or more of the terrain texture categories as a site contamination category;

identifying a total number of the superpixels in the texture classified representation that are assigned the site contamination category; and

determining that the site contamination is present at the geographic region when the total number exceeds a threshold.

14. The method of claim 11 wherein assigning the terrain texture categories to the superpixels comprises:

for each individual superpixel of the superpixels,

identifying pixels in the individual superpixel that belong to at least one of the patches;

identifying one or more of the terrain texture categories assigned to each of the pixels; and

assigning one of the terrain texture categories that is assigned to a majority of the pixels as a terrain texture category for the individual superpixel.

15. The method of claim 11 wherein:

assigning the terrain texture categories to the patches comprises assigning the terrain texture categories to the patches based on a patch classification model; and

the method further comprises:

presenting one or more test images to a user;

receiving input from the user indicating areas in the test images as test patches;

receiving input from the user assigning one of the terrain texture categories to the test patches; and

training the patch classification model based on the test patches.

16. The method of claim 15 wherein:

the site contamination comprises vegetation encroachment, and the terrain texture categories include at least a high vegetation category and a vegetation-free category; and

receiving input from the user comprises:

receiving input from the user indicating first areas of concentrated vegetation within the test images as first test patches;

receiving input from the user assigning the high vegetation category to the first test patches;

receiving input from the user indicating second areas of non-vegetation within the test images as second test patches; and

receiving input from the user assigning the vegetation-free category to the second test patches.

17. The method of claim 15 wherein:

the site contamination comprises erosion, and the terrain texture categories include at least a high erosion category and an erosion-free category;

receiving input from the user comprises:

receiving input from the user indicating first areas of concentrated erosion within the test images as first test patches;

receiving input from the user assigning the high erosion category to the first test patches;

receiving input from the user indicating second areas of non-erosion within the test images as second test patches; and

receiving input from the user assigning the erosion-free category to the second test patches.

18. The method of claim 11 wherein reporting an alert upon determining that the site contamination is present comprises:

sending an alert message via wireless signals while the aerial platform is airborne when the site contamination is present.

19. The method of claim 11 wherein reporting an alert upon determining that the site contamination is present comprises:

sending an alert message via wireless signals while the aerial platform is airborne that a high vegetation contamination is present at the geographic region such that a vegetation removal service can be directed to the location of the geographic region.

20. A unmanned aerial inspection system, comprising:

an Unmanned Aerial Vehicle (UAV);

an imaging device on the UAV; and

a platform controller on the UAV comprising at least one processor and at least one memory that:

navigate the UAV to a location of a geographic region;

capture a digital image of the geographic region with the imaging device while the UAV is airborne; and

process the digital image while the UAV is airborne to:

segment the digital image into superpixels;

select a region of interest from the digital image to define one or more patches associated with the superpixels;

assign terrain texture categories to the patches;

assign the terrain texture categories to the superpixels based on the terrain texture categories of the patches to generate a texture classified representation of the digital image;

determine whether a vegetation encroachment is present at the geographic region based on the texture classified representation of the digital image; and

send an alert message via wireless signals while the UAV is airborne to report when the vegetation encroachment is present.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2019
From: YANG, YAN
To: THE BOEING COMPANY
Reel/Frame 048833/0044 →
Continuity (1)
Related Publication 20200320293A1 · Oct 8, 2020