Granted Patent B2

US 11,842,538 · App. 17/160,231 · Granted Dec 12, 2023

Inventors: Rahul Saxena (Santa Clara, CA); Nitin Das (Santa Clara, CA); Abhishek Vinod Singh (Santa Clara, CA)

Assignee: AIDash, Inc.

G06V20/176G06F18/2431G06N3/04G06V10/454G06V10/764G06V10/82G06V20/188G08B21/182H04L67/53

Abstract

Claims (77)

1. A system comprising:

at least one processor; and

memory, the memory containing instructions to control one or more of the at least one processor to:

receive a first set of aerial images of a geographic area, the geographic area including assets of an infrastructure, the infrastructure including the assets distributed over a geographic region, the first set of aerial images being received from a first image source over a communication network;

receive location information regarding a location of the assets, the first set of aerial images being taken during a first duration of time;

determine a likely location of at least one asset within each image of the first set of aerial images based on the location information;

create one or more bounding boxes within each of the images of the first set of aerial images, the one or more bounding boxes within each of the images of the first set of aerial images encompassing the likely location of one or more assets within a particular image of the first set of aerial images;

provide one or more of the images of the first set of aerial images to a convolutional neural network to classify pixels within the one or more bounding boxes, a classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the first set of aerial images or are part of a different classification;

determine at least one zone within at least one of the one or more of the images, the zone encompassing the at least one asset depicted in the at least one of the one or more of the images;

determine a first distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes and the at least one zone;

generate a criticality score based on the first distance;

compare the criticality score to a threshold;

provide an alert of a future hazardous condition based on the comparison of the criticality score and the threshold

receive a second set of aerial images of the geographic area, the second set of aerial images having been captured during a second duration of time, the second duration of time being after the first duration of time;

create one or more bounding boxes within each of the images of the second set of aerial images, the one or more bounding boxes within each of the images of the second set of aerial images encompassing the likely location of one or more assets within the particular image of the second set of aerial images;

provide one or more of the images of the second set of aerial images to the convolutional neural network to classify pixels within the one or more bounding boxes within each of the images of the second set of aerial images, the classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the second set of aerial images or are part of the different classification;

determine a second distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes within each of the images of the second set of aerial images and the at least one zone;

determine a rate of change between the first distance and the second distance;

determine a future impact based on the rate of change to the asset

generate a second criticality score based on the future impact and

provide an alert of the future hazardous condition based on the second criticality score.

2. The system of claim 1 , wherein the aerial images taken during a first duration of time are satellite images taken in a particular month of a particular year.

3. The system of claim 1 , the location information being provided by third-party over the communication network, the third-party that manages the assets, the third-party being remote and separate from the first image source, the location information including coordinates of the assets.

4. The system of claim 3 , wherein the memory contains instructions to control the one or more of the at least one processor to further correlate the location information with metadata associated with the one or more images of the aerial images to determine the likely location of the at least on asset within the each image of the first set of aerial images, the metadata including coordinates of at least some of the geographic area.

5. The system of claim 1 , wherein determining the at least one zone includes determining a first zone of the at least one zone based on one or more possible hazardous conditions that may be caused by the one or more obstructions on the at least one asset, determining the first zone including determining a width of the at least one zone based on the one or more possible hazardous conditions.

6. The system of claim 5 , wherein the one or more possible hazardous conditions are weighted based on terrain type of the geographic area, weather patterns, and accessibility of the geographic area.

7. The system of claim 6 , wherein the one or more possible hazardous conditions are weighted based on risk of wildfire.

8. The system of claim 1 , wherein

the second duration of time being one year or more from the first duration of time.

9. The system of claim 1 wherein the convolutional neural network is a u-net convolutional neural network.

10. A method comprising:

receiving a first set of aerial images of a geographic area, the geographic area including assets of an infrastructure, the infrastructure including the assets distributed over a geographic region, the images being received from a first image source over a communication network;

receiving location information regarding a location of the assets, the first set of aerial images being taken during a first duration of time;

determining a likely location of at least one asset within each image of the first set of aerial images based on the location information;

creating one or more bounding boxes within each of the images of the first set of aerial images, the one or more bounding boxes encompassing the likely location of one or more assets within a particular image of the first set of aerial images;

providing one or more of the images of the first set of aerial images to a convolutional neural network to classify pixels within the one or more bounding boxes, a classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the first set of aerial images or are part of a different classification;

determining at least one zone within at least one of the one or more of the images, the zone encompassing the at least one asset depicted in the at least one of the one or more of the images;

determining a distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes and the at least one zone;

generating a criticality score based on the distance;

comparing the criticality score to a threshold;

providing an alert of a future hazardous condition based on the comparison of the criticality score and the threshold;

receiving a second set of aerial images of the geographic area, the second set of aerial images having been captured during a second duration of time, the second duration of time being after the first duration of time;

creating one or more bounding boxes within each of the images of the second set of aerial images, the one or more bounding boxes within each of the images of the second set of aerial images encompassing the likely location of one or more assets within the particular image of the second set of aerial images;

providing one or more of the images of the second set of aerial images to the convolutional neural network to classify pixels within the one or more bounding boxes within each of the images of the second set of aerial images, the classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the second set of aerial images or are part of the different classification;

determining a second distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes within each of the images of the second set of aerial images and the at least one zone;

determining a rate of change between the first distance and the second distance;

determining a future impact based on the rate of change to the asset

generating a second criticality score based on the future impact and

providing an alert of the future hazardous condition based on the second criticality score.

11. The method of claim 10 , wherein the aerial images taken during a first duration of time are satellite images taken in a particular month of a particular year.

12. The method of claim 10 , the location information being provided by third-party over the communication network, the third-party that manages the assets, the third-party being remote and separate from the first image source, the location information including coordinates of the assets.

13. The method of claim 12 , the method further comprising correlating the location information with metadata associated with the one or more images of the aerial images to determine the likely location of the at least on asset within the each image of the first set of aerial images, the metadata including coordinates of at least some of the geographic area.

14. The method of claim 10 , wherein determining the at least one zone includes determining a first zone of the at least one zone based on one or more possible hazardous conditions that may be caused by the one or more obstructions on the at least one asset, determining the first zone including determining a width of the at least one zone based on the one or more possible hazardous conditions.

15. The method of claim 14 , wherein the one or more possible hazardous conditions are weighted based on terrain type of the geographic area, weather patterns, and accessibility of the geographic area.

16. The method of claim 14 , wherein the one or more possible hazardous conditions are weighted based on risk of wildfire.

17. The method of claim 10 , wherein

the second duration of time is one year or more from the first duration of time.

18. The method of claim 10 , wherein the convolutional neural network is a u-net convolutional neural network.

19. A non-transitive computer readable medium comprising instructions to a control at least one processor to perform a method, the method comprising:

receiving a first set of aerial images of a geographic area, the geographic area including assets of an infrastructure, the infrastructure including the assets distributed over a geographic region, the images being received from a first image source over a communication network;

receiving location information regarding a location of the assets, the first set of aerial images being taken during a first duration of time;

determining a likely location of at least one asset within each image of the first set of aerial images based on the location information;

creating one or more bounding boxes within each of the images of the first set of aerial images, the one or more bounding boxes encompassing the likely location of one or more assets within a particular image of the first set of aerial images;

providing one or more of the images of the first set of aerial images to a convolutional neural network to classify pixels within the one or more bounding boxes, a classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the first set of aerial images or are part of a different classification;

determining at least one zone within at least one of the one or more of the images, the zone encompassing the at least one asset depicted in the at least one of the one or more of the images;

determining a distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes and the at least one zone;

generating a criticality score based on the distance;

comparing the criticality score to a threshold;

providing an alert of a future hazardous condition based on the comparison of the criticality score and the threshold;

receiving a second set of aerial images of the geographic area, the second set of aerial images having been captured during a second duration of time, the second duration of time being after the first duration of time;

creating one or more bounding boxes within each of the images of the second set of aerial images, the one or more bounding boxes within each of the images of the second set of aerial images encompassing the likely location of one or more assets within the particular image of the second set of aerial images;

providing one or more of the images of the second set of aerial images to the convolutional neural network to classify pixels within the one or more bounding boxes within each of the images of the second set of aerial images, the classification of each of the pixels indicating if the pixels are part of one or more obstructions depicted in the one or more of the images of the second set of aerial images or are part of the different classification;

determining a second distance between at least one pixel that is part of the one or more obstructions within at least one of the bounding boxes within each of the images of the second set of aerial images and the at least one zone;

determining a rate of change between the first distance and the second distance;

determining a future impact based on the rate of change to the asset

generating a second criticality score based on the future impact and

providing an alert of the future hazardous condition based on the second criticality score.

Assignments (1)

Continuity (2)

Cited By (12)

US 12,297,965 US 12,333,801 US 12,359,403 US 12,380,694 US 12,384,508 US 12,485,389 US 12,546,245 US 12,548,131 US 12,597,151 US 12,709,808 US 12,716,379 US 12,718,555