Determining stockpile volume based on digital aerial images and three-dimensional representations of a site
Systems and methods are disclosed for utilizing digital aerial images to determine stockpile volumes. In particular, in one or more embodiments, the disclosed systems and methods utilize a UAV to capture a plurality of digital aerial images portraying stockpiles on a site. The disclosed systems and methods can generate a two-dimensional representation of the site and a three-dimensional representation of the site based on the plurality of digital aerial images. Moreover, the disclosed systems and methods can apply an elevation filter to the three-dimensional representation of the site to identify potential stockpiles. Further, the disclosed systems and methods can utilize a neural network to analyze two-dimensional features and three-dimensional features of the potential stockpiles to identify final stockpiles. The disclosed systems and methods can then utilize three-dimensional stockpile representations to determine stockpile volumes.
1. A method comprising:
utilizing a plurality of digital aerial images of a site captured by a UAV during a flight to generate a three-dimensional representation of the site and a two-dimensional representation of the site;
applying, by at least one processor, an elevation filter to the three-dimensional representation of the site to generate a three-dimensional representation of a potential stockpile on the site and a boundary of the potential stockpile;
generating a two-dimensional representation of the potential stockpile from the two-dimensional representation of the site based on the boundary of the potential stockpile;
determining, by the at least one processor, that the potential stockpile is a stockpile by applying a neural network to the two-dimensional representation of the potential stockpile and the three-dimensional representation of the potential stockpile, wherein the neural network is trained to identify stockpiles from two-dimensional training inputs and three-dimensional training inputs; and
determining, by the at least one processor, a volume of the stockpile based on the three-dimensional representation of the stockpile.
2. The method of claim 1 , wherein applying the elevation filter comprises applying a ground return algorithm that identifies and removes non-ground objects from the three-dimensional representation to generate a filtered three-dimensional representation of the site.
3. The method of claim 2 , wherein the filtered three-dimensional representation of the site comprises a plurality of points and applying the elevation filter further comprises:
identifying a highest point of the plurality of points from the filtered three-dimensional representation of the site;
determining additional points within a radius of the highest point; and
based on a determination that the additional points satisfy a threshold stockpile gradient relative to the highest point, utilizing the additional points to generate the three-dimensional representation of the potential stockpile.
4. The method of claim 3 , wherein applying the elevation filter further comprises:
identifying a set of points in the three-dimensional representation of the site adjacent to the three-dimensional representation of the potential stockpile that do not satisfy the threshold stockpile gradient relative to the three-dimensional representation; and
generating the boundary of the potential stockpile based on the identified set of points.
5. The method of claim 1 , further comprising:
applying a material classifier to the two-dimensional representation of the potential stockpile to identify a material boundary between a first material in the potential stockpile and a second material in the potential stockpile; and
utilizing the three-dimensional representation of the potential stockpile and the material boundary between the first material and the second material to generate a three-dimensional representation of a first revised potential stockpile corresponding to the first material and a three-dimensional representation of a second revised potential stockpile corresponding to the second material.
6. The method of claim 5 , wherein determining that the potential stockpile is the stockpile comprises:
generating a two-dimensional representation of the first revised potential stockpile corresponding to the first material based on the material boundary; and
applying the neural network to the three-dimensional representation of the first revised potential stockpile and the two-dimensional representation of the first revised potential stockpile.
7. The method of claim 1 , wherein determining the volume of the stockpile based on the three-dimensional representation of the stockpile comprises:
determining a ground reference surface based on the boundary of the potential stockpile; and
calculating the volume of the stockpile by comparing the ground reference surface and the three-dimensional representation of the stockpile.
8. The method of claim 1 , further comprising:
receiving a second plurality of digital aerial images of the site captured by one or more UAVs during a second flight;
based on a known location of the stockpile from the plurality of digital aerial images, identifying the stockpile from the second plurality of digital aerial images captured during the second flight;
generating a new three-dimensional representation of the stockpile from the second plurality of digital aerial images; and
determining a change in volume of the stockpile based on the three-dimensional representation of the stockpile and the new three-dimensional representation of the stockpile.
9. A system comprising:
at least one processor; and
at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
utilize a plurality of digital aerial images of a site captured by a UAV during a flight to generate a three-dimensional representation of the site and a two-dimensional representation of the site;
apply an elevation filter to the three-dimensional representation of the site to generate a three-dimensional representation of a potential stockpile on the site and a boundary of the potential stockpile;
generate a two-dimensional representation of the potential stockpile from the two-dimensional representation of the site based on the boundary of the potential stockpile;
determine that the potential stockpile is a stockpile by applying a neural network to the two-dimensional representation of the potential stockpile and the three-dimensional representation of the potential stockpile, wherein the neural network is trained to identify stockpiles from two-dimensional training inputs and three-dimensional training inputs; and
determine a volume of the stockpile based on the three-dimensional representation of the stockpile.
10. The system of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the system to train the neural network by:
providing the neural network with a training two-dimensional representation and a training three-dimensional representation corresponding to a ground-truth stockpile classification;
utilizing the neural network to predict a stockpile classification for the two-dimensional representation and the three-dimensional representation; and
determining one or more loss functions by comparing the predicted stockpile classification and the ground-truth stockpile classification.
11. The system of claim 9 , wherein the three-dimensional representation of the site comprises a three-dimensional point cloud and the two-dimensional representation of the site comprises a two-dimensional orthophoto.
12. The system of claim 11 , wherein the three-dimensional representation of the site comprises a plurality of points and further comprising instructions that, when executed by the at least one processor, cause the system to apply the elevation filter by:
identifying a highest point of the plurality of points from the three-dimensional representation of the site;
determining additional points within a radius of the highest point; and
based on a determination that the additional points satisfy a threshold stockpile gradient relative to the highest point, utilizing the additional points to generate the three-dimensional representation of the potential stockpile.
13. The system of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the system to:
apply a material classifier to the two-dimensional representation of the potential stockpile to identify a material boundary between a first material in the potential stockpile and a second material in the potential stockpile;
generate a two-dimensional representation of a first revised potential stockpile corresponding to the first material based on the material boundary; and
apply the neural network to the two-dimensional representation of the first revised potential stockpile.
14. The system of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the system to:
receive a second plurality of digital aerial images of the site captured by one or more UAVs during a second flight;
based on a known location of the stockpile from the plurality of digital aerial images, identify the stockpile from the second plurality of digital aerial images captured during the second flight;
generate a new three-dimensional representation of the stockpile from the second plurality of digital aerial images; and
determine a change in volume of the stockpile based on the three-dimensional representation of the stockpile and the new three-dimensional representation of the stockpile.
15. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computer system to:
utilize a plurality of digital aerial images of a site captured by a UAV during a flight to generate a three-dimensional representation of the site and a two-dimensional representation of the site;
apply an elevation filter to the three-dimensional representation of the site to generate a three-dimensional representation of a potential stockpile on the site and a boundary of the potential stockpile;
generate a two-dimensional representation of the potential stockpile from the two-dimensional representation of the site based on the boundary of the potential stockpile;
determine that the potential stockpile is a stockpile by applying a neural network to the two-dimensional representation of the potential stockpile and the three-dimensional representation of the potential stockpile, wherein the neural network is trained to identify stockpiles from two-dimensional training inputs and three-dimensional training inputs; and
determine a volume of the stockpile based on the three-dimensional representation of the stockpile.
16. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to apply the elevation filter by applying a ground return algorithm that identifies and removes non-ground objects from the three-dimensional representation to generate a three-dimensional representation of the site.
17. The non-transitory computer readable medium of claim 16 , wherein the three-dimensional representation of the site comprises a plurality of points and further comprising instructions that, when executed by the at least one processor, cause the computer system to apply the elevation filter by:
identifying a highest point of the plurality of points from the three-dimensional representation of the site;
determining additional points within a radius of the highest point; and
based on a determination that the additional points satisfy a threshold stockpile gradient relative to the highest point, utilizing the additional points to generate the three-dimensional representation of the potential stockpile.
18. The non-transitory computer readable medium of claim 17 , further comprising instructions that, when executed by the at least one processor, cause the computer system to apply the elevation filter by:
identifying a set of points in the three-dimensional representation of the site adjacent to the three-dimensional representation of the potential stockpile that do not satisfy the threshold stockpile gradient relative to the three-dimensional representation; and
generating the boundary of the potential stockpile based on the identified set of points.
19. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
apply a material classifier to the two-dimensional representation of the potential stockpile to identify a material boundary between a first material in the potential stockpile and a second material in the potential stockpile;
generate a two-dimensional representation of a first revised potential stockpile corresponding to the first material based on the material boundary; and
apply the neural network to the two-dimensional representation of the first revised potential stockpile.
20. The non-transitory computer readable medium of claim 19 , wherein the material classifier comprises at least one of: a clustering algorithm that clusters portions of the two-dimensional representation based on color or a neural network trained to identify different materials in potential stockpiles based on training two-dimensional representations.