IP Library › Granted Patent US 12,626,368
Granted Patent B2
US 12,626,368 · App. 18/253,120 · Granted May 12, 2026

Image analysis for aerial images

Inventors: Aleksander Buczkowski (Warsaw, PL); Michal Mazur (Warsaw, PL); Adam Wisniewski (Warsaw, PL); Dariusz Ciesla (Warsaw, PL)
Assignee: AI CLEARING INC.
G06T7/149G06T7/11G06T7/174G06T7/75G06T17/05G06V10/776G06V10/82G06V20/17G06T2207/10032G06T2207/20021G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,626,368
App. No.
18/253,120
Granted
May 12, 2026
Kind
B2
Abstract

Disclosed is a system comprising a data-processing system. The data-processing system comprises a data-storage component, a segmentation component and a projection component. The data-storage component is configured for providing an input orthophoto map of an area and an input digital elevation model of the area. The segmentation component is configured for performing a segmentation step, the segmentation step comprises generating at least one or a plurality of polygon(s) based on the input orthophoto map. Each polygon approximates a part of the input orthophoto map. The projection component is configured for performing a projection step and the projection step comprises projecting the polygon(s) on the input digital elevation model of the area. The projection component is further configured for performing a reference surface generation step, the reference surface generation step comprising generating a reference surface for each of at least some of the polygon(s). Further, a corresponding method and a corresponding computer-program product are disclosed.

Claims (38)

1 . A system comprising a data-processing system,

wherein the data-processing system comprises a data-storage component, wherein the data-storage component is configured for providing an input orthophoto map and an input digital elevation model of the area,

wherein the data-processing system further comprises a segmentation component, wherein the segmentation component is configured for generating polygon(s) based on the input orthophoto map, each polygon approximating a part of the input orthophoto map,

wherein the data-processing system comprises a projection component, wherein the projection component is configured for projecting the polygon(s) on the input digital elevation model of the area and for generating a reference surface for each of the at least some of the polygon(s),

wherein the data-processing system comprises a pre-processing component,

wherein the pre-processing component is configured for determining at least a component of a gradient of the input digital elevation model, and the segmentation component is configured for determining the parts of the input orthophoto map based at least on the input orthophoto map and the component(s) of the gradient of the input digital elevation model, and

wherein the pre-processing component is configured for generating tiles of the input orthophoto map and the digital elevation model, and

wherein the segmentation component is configured for processing at least some of the tiles individually.

2 . The system according to claim 1 , wherein the data-processing system comprises a volume determining component configured for determining a volume between a portion of the input digital elevation model and a portion of the reference surface for each reference surface.

3 . The system according to claim 2 , wherein the segmentation component and the projection component are configured for processing

the first orthophoto map as input orthophoto map and the first digital elevation model as input digital elevation model, the segmentation component being configured for thus generating first polygon(s) and the projection component being configured for thus generating first reference surface(s); and

the second orthophoto map as input orthophoto map and the second digital elevation model as input digital elevation model, the segmentation component being configured for thus generating second polygon(s) and the projection component being configured for thus generating second reference surface(s),

wherein the volume determining component is configured for processing the first reference surface(s) and the first digital elevation model and thus generating first volume(s), and for processing the second reference surface(s) and the second digital elevation model and thus generating second volume(s), and

wherein the volume determining component is configured for comparing at least some of the first and second volume(s).

4 . The system according to claim 3 , the volume determining component is configured for at least one of

determining volume differences between at least some of the first and the second volume(s), and

determining volumes that are present in only one of the first and the second volume(s).

5 . The system according to claim 1 , wherein the projection component is configured for processing elevation coordinates of the vertexes of the at least some polygon(s) projected to the input digital elevation model, wherein processing the elevation coordinates of the vertexes comprises generating a statistic measure of the elevation coordinates.

6 . The system according to claim 1 , wherein the segmentation component is configured for determining the parts of the orthophoto map by means of at least one convolutional neural network.

7 . The system according to claim 6 , wherein the segmentation component is configured for assigning different classes to different portions of the orthophoto map and for assigning portions comprising same classes to groups,

wherein the data-processing system comprises a post-processing component, and wherein the post-processing component is configured for applying a conditional random fields algorithm to borders of the groups.

8 . The system according to claim 1 , wherein the data-storage component is further configured for providing design data, wherein the data-processing system further comprises an area-comparison component, wherein the area-comparison component is configured for at least one of

comparing the polygon(s) and the design data, and

generating reporting units based on the design data, wherein generating the reporting units comprises dividing at least one object represented by the design data into a plurality of reporting units spatially different from each other.

9 . A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors of the data-processing system of claim 1 , cause the one or more processors to perform the steps for which the data-processing system is configured.

10 . A method, comprising

providing an input orthophoto map of an area,

providing an input digital elevation model of the area,

performing a segmentation step, wherein the segmentation step comprises generating at least one or a plurality of polygon(s) based on the input orthophoto map, each polygon approximating a part of the input orthophoto map,

performing a projection step, the projection step comprising projecting the polygon(s) on the input digital elevation model of the area, and

a reference surface generation step, the reference surface generation step comprising generating a reference surface for each of at least some of the polygon(s),

wherein the segmentation step comprises generating the polygon(s) based on the input orthophoto map and the input digital elevation model,

wherein the semantic segmentation step comprises a pre-processing step, the pre-processing step comprising determining at least a component of a gradient of the input digital elevation model, and

wherein the segmentation step comprises determining the parts of the input orthophoto map by means of at least one convolutional neural network based at least on the input orthophoto map and the component(s) of the gradient of the input digital elevation model.

11 . The method according to claim 10 , wherein the method further comprises a volume determining step, the volume determining step comprising for each reference surface determining a volume between a portion of the input digital elevation model and a portion of the reference surface, wherein the segmentation step comprises determining the parts of the orthophoto map by means of at least one convolutional neural network.

12 . The method according to claim 10 , wherein the pre-processing step comprises generating tiles of the input orthophoto map and the digital elevation model, wherein the segmentation step comprises assigning different classes to different portions of the orthophoto map by the at least one convolutional neural network, and wherein the method comprises processing at least some tiles individually by means of the at least one convolutional neural network, wherein the segmentation step comprises a post-processing step, wherein the post-processing step comprises applying a conditional random fields algorithm to borders of the groups.

13 . The method according to claim 10 , wherein the method comprises a data comparison step, wherein the data comparison step comprises at least one of

comparing the polygon(s) and the design data, and generating reporting units based on the design data, wherein generating the reporting units comprises dividing at least one object represented by the design data into a plurality of reporting units spatially different from each other.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2023
From: BUCZKOWSKI, ALEKSANDER; MAZUR, MICHAL; WISNIEWSKI, ADAM; CIESLA, DARIUSZ
To: AI CLEARING INC.
Reel/Frame 064690/0670 →
Priority Claims (2)
EP 20207918 · Nov 16, 2020 · regional
EP 20207919 · Nov 16, 2020 · regional
Continuity (1)
Related Publication 20230419501A1 · Dec 28, 2023
References Cited (57)
US 9245201B1 · Jin et al. · 2016 [cited by applicant]
US 9389084B1 · Chen · 2016 [cited by applicant]
US 10339663B2 · Barajas Hernandez · 2019 [cited by applicant]
US 10593108B2 · Barajas Hernandez · 2020 [cited by applicant]
US 20020082942A1 · Shimazu · 2002 [cited by examiner]
US 20050031197A1 · Knopp · 2005 [cited by applicant]
US 20110033110A1 · Shimamura · 2011 [cited by examiner]
US 20210150230A1 · Smolyanskiy · 2021 [cited by examiner]
US 20210304514A1 · Croxford et al. · 2021 [cited by applicant]
Billen, et al. “3D spatial relationships model: a useful concept for 3D cadastre?” (2003) Computers, Environment and Urban Systems, vol. 27(4), pp. 411-425 doi:10.1016/s0198-9715(02)00040-6. [cited by applicant]
Vu, et al. “A Hybrid Neural Network for Graph-Based Human Pose Estimation From 2D Images” (2020) IEEE Access, vol. 8, pp. 52830-52840 doi:10.1109/access.2020.2980565. [cited by applicant]
Xiong, et al. “An End-To-End Bayesian Segmentation Network Based on a Generative Adversarial Network for Remote Sensing Images” (2020) Remote Sensing, vol. 12, 216 doi:10.3390/rs12020216. [cited by applicant]
Peng, et al. “An Improved GrabCut Method Based on a Visual Attention Model for Rare-Earth Ore Mining Area Recognition with High-Resolution Remote Sensing Images”, (2019) Remote Sensing, vol. 11,987 doi:10.3390/rs1108098… [cited by applicant]
Ahmadi, et al. “An Integrated Photogrammetric and Spatial Database Management System for Producing Fully Structured Data Using Aerial and Remote Sensing Images” (2009) Sensors, vol. 9(4), pp. 2320-2333 doi:10.3390/s9040… [cited by applicant]
Huang, et al. “Automatic Mapping of Thermokarst Landforms from Remote Sensing Images Using Deep Learning: A Case Study in the Northeastern Tibetan Plateau” (2018) Remote Sensing, vol. 10, 2067 doi:10.3390/rs10122067. [cited by applicant]
Bokhovkin, et al. “Boundary Loss for Remote Sensing Imagery Semantic Segmentation” (2019) Advances in Neural Networks. [cited by applicant]
Maggiori, et al. “Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark” (2017) IEEE International Geoscience and Remote Sensing Symposium (IGARSS) doi:10.1109/igarss.2017.81276… [cited by applicant]
“QGIS A Free Open Source Geographic Information System” https://qgis.org/en/site/. [cited by applicant]
“Occupation Employment and Wage Statistics” US Bureaus of Labor Statistics, https://www.bls.gov/oes/. [cited by applicant]
International Search Report dated Mar. 10, 2022 for corresponding International Application No. PCT/US2021/059493. [cited by applicant]
Krausse et al., “Extracting Orthogonal Building Objects in Urban Areas from High Resolution Stereo Satellite Image Pairs”, (Sep. 1, 2007) pp. 1-6 http://elib.dlr.de/52632/1/krauss-PIA.pdf. [cited by applicant]
Reinzartz, et al., “Advances in DSM Generation and Higher Level Information Extraction from High Resolution Optical Stereo Satellite Data” Earsel Special Interest Group: 3D Remote Sensing & Urban Remote Sensing, (Jan. 1… [cited by applicant]
Giordan, et al., “The use of unmanned aerial vehicles (UAVs) for engineering geology applications”, Bulletin of Engineering Geology and The Environment, vol. 79, No. 7 (Apr. 1, 2020), pp. 3437-3481 doi:10.1007/s10064-02… [cited by applicant]
Cheng, et al. “Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, (Jun. … [cited by applicant]
Pang, et al. (2016), “SGM-based seamline determination for urban orthophoto mosaicking” ISPRS Journal of Photogrammetry and Remote Sensing, 112, 1-12, (2016) doi:10.1016/j.isprsjprs.2015.11.0. [cited by applicant]
Xiang, et al. (2019). “Mini-Unmanned Aerial Vehicle-Based Remote Sensing: Techniques, applications, and prospects” IEEE Geoscience and Remote Sensing Magazine, vol. 7(3), pp. 29-63 doi:10.1109/mgrs.2019.2918840. [cited by applicant]
Hu, et al. “Classification of Very High-Resolution Remote Sensing Imagery Using a Fully Convolutional Network With Global and Local Context Information Enhancements” IEEE Access, vol. 8, pp. 14606-14619 (2020) doi.10.11… [cited by applicant]
“CS231n Colvolutional Neural Networks for Visual Recognition” Convolutional Neural Networks (CNNs/ConvNets) (2023), 23 pages https://cs231n.github.io/convolutional-networks/. [cited by applicant]
“CS231n Colvolutional Neural Networks for Visual Recognition:Introduction” Course website (2023), 12 pages https://cs231n.github.io/optimization-2/. [cited by applicant]
“CS231n Colvolutional Neural Networks for Visual Recognition:Learning” Course website (2023), 19 pages https://cs231n.github.io/neural-networks-3/. [cited by applicant]
Dos Santos, et al., “Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts” (2014) Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 69-… [cited by applicant]
Buda, et al. “Deep Learning-Based Segementation of Nodules in Thyroid Ultrasound: Improving Performance by Utilizing Markers Present in the Images”, Science Direct, Ultrasound in Medicine & Biology, (2020). vol. 46, iss… [cited by applicant]
Liu, et al. “Dense Dilated Convolutions Merging Network for Land Cover Classification” IEEE Transactions on Geoscience and Remote Sensing, (2020), 11 pages. [cited by applicant]
Bhatnagar, et al. “Drone Image Segmentation Using Machine and Deep Learning for Mapping Raised Bog Vegetation Communities”, Remote Sensing (2020) vol. 12, 2602, 26 pages doi:10.3390/rs12162602. [cited by applicant]
Liu, et al., Efficient Patch-Wise Semantic Segmentation for Large Scale Remote Sensing Images, Sensors (2018) vol. 18(10), 3232, 36 pages. [cited by applicant]
Yang, et al., “End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation” (2016) IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 10 pages doi:… [cited by applicant]
ESRI Shapefile Technical Description, ESRI White Paper (Jul. 1998), 35 pages. [cited by applicant]
Zhou, et al., “Estimating Maize-Leaf Coverage in Field Conditions by Applying a Machine Learning Algorithm to UAV Remote Sensing Images”, Applied Sciences (2019) vol. 9.2389, 18 pages doi:10.3390/app9112389. [cited by applicant]
Singh, et al., “EvalNorm: Estimating Batch Normalization Statistics for Evaluation” (2019) IEEE/CVF International Conference on Computer Vision (ICCV) 10.1109/iccv.2019.00373. [cited by applicant]
Poirson, et al., “Fast Single Shot Detection and Pose Estimation” (2016) Fourth International Conference on 3D Vision (3DV) doi:10.1109/3dv.2016.78. [cited by applicant]
Ren, et al., “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, (2017) IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), pp. 1137-1149 doi:10.1109/tpami.2016.2577031. [cited by applicant]
Reiterer, et al. (2020). Fully Automated Segmentation of 2D and 3D Mobile Mapping Data for Reliable Modeling of Surface Structures Using Deep Learning. Remote Sensing, vol. 12(16), 2530 doi:10.3390/rs12162530. [cited by applicant]
Shelhamer, et al., “Fully Convolutional Networks for Semantic Segmentation” (2017) IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39(4), pp. 640-651 doi:10.1109/tpami.2016.2572683. [cited by applicant]
Chen, et al., “Fully Convolutional Neural Network with Augmented Atrous Spatial Pyramid Pool and Fully Connected Fusion Path for High Resolution Remote Sensing Image Segmentation” (2019) Applied Sciences vol. 9.9, 1816 … [cited by applicant]
Dang, et al., “Fusarium Wilt of Radish Detection Using RGB and Near Infrared Images from Unmanned Aerial Vehicles”, (2020) Remote Sensing vol. 12.17, 2863 doi:10.3390/rs12172863. [cited by applicant]
Krizhevsky, et al., “ImageNet classification with deep convolutional neural networks” (2017) Communications of the ACM, vol. 60(6), pp. 84-90 doi:10.1145/3065386. [cited by applicant]
Mazzia, et al., “Improvement in Land Cover and Crop Classification based on Temporal Features Learning from Sentinel-2 Data Using Recurrent-Convolutional Neural Network (R-CNN)”. (2019) Applied Sciences, vol. 10(1), 238… [cited by applicant]
“OpenGIS Web Map Tile Service Implementation Standard”, OGC Making Location Count, (2021) https://www.ogc.org/standards/wmts. [cited by applicant]
Everingham, et al. “The 2005 PASCAL Visual Object Classes Challenge”, First PASCAL Machine Learning Challenges Workshop (MLCW '05), (Apr. 2005), Southampton, United Kingdom. pp.117-176, doi.10.1007/11736790_8. [cited by applicant]
Paszke, et al. “PyTorch: An Imperative Style, High-Performance Deep Learning Library” (2019) 33rd Conference on Neural Information Processing Systems, 12 pages. [cited by applicant]
Chen, et al., “Rethinking Atrous Convolution for Semantic Image Segmentation” (2017) Google Inc. 14 pages. [cited by applicant]
Zoph, et al., “Rethinking Pre-training and Self-training” (2020) 34th Conference on Neural Information Processing Systems, 13 pages. [cited by applicant]
Lukic, et al., “Speaker identification and clustering using convolutional neural networks” (2016) IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), 6 pages doi:10.1109/mlsp.2016.7738816. [cited by applicant]
Rottensteiner, et al., “The ISPRS Benchmark on Urban Object Classification and 3D Building Reconstruction” (2012) ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 1-3, XXII ISPRS… [cited by applicant]
Ronneberger, et al., “U-Net: Convolutional Networks for Biomedical Image Segmentation” (2015) Medical Image Computing and Computer-Assisted Intervention—MICCAI 234-241 doi:10.1007/978-3-319-24574-4_28. [cited by applicant]
Redmon, et al., “You Only Look Once: Unified, Real-Time Object Detection” (2016) IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 10 pages doi:10.1109/cvpr.2016.91. [cited by applicant]
International Search Report dated Mar. 10, 2022 for corresponding International Application No. PCT/US2021/059512. [cited by applicant]