Method, apparatus, and system for pole extraction from optical imagery
An approach is provided for pole extraction from optical imagery. The approach involves, for instance, processing a plurality of images using a machine learning model to generate a plurality of redundant observations of a pole-like object and/or their semantic keypoints respectively depicted in the plurality of images. The approach also involves performing a photogrammetric triangulation of the plurality of redundant observations to determine three-dimensional coordinate data of the pole-like object and/or their semantic keypoints. The approach further involves providing the three-dimensional coordinate data of the pole-like object as an output.
1 . A computer-implemented method comprising:
processing a plurality of images using a machine learning model to generate a plurality of redundant observations of a pole-like object respectively depicted in the plurality of images;
performing a photogrammetric triangulation of the plurality of redundant observations to determine three-dimensional coordinate data of the pole-like object, wherein the machine learning model generates the plurality of redundant observations based on detecting one or more semantic keypoints comprising a geometric representation of the pole-like object, and wherein the three-dimensional coordinate data include respective three-dimensional coordinates of the one or more semantic keypoints;
determining a location, a geometric attribute, or a combination thereof of the pole-like object based on the three-dimensional coordinate data; wherein the geometric attribute includes a length, an orientation, or a combination thereof; and
providing the three-dimensional coordinate data and the geometric attribute of the pole-like object as an output.
2 . The method of claim 1 , wherein the one or more semantic keypoints include a bottom point, a top point, a midpoint, or a combination thereof of a straight section of the pole-like object.
3 . The method of claim 1 , wherein the machine learning model is a Mask Region-Based Convolutional Neural Network (Mask R-CNN) that is trained using a plurality of reference images respectively labeled with a plurality of ground truth semantic keypoints of a plurality of pole-like objects.
4 . The method of claim 1 , wherein the plurality of pole-like objects in the plurality of reference images is respectively labeled with the plurality of ground truth semantic keypoints in a designated order.
5 . The method of claim 1 , further comprising:
back projecting the three-dimensional coordinate data into the plurality of images to perform a validation of the three-dimensional coordinated data,
wherein the validation is based on determining that the back-projected three-dimensional coordinate data falls within a threshold distance of the plurality of redundant observations.
6 . The method of claim 5 , wherein the validation is further based on at least one of (1) determining that the back-projected three-dimensional coordinate data corresponds to respective two-dimensional features in a minimum number of the plurality of images; (2) determining that a minimum intersection angle of confirmed rays of the photogrammetric triangulation associated with the three-dimensional coordinate data meets an angular threshold; or (3) determining that a maximum misclosure of the confirmed rays meets a distance threshold.
7 . The method of claim 1 , further comprising:
determining that the plurality of redundant observations correspond to a same object of the pole-like object based on a distance threshold from respective epipolar lines of the plurality of images.
8 . An apparatus comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, within the at least one processor, cause the apparatus to perform at least the following:
retrieve a machine learning model that is trained to detect one or more semantic keypoints associated with a pole-like object in a plurality of images;
perform a photogrammetric triangulation of the plurality of redundant observations to determine three-dimensional coordinate data of the pole-like object, the one or more semantic keypoints, or a combination thereof;
process the plurality of images using the machine learning model to generate a plurality of redundant observations of the pole-like object, wherein the plurality of redundant observations includes respective detections of the one or more semantic keypoints;
determine a location, a geometric attribute, or a combination thereof of the pole-like object based on the three-dimensional coordinate data; wherein the geometric attribute includes a length, an orientation, or a combination thereof; and
provide the plurality of redundant observations as an output.
9 . The apparatus of claim 8 , wherein the one or more semantic keypoints include a bottom point, a top point, a midpoint, or a combination thereof of a straight section of the pole-like object.
10 . The apparatus of claim 8 , wherein the machine learning model is trained using a plurality of reference images respectively labeled with the one or more semantic keypoints under one or more different contexts.
11 . The apparatus of claim 10 , wherein one or more semantic keypoints are labeled in a designated order.
12 . A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:
processing a plurality of images using a machine learning model to generate a plurality of redundant observations of an object respectively depicted in the plurality of images;
performing a photogrammetric triangulation of the plurality of redundant observations to determine three-dimensional coordinate data of the object;
wherein the machine learning model generates the plurality of redundant observations based on detecting one or more semantic keypoints comprising a geometric representation of the pole-like object, and wherein the three-dimensional coordinate data include respective three-dimensional coordinates of the one or more semantic keypoints;
determining a location, a geometric attribute, or a combination thereof of the pole-like object based on the three-dimensional coordinate data; wherein the geometric attribute includes a length, an orientation, or a combination thereof; and
providing the three-dimensional coordinate data and the geometric attribute of the object as an output.
13 . The non-transitory computer-readable storage medium of claim 12 , wherein the machine learning model is a Mask Region-Based Convolutional Neural Network (Mask R-CNN) that is trained using a plurality of reference images respectively labeled with a plurality of ground truth semantic keypoints of a plurality of objects.
14 . The non-transitory computer-readable storage medium of claim 12 , wherein the apparatus is caused to further perform:
back projecting the three-dimensional coordinate data into the plurality of images to perform a validation of the three-dimensional coordinated data,
wherein the validation is based on determining the back-projected three-dimensional coordinate data falls within a threshold distance of the one or more redundant observations.