Correcting erroneous UAV positioning information using semantically segmented images
In some embodiments, a method for autonomous navigation of an unmanned aerial vehicle (UAV) is provided. The UAV determines a tracked position using at least one positioning sensor of the UAV. The UAV captures an image using a camera of the UAV. The UAV determines a visual position confidence area using the captured image. The UAV checks the tracked position using the visual position confidence area to determine whether the tracked position is accurate. In response to determining that the tracked position is not accurate, the UAV causes a corrective action based on the visual position confidence area to be taken.
1 . A non-transitory computer-readable medium having logic stored thereon that, in response to execution by one or more processors of an unmanned aerial vehicle (UAV), causes the UAV to perform actions comprising:
determining, by the UAV, a tracked position using at least one positioning sensor of the UAV;
capturing, by the UAV, an image using a camera of the UAV;
processing, by the UAV, the captured image to apply semantic labels to the captured image;
determining, by the UAV, whether the captured image depicts a scene usable for positioning based on the semantic labels; and
in response to determining that the captured image depicts a scene usable for positioning:
determining, by the UAV, a visual position confidence area using the captured image, wherein determining the visual position confidence area includes:
determining correlation scores between the semantic labels applied to the image and reference labels of a reference map; and
assigning locations depicted in the reference map having correlation scores greater than a threshold value to be within the visual position confidence area;
checking, by the UAV, the tracked position using the visual position confidence area to determine whether the tracked position is accurate; and
in response to determining that the tracked position is not accurate, causing, by the UAV, a corrective action based on the visual position confidence area;
wherein determining whether the captured image depicts a scene usable for positioning includes at least one of:
counting the semantic labels applied to the captured image to determine a number of semantic labels and comparing the number of semantic labels to a threshold number of semantic labels; or
counting types of the semantic labels applied to the captured image to determine a number of semantic label types and comparing the number of semantic label types to a threshold number of semantic label types.
2 . The non-transitory computer-readable medium of claim 1 , wherein checking the tracked position using the visual position confidence area to determine whether the tracked position is accurate includes:
determining whether the tracked position is within the visual position confidence area.
3 . The non-transitory computer-readable medium of claim 2 , wherein causing the corrective action based on the visual position confidence area includes:
in response to determining that a size of the visual position confidence area is less than a threshold size, setting the tracked position based on the visual position confidence area.
4 . The non-transitory computer-readable medium of claim 2 , wherein causing a corrective action based on the visual position confidence area includes:
in response to determining that a size of the visual position confidence area is not less than a threshold size, aborting a route being traveled by the UAV.
5 . The non-transitory computer-readable medium of claim 4 , wherein aborting the route being traveled by the UAV includes causing the UAV to land.
6 . The non-transitory computer-readable medium of claim 1 , wherein determining the tracked position using the at least one positioning sensor of the UAV includes using an extended Kalman filter (EKF) to process signals from the at least one positioning sensor.
7 . A method for autonomous navigation of an unmanned aerial vehicle (UAV), the method comprising:
determining, by the UAV, a tracked position using at least one positioning sensor of the UAV;
capturing, by the UAV, an image using a camera of the UAV;
processing, by the UAV, the captured image to apply semantic labels to the captured image;
determining, by the UAV, whether the captured image depicts a scene usable for positioning based on the semantic labels; and
in response to determining that the captured image depicts a scene usable for positioning:
determining, by the UAV, a visual position confidence area using the captured image, wherein determining the visual position confidence area includes:
determining correlation scores between the semantic labels applied to the image and reference labels of a reference map; and
assigning locations depicted in the reference map having correlation scores greater than a threshold value to be within the visual position confidence area;
checking, by the UAV, the tracked position using the visual position confidence area to determine whether the tracked position is accurate; and
in response to determining that the tracked position is not accurate, causing, by the UAV, a corrective action based on the visual position confidence area;
wherein determining whether the captured image depicts a scene usable for positioning includes at least one of:
counting the semantic labels applied to the captured image to determine a number of semantic labels and comparing the number of semantic labels to a threshold number of semantic labels; or
counting types of the semantic labels applied to the captured image to determine a number of semantic label types and comparing the number of semantic label types to a threshold number of semantic label types.
8 . The method of claim 7 , wherein checking the tracked position using the visual position confidence area to determine whether the tracked position is accurate includes:
determining whether the tracked position is within the visual position confidence area.
9 . The method of claim 8 , wherein causing the corrective action based on the visual position confidence area includes:
in response to determining that a size of the visual position confidence area is less than a threshold size, setting the tracked position based on the visual position confidence area.
10 . The method of claim 8 , wherein causing a corrective action based on the visual position confidence area includes:
in response to determining that a size of the visual position confidence area is not less than a threshold size, aborting a route being traveled by the UAV.
11 . The method of claim 10 , wherein aborting the route being traveled by the UAV includes causing the UAV to land.
12 . The method of claim 7 , wherein determining the tracked position using the at least one positioning sensor of the UAV includes using an extended Kalman filter (EKF) to process signals from the at least one positioning sensor.