Aerial refueling systems and methods

1 . A method comprising:

receiving a two-dimensional (2D) image from a camera;

cropping the 2D image based on predefined feature areas of interest of a target object to produce a plurality of cropped images;

resizing one or more of the plurality of cropped images responsive to the one or more of the plurality of cropped images having a resolution different from a training resolution associated with training of a previously trained keypoint detection neural network to produce one or more resized images, the one or more resized images having a resized resolution equal to the training resolution;

determining 2D keypoints of the target object within a plurality of input images by inputting the plurality of input images into the keypoint detection neural network, the plurality of input images comprising the one or more resized images;

estimating a 6 degrees-of-freedom (6DOF) pose based on the 2D keypoints and a three-dimensional (3D) model of the target object to produce an estimated 6DOF pose; and

outputting the 6DOF pose; wherein the target object is a receiving aircraft in a close air operation, and wherein the plurality of cropped images comprises a first cropped image and a second cropped image and neither the first cropped image nor the second cropped image is entirely contained within the other.

2 . The method of claim 1 , wherein each of the cropped images includes a plurality of keypoints.

3 . The method of claim 1 , wherein the close air operation is a refueling operation.

4 . The method of claim 1 , further comprising, before producing the plurality of cropped images, performing an initial single crop of the 2D image.

5 . The method of claim 1 , further comprising performing automated refueling operations responsive to the 6DOF pose.

6 . The method of claim 1 , wherein:

the first cropped image corresponds to a first feature area of interest of the predefined feature areas of interest;

the second cropped image corresponds to a second feature area of interest of the predefined feature areas of interest distinct from the first feature area of interest; and

the first cropped image does not overlap with the second cropped image.

7 . A tanker aircraft comprising:

a refueling boom;

a camera configured to generate a two-dimensional (2D) image of an in-flight refueling operation between a receiver aircraft and the tanker aircraft;

a processor; and

non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:

cropping the 2D image based on predefined feature areas of interest of the receiver aircraft to produce a plurality of cropped images;

resizing one or more of the plurality of cropped images responsive to the one or more of the plurality of cropped images having a resolution different from a training resolution associated with training of a previously trained keypoint detection neural network to produce one or more resized images, the one or more resized images having a resized resolution equal to the training resolution;

determining 2D keypoints of the receiver aircraft within a plurality of input images by inputting the plurality of input images into the keypoint detection neural network, the plurality of input images comprising the one or more resized images;

estimating a 6 degrees-of-freedom (6DOF) pose based on the 2D keypoints and a three-dimensional (3D) model of the receiver aircraft to produce an estimated 6DOF pose; and

outputting the 6DOF pose; wherein the plurality of cropped images comprises a first cropped image and a second cropped image and neither the first cropped image nor the second cropped image is entirely contained within the other.

8 . The tanker aircraft of claim 7 , wherein each of the cropped images includes a plurality of keypoints.

9 . The tanker aircraft of claim 7 , wherein the code is further configured to cause the processor to perform an initial single crop of the 2D image before producing the plurality of cropped images.

10 . The tanker aircraft of claim 7 , wherein the tanker aircraft further comprises:

an automated refueling system; and

a boom operator system; or

a pilot director light system,

wherein the code is further configured to cause the processor to control operations of the automated refueling system, the boom operator system, or the pilot director light system based on the 6DOF pose of the receiver aircraft.

11 . The tanker aircraft of claim 7 , wherein:

the first cropped image corresponds to a first feature area of interest of the predefined feature areas of interest;

the second cropped image corresponds to a second feature area of interest of the predefined feature areas of interest distinct from the first feature area of interest; and

the first cropped image does not overlap with the second cropped image.

12 . A system comprising:

a camera configured to generate a two-dimensional (2D) image of an in-flight refueling operation between a receiver aircraft and a tanker aircraft;

a processor; and

non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:

cropping the 2D image based on predefined feature areas of interest of the receiver aircraft to produce a plurality of cropped images;

resizing one or more of the plurality of cropped images responsive to the one or more of the plurality of cropped images having a resolution different from a training resolution associated with training of a previously trained keypoint detection neural network to produce one or more resized images, the one or more resized images having a resized resolution equal to the training resolution;

determining 2D keypoints of the receiver aircraft within a plurality of input images by inputting the plurality of input images into the keypoint detection neural network, the plurality of input images comprising the one or more resized images;

estimating a 6 degrees-of-freedom (6DOF) pose based on the 2D keypoints and a three-dimensional (3D) model of the receiver aircraft to produce an estimated 6DOF pose; and

outputting the 6DOF pose; wherein the plurality of cropped images comprises a first cropped image and a second cropped image and neither the first cropped image nor the second cropped image is entirely contained within the other.

13 . The system of claim 12 , wherein each of the cropped images includes a plurality of keypoints.

14 . The system of claim 12 , wherein the code is further configured to cause the processor to perform automated refueling operations responsive to the 6DOF pose.

15 . The system of claim 12 , wherein the code is further configured to generate an uncertainty value of the 6DOF pose.

16 . The system of claim 12 , wherein:

the first cropped image corresponds to a first feature area of interest of the predefined feature areas of interest;

the second cropped image corresponds to a second feature area of interest of the predefined feature areas of interest distinct from the first feature area of interest; and

the first cropped image does not overlap with the second cropped image.