Methods and systems for detecting foreign objects on a landing surface

1 . A method of detecting foreign objects on a landing surface comprising:

capturing an image from one or more cameras associated with a vehicle;

detecting the landing surface present in the captured image;

retrieving a reference image for the detected landing surface from a database;

extracting a plurality of feature points present in both the captured image and the reference image;

determining a transformation between the captured image and the reference image by correlating the plurality of feature points between the captured image and the reference image;

creating a virtual image by applying the transformation to one of the captured image or the reference image;

comparing the virtual image to the other one of the captured image or the reference image that was not transformed to detect one or more foreign objects based on the differences between the virtual image and the other one of the captured image or the reference image that was not transformed, wherein comparing the virtual image to the other one of the captured image or the reference image that was not transformed comprises:

generating, in an instance in which a first foreign object of the one or more foreign objects is detected, a reconstructed virtual image by removing the first foreign object from the virtual image;

generating a difference image, wherein, in an instance in which the first foreign object of the one or more foreign objects is detected, the difference image is indicative of the first foreign object; and

segmenting the difference image to identify a feature of the first foreign object; and

determining a landing surface state of the landing surface based on the difference image, wherein determining the landing surface state of the landing surface comprises:

determining, based at least in part on the feature of the first foreign object, that a landing threshold for the vehicle is satisfied; or

determining, based at least in part on the feature of the first foreign object, that the landing threshold for the vehicle is not satisfied.

2 . The method of claim 1 , wherein the one or more cameras are mounted on a lower surface of the vehicle.

3 . The method of claim 1 , wherein the plurality of feature points present in both the captured image and the reference image includes at least three feature points.

4 . The method of claim 3 , wherein correlating the plurality of feature points between the captured image and the reference image includes solving a perspective-n-point problem.

5 . The method of claim 1 , wherein the transformation includes a rotation and a translation.

6 . The method of claim 1 , wherein detecting the one or more foreign objects includes segmenting the difference image to identify the one or more foreign objects.

7 . The method of claim 1 , further including, after detecting the one or more foreign objects, displaying the one or more foreign objects on a display to one or more operators of the vehicle.

8 . A system for detecting foreign objects on a landing surface comprising:

one or more cameras associated with a vehicle;

a memory storing instructions; and

one or more processors configured to execute the instructions to perform operations including:

capturing an image from the one or more cameras associated with the vehicle;

detecting the landing surface present in the captured image;

retrieving a reference image for the detected landing surface from a database;

extracting a plurality of feature points present in both the captured image and the reference image;

determining a transformation between the captured image and the reference image by correlating the plurality of feature points between the captured image and the reference image;

creating a virtual image by applying the transformation to one of the captured image or the reference image;

comparing the virtual image to the other one of the captured image or the reference image that was not transformed to detect one or more foreign objects based on the differences between the virtual image and the other one of the captured image or the reference image that was not transformed, wherein comparing the virtual image to the other one of the captured image or the reference image that was not transformed comprises:

generating, in an instance in which a first foreign object of the one or more foreign objects is detected, a reconstructed virtual image by removing the first foreign object from the virtual image;

generating a difference image, wherein, in an instance in which the first foreign object of the one or more foreign objects is detected, the difference image is indicative of the first foreign object; and

segmenting the difference image to identify a feature of the first foreign object;

and

determining a landing surface state of the landing surface based on the difference image, wherein determining the landing surface state of the landing surface comprises:

determining, based at least in part on the feature of the first foreign object, that a landing threshold for the vehicle is satisfied; or

determining, based at least in part on the feature of the first foreign object, that the landing threshold for the vehicle is not satisfied.

9 . The system of claim 8 , wherein the one or more cameras are mounted on a lower surface of the vehicle.

10 . The system of claim 8 , wherein the plurality of feature points present in both the captured image and the reference image includes at least three feature points.

11 . The system of claim 10 , wherein correlating the plurality of feature points between the captured image and the reference image includes solving a perspective-n-point problem.

12 . The system of claim 8 , wherein the transformation includes a rotation and a translation.

13 . The system of claim 8 , wherein detecting the one or more foreign objects includes segmenting the difference image to identify the one or more foreign objects.

14 . The system of claim 8 , further including, after detecting the one or more foreign objects, displaying the one or more foreign objects on a display to one or more operators of the vehicle.

15 . A computer-implemented method for detecting foreign objects on a landing surface, the method comprising:

capturing, by one or more cameras mounted on a lower surface of a vehicle, an image including the landing surface;

detecting, by one or more processors, the landing surface present in the captured image;

retrieving, from a database, a reference image for the landing surface present in the captured image;

extracting, from both the captured image and the reference image, a plurality of corresponding feature points;

determining a transformation including a rotation and a translation between the captured image and the reference image by correlating the plurality of corresponding feature points;

creating a virtual image of the landing surface by applying the transformation to the captured image;

comparing the virtual image to the reference image to detect one or more foreign objects, wherein comparing the virtual image to the reference image comprises:

generating a reconstructed virtual image by removing the one or more foreign objects from the virtual image;

generating a difference image indicative of the one or more foreign objects; and

segmenting the difference image to detect the one or more foreign objects on the landing surface based on the differences between the virtual image and the reference image;

determining a landing surface state of the landing surface based on the difference image, wherein determining the landing surface state of the landing surface comprises:

determining, based at least in part on the feature of the one or more foreign objects, that a landing threshold for the vehicle is satisfied; or

determining, based at least in part on the feature of the one or more foreign objects, that the landing threshold for the vehicle is not satisfied; and

displaying the one or more foreign objects on a display to one or more operators of the vehicle.

16 . The method of claim 15 , wherein the plurality of corresponding feature points includes at least three feature points, and

wherein correlating the plurality of corresponding feature points includes solving a perspective-n-point problem.

17 . The method of claim 1 , wherein generating the difference image further comprises:

generating the reconstructed virtual image by applying the virtual image to a neural network; and

generating the difference image by comparing the reconstructed virtual image and the virtual image.

18 . The method of claim 1 , wherein detecting the landing surface present in the captured image comprises accessing one or more known landing surface features from the database or a second database.

19 . The method of claim 1 , wherein generating the reconstructed virtual image by removing the first foreign object from the virtual image comprises encoding the first foreign object into a fixed shape feature.