Systems and methods for active-light based precision localization of aircrafts in GPS-denied environments
Systems and methods of providing guidance to assist eVTOL aerial vehicles in performing landing and takeoff operations at landing locations in GPS-denied environments are disclosed. An exemplary system includes an aerial vehicle comprising a camera configured to generate images based on information transmitted by a plurality of light sources located adjacent a landing surface for the aerial vehicle and a controller circuit configured to receive the generated images and determine a position and an orientation of the aerial vehicle based on the received images, wherein the light sources are arranged in a predetermined pattern on the landing surface, and wherein a characteristic of light emitted from each of the light sources is modulated with respect to time.
1. An aerial vehicle, comprising:
a camera configured to generate images based on information received from a plurality of light sources located on a landing surface for the aerial vehicle;
a processor associated with the camera and configured to receive the images and to perform the following operations:
detecting, using a detection algorithm, light sources in the image, the light sources arranged on the landing surface and configured to emit light detectable by the camera;
performing association of locations in the image representing the detected light sources to corresponding locations of the light sources on the landing surface, wherein the processor is configured to perform the association in a first mode of operation and a second mode of operation;
executing one or more association algorithms in the first mode of operation and generating a confidence score of the association;
executing one or more tracking algorithms in the second mode of operation, based on the confidence score obtained from the first mode of operation; and
determining one of a location or orientation of the aerial vehicle based on the performed association.
2. The aerial vehicle of claim 1 , wherein the processor is further configured to automatically switch between the first and the second modes of operation based on a predetermined threshold confidence score.
3. The aerial vehicle of claim 2 , wherein the processor is further configured to request user input to switch between the first and the second modes of operation based on a predetermined threshold confidence score.
4. The aerial vehicle of claim 1 , wherein the processor is further configured to execute the first and the second modes of operation sequentially.
5. The aerial vehicle of claim 1 , wherein the processor is further configured to:
switch from the first mode of operation to the second mode of operation, and
after switching from the first mode of operation to the second mode of operation, execute the first and the second modes of operation in parallel.
6. The aerial vehicle of claim 1 , wherein the one or more association algorithms comprises a grid association algorithm, a Thin Plate Spline Robust Point Matching (TPS-RPM) association algorithm, or an Iterative Closest Point (ICP) algorithm.
7. The aerial vehicle of claim 1 , wherein the one or more tracking algorithms comprises local association point tracking or pose-based point tracking.
8. The aerial vehicle of claim 1 , wherein the detection algorithm is configured to detect a modulation of a characteristic of the plurality of light sources with respect to time.
9. The aerial vehicle of claim 8 , wherein the plurality of light sources comprises a combination of linear light sources and point light sources.
10. The aerial vehicle of claim 1 , wherein executing the one or more association algorithms comprises the steps of:
normalizing the locations in the image representing the detected light sources in a cartesian coordinate space;
transforming the normalized locations to curves in a polar coordinate space, wherein collinear normalized locations in the cartesian coordinate space form curves intersecting at a common point in the polar coordinate space;
discretizing the polar coordinate space into a plurality of bins, each bin represented by a value indicating the number of times a curve passes through a location of the bin;
transforming, upon determining whether the bin value exceeds a predetermined threshold, the location of the bin in the polar coordinate space to the cartesian coordinate space;
forming lines in the cartesian coordinate space, each line connecting at least a number of points equal to the value of the corresponding bin;
grouping, using a clustering algorithm, substantially parallel lines and forming a rectangular frame for an integer grid space from the grouped lines;
calculating a homography matrix configured to move the points from the cartesian coordinate space to the integer grid; and
mapping each point to the integer grid using the calculated homography matrix.
11. The aerial vehicle of claim 10 , wherein the processor is further configured to normalize the locations in the image by constructing a transformation matrix to compute a mean of the locations and setting the variance of the locations to unity.
12. The aerial vehicle of claim 10 , wherein the processor is further configured to normalize the locations in the image by rotating the cartesian coordinate space by an angle to compensate a rotation caused by an angle of approach of the aerial vehicle toward the landing surface.
13. The aerial vehicle of claim 10 , wherein the processor is further configured to increment the bin value by one for every instance of a curve passing through the location of the bin.
14. The aerial vehicle of claim 10 , wherein the processor is further configured to refine one or more lines in the cartesian coordinate space by rejecting the one or more lines based on a fit to the detected locations of the light sources in the image.
15. The aerial vehicle of claim 10 , wherein the processor is further configured to label each location on the integer grid with a reference character, and wherein the labels are based on a predefined sequence.
16. The aerial vehicle of claim 10 , wherein the processor is further configured to map each point to the integer grid indicating an offset distance, the offset distance being a distance between a reference location on the integer grid and a corresponding mapped point.
17. The aerial vehicle of claim 10 , wherein the processor is further configured to reject a false detection from the association based on the offset distance, the rejection of the false detection comprises comparing the offset distance to a threshold offset distance.
18. The aerial vehicle of claim 10 , wherein the processor is further configured to identify, as a false detection, the mapped points for which the offset distance is larger than the threshold offset distance, and to reject the association upon determining that the number of false detections exceeds an allowable threshold.
19. The aerial vehicle of claim 1 , wherein determining the location of the aerial vehicle or the orientation of the aerial vehicle is further based on information from one or more of a global positioning system (GPS) or an internal navigation system (INS).
20. The aerial vehicle of claim 1 , further comprising:
a controller configured to acuate a component of the aerial vehicle based on the determined location or orientation of the aerial vehicle;
wherein the component comprises one of a lift propeller, a tilt propeller, a tilt actuator, or a control surface.
21. A method of operating an aerial vehicle, comprising:
generating images with a camera based on information received from a plurality of light sources located on a landing surface for the aerial vehicle;
detecting, using a detection algorithm, light sources in the image, the light sources arranged on the landing surface and configured to emit light detectable by the camera;
performing association of locations in the image representing the detected light sources to corresponding locations of the light sources on the landing surface, wherein performing the association comprises a first mode of operation and a second mode of operation, wherein
the first mode of operation comprises executing one or more association algorithms and generating a confidence score of the association;
the second mode of operation comprises executing one or more tracking algorithms based on the confidence score obtained from the first mode of operation; and
determining one of a location or orientation of the aerial vehicle based on the performed association.
22. A navigation system for an aerial vehicle, comprising:
a camera configured to generate images based on information received from a plurality of light sources arranged in a predetermined pattern on a landing surface for an aerial vehicle;
a processor associated with the camera and configured to receive the images and to perform the following operations:
activating, using the processor, a camera mounted on the aerial vehicle to enable receiving an input signal associated with light emitted from light sources arranged in a predetermined pattern on the landing surface for the aerial vehicle, the light having a characteristic that is modulated with respect to time;
enabling the camera to generate at least two images of the light sources based on the received input signal;
detecting, using a detection algorithm, the light sources in the at least two images;
performing an association of locations in the image representing the detected light sources to corresponding locations of the light sources on the landing surface, wherein the processor is configured to perform the association in a first mode of operation and a second mode of operation;
executing one or more association algorithms in the first mode of operation;
executing one or more tracking algorithms in the second mode of operation, based results obtained from the first mode of operation; and
determining one of a location or an orientation of the aerial vehicle based on the performed association.
23. The navigation system of claim 22 , further comprising:
the plurality of light sources arranged in the predetermined pattern on the landing surface.
24. The navigation system of claim 23 , wherein the landing surface comprises a landing surface of a vertiport.
25. The navigation system of claim 23 , wherein the landing surface comprises a portable landing surface, the portable landing surface comprising one of a re-deployable landing surface, a rollable mat, a fabric, a tarp, a netting, or a mesh.
26. The navigation system of claim 23 , wherein a characteristic of light emitted from each of the light sources is modulated with respect to time.
27. The navigation system of claim 22 , further comprising:
a controller configured to actuate a component of the aerial vehicle based on the determined location or an orientation of the aerial vehicle,
wherein the component comprises one of a lift propeller, a tilt propeller, a tilt actuator, or a control surface.