Landing site localization for dynamic control of an aircraft toward a landing site
A system having components coupled to an aircraft and components remote from the aircraft processes sensor-derived data, transmits information between aircraft system components and remote system components, and dynamically generates updated analyses of position and orientation of the aircraft relative to a desired landing site, while the aircraft is in flight toward the desired landing site. Based on the position and orientation information, the system generates instructions for flight control of the aircraft toward a flight path to the landing site, and can update flight control instructions as new data is received and processed.
1. A method for landing site localization, the method comprising:
in response to detecting an aircraft within a proximity threshold to a landing site, receiving an image taken from a camera subsystem coupled to the aircraft, the image capturing the landing site within a field of view and comprising a medium wave infrared image;
transmitting the image to an entity remote from the aircraft;
from the entity, receiving a packet characterizing a reference position of a reference object associated with the landing site, the packet comprising a set of coordinate elements upon the image selected by an input device associated with the entity, and the set of coordinate elements correspond to a boundary of the reference object;
from the reference position, generating an image-estimated pose of the aircraft;
updating a pose trajectory of the aircraft upon processing the image-estimated pose and an inertial measurement unit (IMU) output from an IMU coupled to the aircraft, the IMU output having a time stamp corresponding to the image; and
based upon the updated pose trajectory, generating an updated set of instructions for a flight computer of the aircraft, the set of instructions for flight control of the aircraft toward a flight path to the landing site.
2. The method of claim 1 , wherein generating the image-estimated pose comprises:
generating a color-filtered image upon processing the image with a color filter corresponding to a color feature of the reference object;
generating a transformed image upon transforming the color-filtered image from a first dimensional space to a second dimensional space;
extracting the reference position from the transformed image; and
generating the image-estimated pose upon processing the reference position, the transformed image, and a set of images of candidate reference objects with a matching operation.
3. The method of claim 2 , wherein the reference object comprises a set of landing site lights.
4. The method of claim 1 , wherein the reference object comprises at least one of a landing site boundary, a pattern of landing site markers, and a set of landing site lights.
5. The method of claim 1 , wherein updating the pose trajectory comprises replacing components of the IMU output with components of the image-estimated pose.
6. The method of claim 5 , wherein updating the pose trajectory further comprises forward propagating components of the image-estimated pose to a set of IMU outputs generated subsequent to the time stamp.
7. The method of claim 1 , wherein receiving the image further comprises receiving a longwave infrared (LWIR) image.
8. The method of claim 1 , wherein the camera subsystem comprises:
a first camera mounted to a port side of the aircraft and a second camera mounted to a starboard side of the aircraft,
wherein the image comprises a pair of stereoscopic images, and
wherein receiving the packet characterizing the reference position comprises transmitting the pair of stereoscopic images to the entity through a user interface device comprising a display, and receiving a selection of a bounding feature corresponding to the reference position through the user interface device.
9. A system for landing site localization, the system comprising:
a camera subsystem mounted to an aircraft with a field of view of a flight path of the aircraft;
an inertial measurement unit (IMU) mounted to the aircraft;
a data transmission subsystem in communication with the camera subsystem and the IMU; and
a computing system comprising machine-readable instructions in non-transitory media for:
transmitting, from the data transmission subsystem to a component of the computing system remote from the aircraft, an image taken from the camera subsystem that captures a landing site and comprises a medium wave infrared image,
receiving, at the component, a packet characterizing a reference position of a reference object associated with the landing site within the image, the packet comprising a set of coordinate elements upon the image selected by an input device associated with the entity, and the set of coordinate elements correspond to a boundary of the reference object;
from the reference position, generating an image-estimated pose of the aircraft,
updating a pose trajectory of the aircraft upon processing the image-estimated pose and an output from the IMU having a time stamp corresponding to the image; and
based upon the pose trajectory, generating a set of instructions for flight control of the aircraft toward a flight path to the landing site.
10. The system of claim 9 , wherein the camera subsystem further comprises a longwave infrared (LWIR) sensor.
11. The system of claim 9 , wherein the camera subsystem comprises a first camera mounted to a port side of the aircraft and a second camera mounted to a starboard side of the aircraft, wherein the image comprises a stereoscopic image, and wherein the computing system comprises instructions for generating the image-estimated pose of the aircraft from the stereoscopic image.
12. The system of claim 9 , further comprising an electronic interface between the computing system and a flight computer of the aircraft, the flight computer coupled to and controlling operational configurations of control surfaces of the aircraft, and the electronic interface operable in a mode that transmits the set of instructions to the flight computer and controls flight of the aircraft toward the flight path.