Vision-based landing system
A system includes one or more cameras configured to attach to an aircraft and capture a plurality of images. The plurality of images includes a first image including a runway and a subsequently captured second image including the runway. The system includes an aircraft computing system configured to identify common features in the first and second images, determine changes in locations of the common features between the first and second images, and determine a predicted landing location of the aircraft in the second image based on the changes in locations of the common features. The aircraft computing system is configured to abort landing on the runway based on the predicted landing location relative to the runway.
1. A system comprising:
one or more cameras configured to attach to an aircraft and capture a plurality of images, wherein the plurality of images includes a first image including a runway and a subsequently captured second image including the runway; and
an aircraft computing system configured to:
identify common features in the first and second images;
determine changes in locations of the common features between the first and second images;
determine an image aspect associated with the first and second images, wherein the image aspect comprises at least one of: a location of an invariant region associated with the first and second images, an intersection of lines associated with each of the common features, a change in orientation of the aircraft between the first and second image, or a reprojection of one of the first or second images such that the first and second images represent a view of the runway from the same orientation;
determine a predicted landing location of the aircraft in the second image based on the changes in locations of the common features; and
abort landing on the runway based on the predicted landing location relative to the runway.
2. The system of claim 1 , wherein the aircraft computing system is configured to abort landing by executing a missed approach flight procedure.
3. The system of claim 1 , wherein the aircraft computing system is configured to:
identify at least one of a runway centerline, a runway threshold, and a runway edge in one or more of the plurality of images; and
abort landing on the runway based on the predicted landing location relative to at least one of the runway centerline, the runway threshold, and the runway edge.
4. The system of claim 1 , wherein the aircraft computing system is configured to:
determine a location of an invariant region associated with the first and second images based on the changes in locations of the common features; and
determine the predicted landing location in the second image based on the location of the invariant region.
5. The system of claim 4 , wherein the aircraft computing system is configured to:
determine lines associated with movement of each of the common features; determine an intersection region based on a location where one or more of the lines intersect; and
determine the location of the invariant region based on the location of the intersection region.
6. The system of claim 4 , wherein the invariant region is defined as an invariant point.
7. The system of claim 1 , wherein the aircraft computing system is configured to:
determine a change in orientation of the aircraft between the first and second images based on data acquired from one or more aircraft orientation sensors; and
determine the predicted landing location based on the change in orientation of the aircraft.
8. The system of claim 7 , wherein the aircraft computing system is configured to:
reproject one of the first and second images such that the first and second images represent a view of the runway from the same orientation; and
determine the predicted landing location using the first and second images that represent a view of the runway from the same orientation.
9. The system of claim 1 , further comprising a ground-based computing system, wherein the aircraft computing system is configured to transmit landing validation data to the ground-based computing system, wherein the landing validation data includes the second image and indicates the predicted landing location, and wherein the ground-based computing system is configured to render the predicted landing location on the second image based on the received landing validation data.
10. The system of claim 9 , wherein the ground-based computing system is configured to:
receive a user-input landing abort command; and
transmit the landing abort command to the aircraft, wherein the aircraft computing system is configured to receive the transmitted landing abort command and execute a missed approach flight procedure in response to the received landing abort command.
11. The system of claim 9 , wherein the ground-based computing system is configured to:
automatically generate a landing abort command based on the predicted landing location relative to the runway; and
transmit the landing abort command to the aircraft, wherein the aircraft computing system is configured to receive the transmitted landing abort command and execute a missed approach flight procedure in response to the received landing abort command.
12. The system of claim 9 , wherein the ground-based computing system is configured to render the predicted landing location based on a level of uncertainty associated with determining the predicted landing location.
13. The system of claim 9 , wherein the ground-based computing system is configured to:
render images received from the aircraft on a display; and render the predicted landing location over the images.
14. The system of claim 9 , wherein the landing validation data includes image coordinates that define the predicted landing location.
15. The system of claim 9 , wherein the landing validation data includes the predicted landing location rendered over the second image.
16. A non-transitory computer-readable medium comprising computer-executable instructions configured to cause one or more processing units of an aircraft to:
capture a plurality of images from one or more cameras attached to the aircraft, wherein the plurality of images includes a first image including a runway and a subsequently captured second image including the runway;
identify common features in the first and second images; determine changes in locations of the common features between the first and second images;
determine an image aspect associated with the first and second images, wherein the image aspect comprises at least one of: a location of an invariant region associated with the first and second images, an intersection of lines associated with each of the common features, a change in orientation of the aircraft between the first and second image, or a reprojection of one of the first or second images such that the first and second images represent a view of the runway from the same orientation;
determine a predicted landing location of the aircraft in the second image based on the changes in locations of the common features; and
abort landing on the runway based on the predicted landing location relative to the runway.
17. The computer-readable medium of claim 16 , further comprising instructions that cause the one or more processing units of the aircraft to abort landing by executing a missed approach flight procedure.
18. The computer-readable medium of claim 16 , further comprising instructions that cause the one or more processing units of the aircraft to:
identify at least one of a runway centerline, a runway threshold, and a runway edge in one or more of the plurality of images; and
abort landing on the runway based on the predicted landing location relative to at least one of the runway centerline, the runway threshold, and the runway edge.
19. The computer-readable medium of claim 16 , further comprising instructions that cause the one or more processing units of the aircraft to:
determine a location of an invariant region associated with the first and second images based on the changes in locations of the common features; and
determine the predicted landing location in the second image based on the location of the invariant region.
20. The computer-readable medium of claim 19 , further comprising instructions that cause the one or more processing units of the aircraft to:
determine lines associated with movement of each of the common features;
determine an intersection region based on a location where one or more of the lines intersect; and
determine the location of the invariant region based on the location of the intersection region.
21. The computer-readable medium of claim 19 , wherein the invariant region is defined as an invariant point.
22. The computer-readable medium of claim 16 , further comprising instructions that cause the one or more processing units of the aircraft to:
determine a change in orientation of the aircraft between the first and second images based on data acquired from one or more aircraft orientation sensors; and
determine the predicted landing location based on the change in orientation of the aircraft.
23. The computer-readable medium of claim 22 , further comprising instructions that cause the one or more processing units of the aircraft to:
reproject one of the first and second images such that the first and second images represent a view of the runway from the same orientation; and
determine the predicted landing location using the first and second images that represent a view of the runway from the same orientation.
24. The computer-readable medium of claim 16 , further comprising instructions that cause the one or more processing units of the aircraft to transmit landing validation data to a ground-based computing system, wherein the landing validation data includes the second image and indicates the predicted landing location, and wherein the computer-readable medium comprises instructions that cause an additional one or more processing units of the ground-based computing system to render the predicted landing location on the second image based on the received landing validation data.
25. The computer-readable medium of claim 24 , further comprising instructions that cause the additional one or more processing units of the ground-based computing system to:
receive a user-input landing abort command; and
transmit the landing abort command to the aircraft, wherein the instructions cause the one or more processing units of the aircraft computing system to receive the transmitted landing abort command and execute a missed approach flight procedure in response to the received landing abort command.
26. The computer-readable medium of claim 24 , further comprising instructions that cause the additional one or more processing units of the ground-based computing system to:
automatically generate a landing abort command based on the predicted landing location relative to the runway; and
transmit the landing abort command to the aircraft, wherein the instructions cause the one or more processing units of the aircraft computing system to receive the transmitted landing abort command and execute a missed approach flight procedure in response to the received landing abort command.
27. The computer-readable medium of claim 24 , further comprising instructions that cause the additional one or more processing units of the ground-based computing system to render the predicted landing location based on a level of uncertainty associated with determining the predicted landing location.
28. The computer-readable medium of claim 24 , further comprising instructions that cause the additional one or more processing units of the ground-based computing system to:
render images received from the aircraft on a display; and render the predicted landing location over the images.
29. The computer-readable medium of claim 24 , wherein the landing validation data includes image coordinates that define the predicted landing location.
30. The computer-readable medium of claim 24 , wherein the landing validation data includes the predicted landing location rendered over the second image.