Graphical user interface for enhanced unmanned aerial vehicle flight along computed splines
Technology for generating and displaying a graphical user interface for operating an unmanned aerial vehicle (UAV) is disclosed herein that generates and updates a representation of a spline flight path. In various implementations, a graphical user interface detects user interactions with a remote control device directing the flight control subsystem of the UAV to record keyframes and to compute a spline based on the keyframes during flight. The graphical user interface displays a real-time perspective of the UAV with a representation of the spline and the keyframes overlaying the view. The graphical user interface continually updates the representation as the UAV flies and when the spline is updated as the keyframes are updated.
1. A method for controlling an aircraft, the method comprising:
continually presenting, by a user computing device, a graphical user interface including a display of a view of a physical environment from a perspective of the aircraft, the view generated based on sensor data from a sensor device onboard the aircraft;
detecting, by the user computing device, user interactions with the graphical user interface indicating requests to add keyframes, wherein each keyframe comprises a location in three-dimensional space of the aircraft; and
continually displaying, by the user computing device, a linear representation of a computed spline of the aircraft, wherein the computed spline comprises a flight path between the keyframes and wherein the linear representation of the computed spline includes graphical marker representations of the keyframes spaced at locations on the computed spline representative of spatial locations of the keyframes within the physical environment.
2. The method for controlling the aircraft of claim 1 , wherein a portion of the linear representation of the computed spline is color-coded according to the portion of the computed spline flown by the aircraft.
3. The method for controlling the aircraft of claim 2 , wherein the linear representation of the computed spline comprises a graphical device for adding a keyframe to the computed spline.
4. The method for controlling the aircraft of claim 3 , further comprising detecting, by the user computing device, a user selection of a point on the linear representation of the computed spline causing the aircraft to fly directly to a location on the computed spline corresponding to the point.
5. The method for controlling the aircraft of claim 1 , further comprising displaying, by the user computing device, a graphical device for controlling a speed of the aircraft, wherein the graphical device is color-coded according to a dynamic feasibility of the aircraft with respect to speed.
6. The method for controlling the aircraft of claim 1 , further comprising continually displaying, by the user computing device, a graphical representation of the computed spline overlaid on the view of the physical environment from the perspective of the aircraft.
7. The method for controlling the aircraft of claim 6 , further comprising dynamically varying a width of the graphical representation of the computed spline according to a distance from the aircraft.
8. The method for controlling the aircraft of claim 7 , wherein the graphical representation of the computed spline comprises markers indicating the spatial locations of the multiple keyframes, and wherein the method further comprises dynamically resizing the markers according to a distance from the aircraft.
9. A computing apparatus comprising:
one or more non-transitory computer-readable storage media;
one or more processors operatively coupled with the one or more non-transitory computer-readable storage media; and
program instructions stored on the one or more non-transitory computer-readable storage media that, when executed by the one or more processors, direct the computing apparatus to at least:
continually present a graphical user interface including a display of a view of a physical environment from a perspective of an aircraft, wherein the view is generated based on sensor data from a sensor device onboard the aircraft;
detect user interactions with the graphical user interface indicating requests to add keyframes, wherein each keyframe comprises a location in three-dimensional space of the aircraft; and
continually display a linear representation of a computed spline of the aircraft, wherein the computed spline comprises a flight path between the keyframes and wherein the linear representation of the computed spline includes graphical marker representations of the keyframes spaced at locations on the computed spline representative of spatial locations of the keyframes within the physical environment.
10. The computing apparatus of claim 9 , wherein a portion of the linear representation of the computed spline is color-coded according to the portion of the computed spline flown by the aircraft.
11. The computing apparatus of claim 10 , wherein the linear representation of the computed spline comprises a graphical device for adding a keyframe to the computed spline.
12. The computing apparatus of claim 11 , wherein the program instructions further direct the computing apparatus to detect a user selection of a point on the linear representation of the computed spline causing the aircraft to fly directly to a location on the computed spline corresponding to the point.
13. The computing apparatus of claim 9 , wherein the program instructions further direct the computing apparatus to display a graphical device for controlling a speed of the aircraft, wherein the graphical device is color-coded according to a dynamic feasibility of the aircraft with respect to speed.
14. The computing apparatus of claim 9 , wherein the program instructions further direct the computing apparatus to continually display a graphical representation of the computed spline overlaid on the view of the physical environment from the perspective of the aircraft.
15. The computing apparatus of claim 14 , wherein to display the graphical representation of the computing spline, the program instructions direct the computing apparatus to dynamically vary a width of the graphical representation of the computed spline according to a distance from the aircraft.
16. The computing apparatus of claim 15 , wherein the graphical representation of the computed spline comprises markers indicating the spatial locations of the multiple keyframes, and wherein the program instructions further direct the computing apparatus to dynamically resize the markers according to a distance from the aircraft.
17. One or more non-transitory computer-readable storage media having program instructions stored thereon that, when executed by one or more processors of a computing device, direct the computing device to at least:
continually present a graphical user interface including a display of a view of a physical environment from a perspective of an aircraft, wherein the view is generated based on sensor data from a sensor device onboard the aircraft;
detect user interactions with the graphical user interface indicating requests to add keyframes, wherein each keyframe comprises a location in three-dimensional space of the aircraft; and
continually display a linear representation of a computed spline of the aircraft, wherein the computed spline comprises a flight path between the keyframes and wherein the linear representation of the computed spline includes graphical marker representations of the keyframes spaced at locations on the computed spline representative of spatial locations of the keyframes within the physical environment.
18. The one or more non-transitory computer-readable storage media of claim 17 , wherein a portion of the linear representation of the computed spline is color-coded according to the portion of the computed spline flown by the aircraft.
19. The one or more non-transitory computer-readable storage media of claim 18 , wherein the linear representation of the computed spline comprises a graphical device for adding a keyframe to the computed spline.
20. The one or more non-transitory computer-readable storage media of claim 19 , wherein the program instructions further direct the computing device to detect a user selection of a point on the linear representation of the computed spline causing the aircraft to fly directly to a location on the computed spline corresponding to the point.