IP Library › Granted Patent US 9,928,649
Granted Patent B2
US 9,928,649 · App. 15/130,619 · Granted Mar 27, 2018

Interface for planning flight path

Inventors: Botao Hu (Atherton, CA); Jiajie Zhang (Atherton, CA)
Assignee: Amber Garage, Inc.
G06T19/003G01C21/20G06F3/04815G06F3/04845G06T13/20G06T17/05G08G5/0013G08G5/0052G08G5/0069G08G5/0086
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,928,649
App. No.
15/130,619
Granted
Mar 27, 2018
Kind
B2
Abstract

A flight path of a physical aircraft vehicle is planned. A virtual three-dimensional model of a physical environment is provided. A placement indicator is tracked within the virtual three-dimensional model of the physical environment. Tracking the placement indicator includes tracking a location and an orientation of the placement indicator within the virtual three-dimensional model. A viewfinder display window that displays a simulated image captured from a simulated camera of a simulated vehicle located at the location of the placement indicator and oriented at a direction of the orientation of the placement indicator is provided. For the physical aircraft vehicle, at least a flight path and a camera image capture are planned using the placement indicator and the viewfinder display window within the virtual three-dimensional model.

Claims (40)

1. A system for planning a flight path of a physical aircraft vehicle, comprising:

a processor configured to:

provide a virtual three-dimensional model of a physical environment;

track a placement indicator within the virtual three-dimensional model of the physical environment, wherein tracking the placement indicator includes tracking a location and an orientation of the placement indicator within the virtual three-dimensional model;

provide a computer rendered viewfinder display window that displays a simulated image captured from a simulated camera of a simulated vehicle located at the location of the placement indicator and oriented at a direction of the orientation of the placement indicator;

plan for the physical aircraft vehicle at least the flight path and a physical camera image capture using the placement indicator and the viewfinder display window within the virtual three-dimensional model, wherein the placement indicator has been utilized to specify based on the location and the orientation of the placement indicator at least one key frame of a plurality of key frames, a camera look-at line associated with a key frame included in the plurality of key frames is visually indicated in the virtual three-dimensional model, and the computer rendered viewfinder display window has been utilized to preview at least one key frame of the plurality of key frames;

calculate the flight path between physical locations corresponding the plurality of key frames specified in the virtual three-dimensional model; and

generate digital instructions that cause the physical aircraft vehicle to at least in part autonomously follow the calculated flight path; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 , wherein the virtual three-dimensional model is provided via a virtual reality headset.

3. The system of claim 1 , wherein the virtual three-dimensional model is provided via an augmented reality display device.

4. The system of claim 3 , wherein the virtual three-dimensional model is displayed on a surface of the physical environment.

5. The system of claim 1 , wherein a specified environmental condition is simulated in the virtual three-dimensional model.

6. The system of claim 1 , wherein the virtual three-dimensional model is provided via a plugin to a three-dimensional animation modeling software.

7. The system of claim 1 , wherein the physical environment corresponds to a user indicated physical location where the physical aircraft vehicle will be operated.

8. The system of claim 1 , wherein the virtual three-dimensional model was generated using images captured by the physical aircraft vehicle.

9. The system of claim 1 , wherein the placement indicator corresponds to a physical controller and tracking the placement indicator includes translating a physical movement of the physical controller to a movement of the placement indicator.

10. The system of claim 1 , wherein the placement indicator corresponds to a body part of a user.

11. The system of claim 1 , wherein the placement indicator is associated with a virtual representation of the physical aircraft vehicle in the virtual three-dimensional model.

12. The system of claim 1 , wherein the simulated image of the viewfinder display window was generated based on one or more specified camera configuration parameters.

13. The system of claim 1 , wherein the viewfinder display window is visually coupled to the placement indicator.

14. The system of claim 1 , wherein each key frame of the plurality of key frames is visually indicated in the virtual three-dimensional model.

15. The system of claim 14 , wherein at least one key frame of the plurality of key frames is able to be modified by selecting and moving the corresponding visual indication of the key frame in the virtual three-dimensional model.

16. The system of claim 1 , wherein a planned traversal trajectory path is visually plotted in the virtual three-dimensional model.

17. The system of claim 1 , wherein the processor is further configured to provide in the virtual three-dimensional model, a preview window of a simulated video footage captured by the simulated camera of the simulated vehicle traversing a planned traversal path in the virtual three-dimensional model.

18. A method for planning a flight path of a physical aircraft vehicle, comprising:

using a processor to provide a virtual three-dimensional model of a physical environment;

tracking a placement indicator within the virtual three-dimensional model of the physical environment, wherein tracking the placement indicator includes tracking a location and an orientation of the placement indicator within the virtual three-dimensional model;

providing a computer rendered viewfinder display window that displays a simulated image captured from a simulated camera of a simulated vehicle located at the location of the placement indicator and oriented at a direction of the orientation of the placement indicator;

planning for the physical aircraft vehicle at least the flight path and a camera image capture using the placement indicator and the viewfinder display window within the virtual three-dimensional model, wherein the placement indicator has been utilized to specify based on the location and the orientation of the placement indicator at least one key frame of a plurality of key frames, a camera look-at line associated with a key frame included in the plurality of key frames is visually indicated in the virtual three-dimensional model, and the computer rendered viewfinder display window has been utilized to preview at least one key frame of the plurality of key frames;

calculating the flight path between physical locations corresponding the plurality of key frames specified in the virtual three-dimensional model; and

generating digital instructions that cause the physical aircraft vehicle to at least in part autonomously follow the calculated flight path.

19. The method of claim 18 , wherein the virtual three-dimensional model is provided via a virtual reality headset.

20. A computer program product for planning a flight path of a physical aircraft vehicle, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

providing a virtual three-dimensional model of a physical environment;

tracking a placement indicator within the virtual three-dimensional model of the physical environment, wherein tracking the placement indicator includes tracking a location and an orientation of the placement indicator within the virtual three-dimensional model;

providing a computer rendered viewfinder display window that displays a simulated image captured from a simulated camera of a simulated vehicle located at the location of the placement indicator and oriented at a direction of the orientation of the placement indicator;

planning for the physical aircraft vehicle at least the flight path and a camera image capture using the placement indicator and the viewfinder display window within the virtual three-dimensional model, wherein the placement indicator has been utilized to specify based on the location and the orientation of the placement indicator at least one key frame of a plurality of key frames, a camera look-at line associated with a key frame included in the plurality of key frames is visually indicated in the virtual three-dimensional model, and the computer rendered viewfinder display window has been utilized to preview at least one key frame of the plurality of key frames;

calculating the flight path between physical locations corresponding the plurality of key frames specified in the virtual three-dimensional model; and

generating digital instructions that cause the physical aircraft vehicle to at least in part autonomously follow the calculated flight path.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2020
From: AMBER GARAGE, INC.
To: FUZHOU LIGHTFLOW TECHNOLOGY CO., LTD
Reel/Frame 051726/0570 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2016
From: HU, BOTAO; ZHANG, JIAJIE
To: AMBER GARAGE, INC.
Reel/Frame 038984/0149 →
Continuity (2)
Provisional Application 62200128 · Aug 3, 2015
Related Publication 20170039764A1 · Feb 9, 2017