Orienting a user interface of a controller for operating a self-propelled device
A self-propelled device determines an orientation for its movement based on a pre-determined reference frame. A controller device is operable by a user to control the self-propelled device. The controller device includes a user interface for controlling at least a direction of movement of the self-propelled device. The self-propelled device is configured to signal the controller device information that indicates the orientation of the self-propelled device. The controller device is configured to orient the user interface, based on the information signaled from the self-propelled device, to reflect the orientation of the self-propelled device.
1. A controller device for operating a self-propelled device, the controller device comprising:
a touch-sensitive display;
one or more processors; and
one or more memory resources storing instructions that, when executed by the one or more processors, cause the one or more processors to:
generate a user interface on the touch-sensitive display, the user interface comprising a virtual controls to enable a user to remotely operate the self-propelled device;
using a camera, detect the self-propelled device in image data captured by the camera based on a location of the self-propelled device in a real-world environment, generate a virtual representation of the self-propelled device in a virtual environment on the user interface;
receive one or more user interactions with the virtual controls, the one or more user interactions to maneuver the self-propelled device; and
transmit one or more commands to the self-propelled device, the one or more commands to maneuver the self-propelled device in accordance with the one or more user interactions;
wherein the executed instructions cause the one or more processors to generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment, and wherein the executed instructions cause the one or more processors to generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment in accordance with execution of a gaming application that provides a gaming environment.
2. The controller device of claim 1 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
directionally calibrate the user interface based on an orientation of the self-propelled device in relation to an initial frame of reference.
3. The controller device of claim 1 , wherein the executed instructions cause the one or more processors to detect the self-propelled device in the image data as a fiducial marker for the virtual representation of the self-propelled device.
4. The controller device of claim 1 , wherein the self-propelled device comprises a remotely operated aircraft.
5. The controller device of claim 1 , wherein the executed instructions cause the one or more processors to detect the self-propelled device in the image data based on a pattern of light emitted by the self-propelled device.
6. The controller device of claim 1 , wherein the gaming application corresponds to one of a plurality of gaming applications, executable by the controller device, and each providing a unique gaming environment.
7. A computer-implemented method for operating a self-propelled device, the method performed by one or more processors of a controller device and comprising:
generating a virtual user interface on a touch-sensitive display of the controller device, the user interface comprising virtual controls enabling a user to remotely operate the self-propelled device;
using a camera, detecting the self-propelled device in image data captured by the camera;
based on a location of the self-propelled device in a real-world environment, generating a virtual representation of the self-propelled device in a virtual environment on the user interface;
receiving one or more user interactions with the virtual controls, the one or more user interactions to maneuver the self-propelled device; and
transmitting one or more commands to the self-propelled device, the one or more commands to maneuver the self-propelled device in accordance with the one or more user interactions;
wherein the one or more processors generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment, and wherein the one or more processors generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment in accordance with execution of a gaming application that provides a gaming environment.
8. The method of claim 7 , further comprising:
directionally calibrating the user interface based on an orientation of the self-propelled device in relation to an initial frame of reference.
9. The method of claim 7 , wherein the one or more processors detect the self-propelled device in the image data as a fiducial marker for the virtual representation of the self-propelled device.
10. The method of claim 7 , wherein the self-propelled device comprises a remotely operated aircraft.
11. The method of claim 7 , wherein the one or more processors detect the self-propelled device in the image data based on a pattern of light emitted by the self-propelled device.
12. The method of claim 7 , wherein the gaming application corresponds to one of a plurality of gaming applications, executable by the controller device, and each providing a unique gaming environment.
13. A non-transitory computer readable medium storing instructions for operating a self-propelled device, wherein the instructions, when executed by one or more processors of a controller device, cause the one or more processors to:
generate a user interface on a touch-sensitive display of the controller device, the user interface comprising virtual controls enabling a user to remotely operate the self-propelled device;
using a camera, detect the self-propelled device in image data captured by the camera;
based on a location of the self-propelled device in a real-world environment, generate a virtual representation of the self-propelled device in a virtual environment on the user interface;
one or more user interactions with the virtual controls, the one or more user interactions to maneuver the self-propelled device; and
transmit one or more commands to the self-propelled device, the one or more commands to maneuver the self-propelled device in accordance with the one or more user interactions;
wherein the executed instructions cause the one or more processors to generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment, and wherein the executed instructions cause the one or more processors to generate the virtual representation of the self-propelled device in the virtual environment to reflect user control of the self-propelled device in the real-world environment in accordance with execution of a gaming application that provides a gaming environment.
14. The non-transitory computer readable medium of claim 13 , wherein the instructions when executed by the one or more processors further cause the one or more processors to:
directionally calibrate the user interface based on an orientation of the self-propelled device in relation to an initial frame of reference.
15. The non-transitory computer readable medium of claim 13 , wherein the executed instructions cause the one or more processors to detect the self-propelled device in the image data as a fiducial marker for the virtual representation of the self-propelled device.
16. The non-transitory computer readable medium of claim 15 , wherein the self-propelled device comprises a remotely operated aircraft.
17. The non-transitory computer readable medium of claim 13 , wherein the executed instructions cause the one or more processors to detect the self-propelled device in the image data based on a pattern of light emitted by the self-propelled device.