Augmented reality-based system and method providing status and control of unmanned vehicles
View Patent ↗An augmented reality system identifies and controls a vehicle located within an environment. A tracking system obtains viewpoint information corresponding to a real-time view of the environment. A processing system receives information from one or more sensors. Information includes sensor location information and status information about the vehicle. Processing system generates graphics using said sensor location information and said viewpoint information. Graphics include visual representations of said status information and controls. A display displays the generated graphics such that the graphics are superimposed on the real-time view. The graphics appear attached to the vehicle. An interaction device activates a displayed control.
1. A method of identifying and controlling a vehicle located within an environment comprising the steps of:
receiving information from one or more sensors coupled to the vehicle, where the information includes sensor location information and status information about the vehicle;
obtaining viewpoint information corresponding to a real-time view of said environment;
generating graphics using said sensor location information and viewpoint information, wherein the graphics include visual representations of the status information and controls;
displaying the generated graphics on a display such that the graphics are superimposed on the real-time view, wherein the graphics appear attached to the vehicle; and
activating a displayed control.
2. The method of claim 1 , wherein visual representations of status information include:
an identifier that identifies the vehicle.
3. The method of claim 1 , wherein the displayed control includes:
an action button that when activated displays information captured from a number of said one or more sensors coupled to the vehicle.
4. The method of claim 1 , wherein the displayed control includes:
arrows that upon being selected move the vehicle in the direction selected.
5. The method of claim 1 , wherein the displayed control includes:
an icon next to a vehicle that upon being moved to a location moves the vehicle to a corresponding location.
6. The method of claim 1 , wherein the displayed control includes:
an action button next to a vehicle that upon being activated commands the vehicle to move to a predetermined location.
7. The method of claim 1 , wherein the displayed control includes:
an action button next to a vehicle that upon being activated commands the vehicle to perform a series of operations.
8. An augmented reality system for identifying and controlling a vehicle located within an environment comprising:
one or more sensors coupled to the vehicle;
a tracking system that obtains viewpoint information corresponding to a real-time view of said environment;
a processing system that receives information from said one or more sensors, where the information includes sensor location information and status information about the vehicle, and generates graphics using said sensor location information and said viewpoint information, wherein the graphics include visual representations of said status information and controls; a display that displays the generated graphics such that the graphics are superimposed on the real-time view, wherein the graphics appear attached to the vehicle; and
an interaction device that activates a displayed control.
9. The augmented reality system of claim 8 , wherein visual representations of status information include:
an identifier that identifies the vehicle.
10. The augmented reality system of 8 , wherein the displayed control includes:
an action button that when activated displays information captured from a number of said one or more sensors coupled to the vehicle.
11. The augmented reality system of claim 8 , wherein the displayed control includes:
arrows that upon being selected move the vehicle in the direction selected.
12. The augmented reality system of claim 8 , wherein the displayed control includes:
an icon next to a vehicle that upon being moved to a location moves the vehicle to a corresponding location.
13. The augmented reality system of claim 8 , wherein the displayed control includes:
an action button next to a vehicle that upon being activated commands the vehicle to move to a predetermined location.
14. The augmented reality system of claim 8 , wherein the displayed control includes:
an action button next to a vehicle that upon being activated commands the vehicle to perform a series of operations.
15. An augmented reality system for identifying and controlling an apparatus located within an environment comprising:
one or more sensors coupled to the apparatus;
a tracking system that obtains viewpoint information corresponding to a real-time view of said environment;
a processing system that receives information from said one or more sensors, where the information includes sensor location information and status information about the apparatus, and generates graphics using said sensor location information and said viewpoint information, wherein the graphics include visual representations of said status information and controls;
a display that displays the generated graphics such that the graphics are superimposed on the real-time view, wherein the graphics appear attached to the apparatus; and
an interaction device that activates a displayed control.
16. The augmented reality system of claim 15 , wherein the apparatus is a robotic device and the interaction device controls the robotic device.