IP Library Granted Patent US 8,301,318
Granted Patent B2
US 8,301,318 · App. 12/398,610 · Granted Oct 30, 2012

Robotic vehicle remote control system having a virtual operator environment

Assignee: Robotic Research LLC
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,301,318
App. No.
12/398,610
Granted
Oct 30, 2012
Kind
B2
Abstract

A control system for a remotely operated vehicle is disclosed. The control system includes a sensor tracking system configured to sense the remotely operated vehicle. The control system is coupled to the sensor tracking system and is configured to remotely control the remotely operated vehicle. The control system includes a synthetic view generator configured to construct a virtual model of the remotely operated vehicle and its surrounding environment based upon an input from the sensor tracking system. The control system also includes a graphical user interface configured to display a synthetic view of the virtual model. In addition, the control system includes a synthetic viewer control configured to manipulate an orientation of the synthetic view.

Claims (41)

1. A system for controlling a remotely operated vehicle, comprising:

a sensor tracking system configured to sense a remotely operated vehicle, the sensor tracking system being located at a first location and the remotely operated vehicle being located at a second location remote from the first location;

a control system at the first location coupled to the sensor tracking system, the control system configured to remotely control the remotely operated vehicle; the control system comprising:

a synthetic view generator configured to construct a virtual model of the remotely operated vehicle and its surrounding environment based upon an input from the sensor tracking system;

a graphical user interface configured to display a synthetic view of the virtual model;

a synthetic viewer control configured to manipulate an orientation of the synthetic view; and

an operator control unit configured to control the remotely operated vehicle.

2. The system of claim 1 , further comprising a prediction module configured to predict a future state of the remotely operated vehicle based upon information from the sensor tracking system.

3. The system of claim 1 , wherein the synthetic view is generated using a stored CAD drawing of the remotely operated vehicle.

4. The system of claim 1 , further comprising an operator aid module configured to generate a recommended course of action based upon information from the sensor tracking system.

5. The system of claim 1 , further comprising an operator aid module configured to generate warnings regarding a potential course of action based upon information from the sensor tracking system.

6. The system of claim 1 , wherein the sensor tracking system includes one or more of the following sensor systems: a LADAR, a camera, a GPS device, and an inertial guidance system.

7. The system of claim 1 , wherein the remotely operated vehicle is unmanned.

8. The system of claim 1 , wherein the remotely operated vehicle is manned.

9. The system of claim 8 , wherein an operator on the remotely operated vehicle has an option of exercising at least partial control over an operation of the remotely operated vehicle.

10. The system of claim 1 , wherein the virtual model is constructed from a surface mesh generated from information provided by the sensor tracking system.

11. A virtual control environment for remotely operating a robotic vehicle, comprising:

a sensor array that detects the robotic vehicle and its surrounding environment, the sensor array being located at a first location and the robotic vehicle being located at a second location remote from the first location;

a module configured to generate a computer model of the robotic vehicle and its surrounding environment based upon information collected by the sensor array;

a synthetic view generator module configured to create virtual views of the robotic vehicle and its surrounding environment based upon the computer model;

a graphical user interface configured to display a virtual view created by the synthetic view generator module; and

an operator control unit located at the first location and configured to control the robotic vehicle based upon the virtual view.

12. The virtual control environment of claim 11 , further comprising a prediction module configured to predict a future state of the remotely operated vehicle based upon information from the sensor array.

13. The virtual control environment of claim 11 , wherein the virtual view is generated using a stored CAD drawing of the remotely operated vehicle.

14. The virtual control environment of claim 11 , further comprising an operator aid module configured to generate a recommended course of action based upon information from the sensor array.

15. The virtual control environment of claim 11 , further comprising an operator aid module configured to generate warnings regarding a potential course of action based upon information from the sensor array.

16. The virtual control environment of claim 11 , wherein the sensor array includes one or more of the following sensor systems: a LADAR, a camera, a GPS device, and an inertial guidance system.

17. The virtual control environment of claim 11 , wherein the robotic vehicle is unmanned.

18. The virtual control environment of claim 11 , wherein the robotic vehicle is manned.

19. The virtual control environment of claim 18 , wherein an operator on the robotic vehicle has an option of exercising at least partial control over an operation of the robotic vehicle.

20. The virtual control environment of claim 11 , wherein the computer model is constructed from a surface mesh generated from information provided by the sensor array.

21. A method of controlling a remotely operated vehicle with a virtual images of the remotely operated vehicle, comprising:

remotely sensing the remotely operated vehicle and its surrounding environment with a sensor array, the sensor array being located at a first location and the remotely operated vehicle being located at a second location remote from the first location;

generating a virtual model of the remotely operated vehicle and its surrounding environment based upon information gathered by the sensor array;

generating a virtual view of the remotely operated vehicle and its surrounding environment;

displaying the virtual view on a graphical user interface; and

controlling, using a controller, the operation of the remotely operated vehicle based upon viewing the virtual view, the controller being located at the first location.

22. The method of claim 21 , further comprising predicting a future state of the remotely operated vehicle based upon information from the sensor array.

23. The method of claim 21 , further comprising displaying a live image of the remotely operated vehicle on the graphical user interface.

24. The method of claim 21 , further comprising generating a recommended course of action based upon information from the sensor array.

25. The method of claim 21 , further comprising generating warnings regarding a potential course of action based upon information from the sensor array.

Assignments (5)
SECURITY INTEREST Recorded Jul 14, 2025
From: ROBOTIC RESEARCH OPCO, LLC
To: CRESCENT COVE OPPORTUNITY LENDING, LLC
Reel/Frame 071945/0001 →
TERMINATION AND RELEASE OF PATENT SECURITY INTEREST RECORDED AT REEL 066382, FRAME 0141 Recorded Jul 16, 2024
From: CRESCENT COVE OPPORTUNITY LENDING, LLC, AS ADMINISTRATIVE AGENT
To: ROBOTIC RESEARCH OPCO, LLC
Reel/Frame 068389/0707 →
SECURITY INTEREST Recorded Jan 30, 2024
From: ROBOTIC RESEARCH OPCO, LLC
To: CRESCENT COVE OPPORTUNITY LENDING, LLC
Reel/Frame 066382/0141 →
MERGER Recorded Jun 20, 2022
From: ROBOTIC RESEARCH, LLC
To: ROBOTIC RESEARCH OPCO, LLC
Reel/Frame 060877/0929 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2009
From: LACAZE, ALBERTO D.; MURPHY, KARL N.; SMITH, KYLE R.; WILHELM, RAYMOND P., III
To: ROBOTIC RESEARCH LLC
Reel/Frame 022940/0106 →
Continuity (2)
Provisional Application 61064433 · Mar 5, 2008
Related Publication 20090276105A1 · Nov 5, 2009