Methods and systems for collaborative control of a remote vehicle
Systems and methods are for enabling a group of individuals, each using an individual computing device, to collaboratively control a vehicle in real-time as a unified intelligence. The collaboration system comprises a plurality of computing devices, each of the devices being used by an individual user, each of the computing devices enabling its user to contribute to the emerging real-time group-wise intent. A collaboration server is disclosed that moderates the closed-loop system, enabling convergence upon a unified group intent. Control commands are repeatedly sent to the vehicle based on the determined group intent.
1. A real-time collaborative vehicle control system comprising:
a plurality of computing devices each comprising a communications infrastructure coupled to each of a processor, a memory, and a collaborative intent application stored on each memory and configured to run on each processor, and a user interface coupled to a display and configured to receive input from a user;
a collaborative server in networked communication with the plurality of computing devices, the collaboration server including a processor and a memory;
a vehicle controller configured to receive commands from the collaborative server; and
a vehicle operatively controlled by the vehicle controller, wherein the control system is configured to perform the following steps, wherein computing device steps are performed substantially simultaneously on each of the plurality of computing devices:
display, on each of the computing devices, vehicle control information, and a collaboratively controlled graphical indication having a coordinate location in relation to the vehicle control information;
receive, on each of the computing devices, repeatedly in real-time, user input of a user intent vector through the user interface, the user intent vector having a direction in relation to the user control information and a magnitude,
send, repeatedly in real-time, the user intent vector to the collaboration server,
responsively determine, repeatedly in real-time by the collaboration server, a group intent vector from the plurality of received user intent vectors, and
determine, repeatedly in real-time by the collaboration server, an updated coordinate location based on the group intent vector;
determine, repeatedly in real-time by the collaboration server, at least one vehicle command in response to the updated coordinate location;
send, repeatedly in real-time by the collaboration server, the at least one a vehicle command to the vehicle controller, whereby the vehicle operates in response to the at least one vehicle command;
send, repeatedly in real-time by the collaboration server, the updated coordinate location to the plurality of computing devices; and
update, repeatedly in real-time by the computing devices, the location of the collaboratively controlled graphical indication using the received updated coordinate location, whereby a closed-loop system is formed between the collaboration server and each collaborative intent application.
2. The real-time collaborative vehicle control system of claim 1 , wherein the vehicle is a flying drone vehicle.
3. The real-time collaborative vehicle control system of claim 2 , wherein the at least one control axis includes at least one of a control axis indicating forward/backward motion of the vehicle and a control axis indicating leftward/rightward motion of the vehicle.
4. The real-time collaborative vehicle control system of claim 2 , wherein the collaboratively controlled graphical indication is influenced by a simulated spring force, said simulated spring force configured to pull the displayed collaboratively controlled graphical indication towards an origin position on the at least one control axis.
5. The real-time collaborative vehicle control system of claim 1 , wherein the vehicle control information includes at least one control axis, wherein a location of the graphical indication relative to the at least one control axis indicates the at least one command sent to the vehicle.
6. The real-time collaborative vehicle control system of claim 5 , wherein the first control axis and the second control axis intersect at an origin location.
7. The real-time collaborative vehicle control system of claim 5 , wherein the at least one control axis includes a first control axis oriented vertically and a second control axis oriented horizontally.
8. The real-time collaborative vehicle control system of claim 1 , the system further comprising a camera coupled to the vehicle and operatively controlled by the vehicle controller.
9. The real-time collaborative vehicle control system of claim 8 , wherein the camera is configured to stream live video to the plurality of computing devices.
10. The real-time collaborative vehicle control system of claim 8 , wherein each of the plurality of computing devices displays the live video on a same display as the collaboratively controlled graphical indication.
11. The real-time collaborative vehicle control system of claim 1 , wherein the at least one vehicle command includes at least one command for camera operation.
12. The real-time collaborative vehicle control system of claim 1 , wherein the at least one command for camera operation includes at least one of forward/backward tilt and leftward/rightward pan.
13. The real-time collaborative vehicle control system of claim 1 , wherein the system is configured to display a graphical indication of the user input.
14. The real-time collaborative vehicle control system of claim 13 , wherein the graphical indication is a U-shaped magnet icon.
15. The real-time collaborative vehicle control system of claim 14 , wherein the system is configured to change at least one of the size, orientation, and location of the magnet icon based on the user input.
16. The real-time collaborative vehicle control system of claim 1 , wherein the collaboratively controlled graphical indication is displayed on a control area corresponding to a plan area of operation of the vehicle.
17. The real-time collaborative vehicle control system of claim 16 , wherein the collaboratively controlled graphical indication represents a location of the vehicle in the control area.
18. The real-time collaborative vehicle control system of claim 16 , the control area further including a display of plan coordinate axes.
19. The real-time collaborative vehicle control system of claim 1 , wherein the collaboratively controlled graphical indication is an icon indicating the orientation of the vehicle.
20. The real-time collaborative vehicle control system of claim 1 , wherein the collaboratively controlled graphical indication has at least one of a simulated mass value and a simulated damping value, the at least one of the simulated mass value and the simulated damping value selected to slow a displayed collaboratively controlled motion of the collaboratively controlled graphical indication to better match dynamics of the vehicle.