Long distance trans-continental remote drone piloting system
A remote drone control system includes a pilot endpoint system comprising a pilot endpoint and a controller connected to the pilot endpoint. The remote drone control system includes a control endpoint system including a control endpoint, a signal adaptor connected to the control endpoint, and a transmitter connected to the signal adaptor. A drone is arranged to communicate with the transmitter to receive and send drone operating data to the control endpoint system. The drone is also arranged to communicate drone video data to the control endpoint system. A remote bridge including a server is arranged to connect the pilot endpoint and the control endpoint such that data is communicated amongst the pilot endpoint, control endpoint, and drone in real-time.
1 . A remote drone pilot system comprising:
a pilot endpoint system comprising a pilot endpoint and a controller connected to the pilot endpoint;
a control endpoint system comprising a control endpoint, a signal adaptor connected to the control endpoint and configured to convert between a drone communication protocol and an internet protocol, and a transmitter connected to the signal adaptor;
a remote bridge, including a server; and
a drone,
wherein:
the drone is configured to communicate with the transmitter of the control endpoint system to receive and send drone operating data to the control endpoint system and to communicate drone video data to the control endpoint system,
the pilot endpoint is configured to communicate with the control endpoint only through the remote bridge such that no direct communication path exists between the pilot endpoint and the control endpoint that bypasses the remote bridge,
the remote bridge is configured to pass all communications between the pilot endpoint and the control endpoint using the internet protocol, thereby allowing the drone operating data and drone video data to be communicated amongst the pilot endpoint, the control endpoint, and the drone in real-time, and
the remote bridge further includes an artificial intelligence system engine hosted on the server and arranged to conduct real-time analysis on the drone video data and the drone operating data.
2 . The remote drone pilot system of claim 1 , wherein the remote bridge includes a broker module arranged to authenticate a connection between the remote bridge and each of the control endpoint and a remote endpoint.
3 . The remote drone pilot system of claim 1 , wherein the controller includes a joystick, wherein each of the pilot endpoint and the control endpoint comprise a graphical user interface configured to display the drone video data and at least some of the drone operating data.
4 . The remote drone pilot system of claim 1 , wherein the drone includes a camera, wherein the drone video data from the drone is arranged to be communicated to a video device on the signal adaptor.
5 . The remote drone pilot system of claim 1 , wherein the pilot endpoint system further includes an audio device, wherein the audio device is connected to the pilot endpoint such that commands can be input into the pilot endpoint verbally via the audio device.
6 . The remote drone pilot system of claim 1 , wherein the pilot endpoint comprises a plurality of pilot endpoints, wherein a first of the plurality of pilot endpoints is arranged to provide commands to control the drone, wherein a second of the plurality of pilot endpoints is configured to only view the drone video data and the drone operating data, wherein the drone video data and the drone operating data is displayed identically on a graphical user interface display on each of the first and second of the plurality of pilot endpoints.
7 . The remote drone pilot system of claim 1 , wherein the signal adaptor is connected to the control endpoint by a first USB connection and the controller is connected to the pilot endpoint by a second USB connection.
8 . The remote drone pilot system of claim 1 , wherein the drone, the remote bridge, the control endpoint system, and the pilot endpoint system are arranged to operate with a connection latency of about or less than 150 ms.
9 . The remote drone pilot system of claim 1 , wherein the drone, the remote bridge, the control endpoint system, and the pilot endpoint system are arranged to operate with a connection latency of between about 100 ms and about 150 ms.
10 . A method of remotely piloting a drone, the method comprising:
(a) connecting a pilot endpoint located in first geographic area to a control endpoint located in a second geographic area through a remote bridge, wherein the pilot endpoint, the control endpoint, and the remote bridge connect using an internet protocol, and wherein no direct communication path exists between the pilot endpoint and the control endpoint that bypasses the remote bridge;
(b) communicating a command from the pilot endpoint to the control endpoint through the remote bridge;
(c) communicating the command from the control endpoint through a signal adaptor to a transmitter;
(d) communicating the command from the transmitter to the drone such that the drone is operated based on the command;
(e) communicating drone operating data from the drone to the signal adaptor through the transmitter;
(f) converting the drone operating data into a form readable by the control endpoint;
(g) communicating drone video data from the drone to the signal adaptor;
(h) communicating the converted drone operating data and the drone video data to the control endpoint, and then to the pilot endpoint through the remote bridge;
(i) displaying the drone video data and the converted drone operating data on the pilot endpoint;
(j) providing the drone video data to an artificial intelligence system engine of a server of the remote bridge;
(k) the artificial intelligence system engine conducting a real-time analysis on the drone video data to identify at least one of a number of objects viewed, a frequency of objects viewed, or a particular targeted object; and
(l) communicating the real-time analysis to at least one of the pilot endpoint or the control endpoint,
wherein steps (b)-(i) occur with a connection latency of about or less than 150 ms.
11 . The method of claim 10 , wherein connecting the pilot endpoint to the control endpoint through the remote bridge includes authenticating the pilot endpoint and the control endpoint using a broker module of the server of the remote bridge.
12 . The method of claim 10 , wherein the remote bridge is a first remote bridge, and wherein the server includes a search module, and further comprising:
determining using the search module a desirable second remote bridge from a plurality of second remote bridges; and
transferring the control endpoint and the pilot endpoint to be connected through the desirable second remote bridge.
13 . The method of claim 12 , wherein the search module determines the desirable second remote bridge based on one or more of shortest latency between the control endpoint and the pilot endpoint, shortest physical distance between the control endpoint and the pilot endpoint, or geographical proximity to the control endpoint or the pilot endpoint.
14 . The method of claim 10 , wherein conducting the real-time analysis further includes classifying the at least one of the number of objects viewed, the frequency of objects viewed, or the particular targeted object to detect a component, sub-assembly, or system on the drone video data and create a report of a classification.
15 . The method of claim 10 , further comprising transmitting the command verbally to the pilot endpoint using an audio device including a microphone.
16 . The method of claim 10 , wherein the pilot endpoint is a first pilot endpoint, and further comprising connecting a second pilot endpoint to the control endpoint, wherein the first pilot endpoint communicates commands to pilot the drone and the second pilot endpoint communicates commands to control operation of a camera on the drone providing the drone video data.
17 . The method of claim 10 , further comprising:
multiplexing the drone video data communicated from the control endpoint using the remote bridge; and
displaying the multiplexed drone video data on a graphical user interface of each of a plurality of additional pilot endpoints.
18 . The method of claim 10 , wherein the drone is an unmanned aerial vehicle.