Low latency wireless communication system for teleoperated vehicle environments
A vehicle remote support system includes a communication system that operates over a plurality of parallel wireless network connections to provide low-latency video from vehicle to a remote support server that provides remote support to the vehicle dependent on real-time video. The vehicle includes a source that encodes multiple versions of the original video segments (e.g., one per wireless network connection) and transmits the multiple versions of the segments to a sink at the remote support server over the respective wireless connections. This redundant multi-path communication system rationally allocates network resources to the managed video streams and balances bandwidth against latency in order to avoid network congestion and safety issues associated with single-path transmissions. In other embodiments, a similar communication system that transmits video or other real-time messages between a source and a sink may be utilized in cloud robotics applications.
1. A method for wirelessly receiving low latency video at a remote support server supporting a teleoperated vehicle during a remote support session, the method comprising:
establishing the remote support session with the vehicle;
receiving at a video processor, a first encoded video segment and a first segment identifier for the first encoded video segment from a first wireless network connection;
determining that the first segment identifier is absent from a segment identifier buffer;
responsive to determining that the first segment identifier is absent from the segment identifier buffer, storing the first segment identifier to the segment identifier buffer and providing the first encoded video segment to a video device;
subsequent to receiving the first encoded video segment, receiving at the video processor from a second wireless network connection different than the first wireless network connection, a second encoded video segment that represents redundant underlying video content as the first encoded video segment, and receiving a second segment identifier for the second encoded video segment;
determining that the second segment identifier matches the first segment identifier previously stored to the segment identifier buffer;
responsive to determining that the second segment identifier matches the first segment identifier in the segment identifier buffer, discarding the second encoded video segment.
2. The method of claim 1 , wherein the first wireless network connection comprises a first cellular network connection to a first cellular network operated by a first wireless carrier, and wherein the second wireless network connection comprises a second cellular network connection to a second cellular network operated by a second wireless carrier different than the first wireless carrier.
3. The method of claim 1 , wherein the first wireless network connection comprises a first WiFi network connection and the second wireless network connection comprises a second WiFi network connection.
4. The method of claim 1 , wherein the first wireless network connection comprises a cellular network connection to a cellular network and the second wireless network comprises a WiFi network connection to a WiFi network.
5. The method of claim 1 , wherein the first wireless network connection operates according to a first communication protocol and wherein the second wireless network connection operates according to a second communication protocol different than the first communication protocol.
6. The method of claim 1 , wherein the first wireless network connection is facilitated by first networking infrastructure and wherein the second wireless network connection is facilitated by second networking infrastructure different than the first networking infrastructure.
7. The method of claim 1 , wherein communications over the first wireless network connection comprise different performance parameters than communications over the second wireless network connection.
8. The method of claim 1 , wherein the second encoded video segment is encoded according to a different encoding format or encoding parameters than the first encoded video segment.
9. The method of claim 1 , further comprising:
receiving at a message processor, a first message and a message identifier for the first message from the first wireless network connection;
determining that the message identifier is absent from a message identifier buffer;
responsive to determining that the message identifier is absent from the message identifier buffer, storing the message identifier to the message identifier buffer and providing the first message to a remote support device;
receiving at the message processor, a second message and the message identifier from the second wireless network connection, the second message including redundant data to the first message;
determining that the message identifier is present in the message identifier buffer;
responsive to determining that the message identifier is present in the message identifier buffer, discarding the second message.
10. The method of claim 9 , wherein the first and second message comprise telemetry data associated with sensor readings from the vehicle.
11. A non-transitory computer-readable storage medium storing instructions for wirelessly receiving low latency video at a remote support server supporting a teleoperated vehicle during a remote support session, the instructions when executed by one or more processors causing the one or more processors to perform steps comprising:
establishing the remote support session with the vehicle;
receiving at a video processor, a first encoded video segment and a first segment identifier for the first encoded video segment from a first wireless network connection;
determining that the first segment identifier is absent from a segment identifier buffer;
responsive to determining that the first segment identifier is absent from the segment identifier buffer, storing the first segment identifier to the segment identifier buffer and providing the first encoded video segment to a video device;
subsequent to receiving the first encoded video segment, receiving at the video processor from a second wireless network connection different than the first wireless network connection, a second encoded video segment that represents redundant underlying video content as the first encoded video segment, and receiving a second segment identifier for the second encoded video segment;
determining that the second segment identifier matches the first segment identifier previously stored to the segment identifier buffer;
responsive to determining that the second segment identifier matches the first segment identifier in the segment identifier buffer, discarding the second encoded video segment.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the first wireless network connection comprises a first cellular network connection to a first cellular network operated by a first wireless carrier, and wherein the second wireless network connection comprises a second cellular network connection to a second cellular network operated by a second wireless carrier different than the first wireless carrier.
13. The non-transitory computer-readable storage medium of claim 11 , wherein the first wireless network connection comprises a first WiFi network connection and the second wireless network connection comprises a second WiFi network connection.
14. The non-transitory computer-readable storage medium of claim 11 , wherein the first wireless network connection comprises a cellular network connection to a cellular network and the second wireless network connection comprises a WiFi network connection to a WiFi network.
15. The non-transitory computer-readable storage medium of claim 11 , wherein the first wireless network connection operates according to a first communication protocol and wherein the second wireless network connection operates according to a second communication protocol different than the first communication protocol.
16. The non-transitory computer-readable storage medium of claim 11 , wherein the first wireless network is facilitated by first networking infrastructure and wherein the second wireless network connection is facilitated by second networking infrastructure different than the first networking infrastructure.
17. The non-transitory computer-readable storage medium of claim 11 , wherein communications over the first wireless network connection comprise different performance parameters than communications over the second wireless network connection.
18. The non-transitory computer-readable storage medium of claim 11 , wherein the second encoded video segment is encoded according to a different encoding format or encoding parameters than the first encoded video segment.
19. The non-transitory computer-readable storage medium of claim 11 , the instructions when executed further causing the one or more processors to perform steps comprising:
receiving at a message processor, a first message and a message identifier for the first message from the first wireless network connection;
determining that the message identifier is absent from a message identifier buffer;
responsive to determining that the message identifier is absent from the message identifier buffer, storing the message identifier to the message identifier buffer and providing the first message to a remote support device;
receiving at the message processor, a second message and the message identifier from the second wireless network connection, the second message including redundant data to the first message;
determining that the message identifier is present in the message identifier buffer;
responsive to determining that the message identifier is present in the message identifier buffer, discarding the second message.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the first and second message comprise telemetry data associated with sensor readings from the vehicle.