Multipath video streaming over a wireless network
View Patent ↗The method and system as disclosed relates to streaming of large quantities of time critical data over multiple distinct channels from a wireless communications device to a central receiver. More specifically the disclosure deals with the challenges and problems of maintaining consistent data reception quality when faced with the anomalies of a moving sender that is sending data using a relatively unstable radio frequency (RF) method. This is achieved by converting single source data into multiple data streams, placing them in transport buffers and storing them for forwarding. A plurality of radio frequency modules provide wireless connectivity to a plurality of wireless network. Links are maintained to provide feedback on network connections to allow for the transfer of data from one network to another and to adjust the amount of data being transmitted.
1. A system comprising:
a buffer management and transport controller configured to divide encoded data streams into data portions for buffering into multiple transport buffers;
a plurality of radio frequency modules configured to wirelessly transmit the data portions simultaneously to a plurality of different wireless networks to aggregate throughput across the plurality of different wireless networks, wherein the plurality of radio frequency modules are further configured to provide network feedback to the buffer management and transport controller regarding network conditions for the plurality of different wireless networks;
a data transmission control and reassembly configured to receive the data portions from the plurality of different wireless networks for reassembly and to provide receiver feedback to the buffer management and transport controller regarding additional network conditions for the plurality of different wireless networks;
wherein the buffer management and transport controller is further configured to use the network feedback and the receiver feedback to, for each data portion and prior to transmission of the data portion over at least one of the plurality of different wireless networks, withdraw the data portion from a transport buffer and dynamically select a radio frequency module of the plurality of radio frequency modules for the transmission of the data portion over the at least one of the plurality of different wireless networks;
wherein the buffer management and transport controller is configured to dynamically select a first wireless network for transmission of a first data portion, and a second different wireless network for transmission of a second data portion such that different data portions are transmitted to different wireless networks; and
wherein the buffer management and transport controller is configured to match a buffering rate for the data portions with a withdraw rate for the data portions using the network feedback and the receiver feedback.
2. The system of claim 1 , wherein the network feedback and the receiver feedback comprises an error rate for each radio frequency module based on data portions that are not delivered at the destination for re-assembly; and a capacity condition for each radio frequency module based on at least one of a latency for transmission of the data portions and data portions that are delivered at the destination for re-assembly.
3. The system of claim 1 , wherein:
the data portions withdrawn from the buffer management and transport controller are assembled packets of datagrams having respective sequence numbers that are delivered to the dynamically selected radio frequency module; and
the delivered assembled packets from each radio frequency module are re-assembled at the destination using the sequence numbers in the corresponding datagrams.
4. The system of claim 1 , further comprising a data source configured to:
capture audio-visual data for the encoded data streams; and
use the network feedback and the receiver feedback to control at least one of the quantity and the quality of the captured audio-visual data.
5. The system of claim 1 , wherein the buffer management and transport controller is controlled by a second buffer management and transport controller that is configured to control:
the withdrawal by the buffer management and transport controller of the data portions;
the dynamic selection of the radio frequency modules for the transmission of the data portions; and
the transmission by selected radio frequency modules to simultaneously available different wireless networks.
6. The system of claim 1 , further comprising an encoder for encoding data streams, wherein the encoded data streams comprise redundant information to be delivered for increased reliability as the data portions from each radio frequency module for the wireless radio transmission to the plurality of different wireless networks for re-assembly at a destination.
7. The system of claim 1 , wherein the network feedback and the receiver feedback is selected from a group consisting of:
latency for transmission of data portions;
transmit collisions at the plurality of different wireless networks;
the number of the radio frequency modules in the wireless radio transmission with each of the plurality of different wireless networks;
throughput requirements of the plurality of radio frequency modules for the wireless radio transmission on each of the plurality of different wireless networks;
coverage drop-off indicators;
time between successful transmission attempts;
status values;
Received Signal Strength Indicator (RSSI);
packet retransmissions;
average packet loss per link;
sender bandwidth;
destination bandwidth;
current available bandwidth; and
combinations thereof.
8. The system of claim 1 , wherein:
the withdrawal of encoded data portions from the buffer management and transport controller are assembled packets of datagrams having respective sequence numbers that are delivered to the selected radio frequency module;
the delivered data portions from each radio frequency module are re-assembled at the destination using the sequence numbers in the corresponding datagrams; and
the quantity at which the delivered data portions from each radio frequency modules are being re-assembled at the destination includes:
a rate at which the delivered data portions from each radio frequency modules are being re-assembled at the destination; and
an amount at which the delivered data portions from each radio frequency modules are being re-assembled at the destination.
9. The system of claim 1 , wherein when the buffer management and transport controller is intermittently stationary and in motion relative to the plurality of different wireless networks, to use the network feedback and the receiver feedback, to control:
original data streams that are encoded into the buffer management and transport controller;
the withdrawal by the buffer management and transport controller of the data portions;
the dynamic selection of the radio frequency modules for the transmission of the data portions; and
the transmission by selected radio frequency modules to simultaneously available different wireless networks.
10. The system of claim 9 , wherein the network feedback and the receiver feedback is selected from a group consisting of:
latency for transmission of data portions;
transmit collisions at the plurality of different wireless networks;
the number of the radio frequency modules in the wireless radio transmission with each of the plurality of different wireless networks;
throughput requirements of the plurality of radio frequency modules for the wireless radio transmission on each of the plurality of different wireless networks;
coverage drop-off indicators;
time between successful transmission attempts;
status values;
Received Signal Strength Indicator (RSSI);
packet retransmissions;
average packet loss per link;
sender bandwidth;
destination bandwidth;
current available bandwidth; and
combinations thereof.
11. The system of claim 1 , wherein encoded data streams are generated from a continuous real-time stream of audio-visual data by a wireless device.
12. The system of claim 1 further comprising:
a data source configured to capture a continuous real-time stream of audio-visual data;
a converter configured to convert the captured audio-visual data into original data streams; and
an encoder configured to encode the original data streams into the encoded data streams.
13. The system of claim 12 wherein the buffer management and transport controller is configured to control the encoder based on the network feedback and the receiver feedback.
14. The system of claim 1 , wherein two or more the radio frequency modules are further configured to enable said wireless radio transmission of the data portions to one of the plurality of different wireless networks.
15. The system of claim 1 , wherein the buffer management and transport controller is further configured, using the network feedback and the receiver feedback, to control delivery rate of data portions to selected radio frequency modules.
16. The system of claim 1 , wherein the buffer management and transport controller is further configured, using the network feedback and the receiver feedback, to dynamically and selectively redirect the data portions from radio frequency module to another radio frequency module, and from one the different wireless networks to another of the different wireless networks, whereby data throughput to the destination is maintained.
17. The system of claim 1 , wherein each of the radio frequency modules is further configured to monitor and control the wireless radio transmission of the data portions, whereby data throughput to the destination is maintained.