System and method for dynamic online backup optimization
Methods of optimizing transmission of data from a client to a remote data center are disclosed, as well as systems and computer program products related to the same. An exemplary method comprises: receiving data, at a first intervening data center, transmitted from a client and addressed to a terminal data center, wherein the first intervening data center and the terminal data center are selected from a plurality of connected data centers; and transferring the data, from the first intervening data center through one or more additional intervening data centers selected from the plurality of connected data centers, until the data reaches the terminal data center; wherein each data center in the plurality of data centers independently selects a connected data center to transfer the data to based upon the amount of latency and/or bandwidth available at the connected data center.
1. A method of transmitting data for backup from a client device to a remote terminal data center via one or more intervening data centers, the method comprising:
receiving, at an intervening data center, one or more data streams of the backup data;
recombining the received data streams into an archival copy of the backup data;
storing the archival copy of the backup data at the current intervening data center;
obtaining network parameters of the terminal data center and one or more other intervening data centers located between the current intervening data center and the terminal data center, the network parameters including at least one of bandwidth restrictions and network latencies;
calculating, based on the network parameters, a transmission time of the backup data via a direct route from the current intervening data center to the terminal data center;
calculating, based on the network parameters, transmission times of the backup data via one or more different indirect routes from the current intervening data center to the terminal data center via one or more of the other intervening data centers;
determining, from all calculated transmission times, the fastest direct or indirect route for transmitting the backup data from the current intervening data center to the terminal data center;
determining a bandwidth restriction that a next data center associated with the fastest direct or indirect route imposes for an individual data session on a network of the next data center, wherein the bandwidth restriction is less than an available bandwidth at the next data center;
re-splitting the backup data into a plurality of data streams based on the bandwidth restriction;
transmitting each of the plurality of data streams in a plurality of independent data sessions to next data center; and
repeating the above steps at the next data center if it is not the terminal data center.
2. The method of claim 1 , wherein the current intervening data center is a transparent proxy.
3. The method of claim 1 , wherein the terminal data center is: selected by a user; or provided by the client device.
4. The method of claim 1 , wherein the next data center is not a closest data center of the one or more different indirect routes based on geographic proximity to the current intervening data center.
5. The method of claim 1 , wherein the plurality of data streams is transmitted from the current intervening data center to the next data center in real-time.
6. The method of claim 1 , wherein intervening data centers are each configured to report at least one of bandwidth and latency levels to a proxy server in real-time or at a predetermined interval.
7. A system of transmitting data for backup from a client device to a remote terminal data center via one or more intervening data centers, the system comprising:
a network interface configured to receive and transmit data; and
a hardware processor communicatively coupled to the network interface, wherein the hardware processor is configured to:
receive, at an intervening data center, one or more data streams of the backup data;
recombine the received data streams into an archival copy of the backup data;
store the archival copy of the backup data at the current intervening data center;
obtain network parameters of the terminal data center and one or more other intervening data centers located between the current intervening data center and the terminal data center, the network parameters including at least one of bandwidth restrictions and network latencies;
calculate, based on the network parameters, a transmission time of the backup data via a direct route from the current intervening data center to the terminal data center;
calculate, based on the network parameters, transmission times of the backup data via one or more different indirect routes from the current intervening data center to the terminal data center via one or more of the other intervening data centers;
determine, from all calculated transmission times, the fastest direct or indirect route for transmitting the backup data from the current intervening data center to the terminal data center;
determine a bandwidth restriction that a next data center associated with the fastest direct or indirect route imposes for an individual data session on a network of the next data center, wherein the bandwidth restriction is less than an available bandwidth at the next data center;
re-split the backup data into a plurality of data streams based on the bandwidth restriction;
transmit each of the plurality of data streams in a plurality of independent data sessions to next data center; and
repeat the above steps at the next data center if it is not the terminal data center.
8. The system of claim 7 , wherein the current intervening data center is a transparent proxy, and
wherein the terminal data center is at least one of selected by a user and provided by the client device.
9. The system of claim 7 , wherein the next data center is not a closest data center of the one or more different indirect routes based on geographic proximity to the current intervening data center.
10. The system of claim 7 , wherein the plurality of data streams is transmitted from the current intervening data center to the next data center in real-time.
11. A non-transitory computer readable medium comprising computer executable instructions for transmitting data for backup from a client device to a remote terminal data center via one or more intervening data centers, including instructions for:
receiving, at an intervening data center, one or more data streams of the backup data;
recombining the received data streams into an archival copy of the backup data;
storing the archival copy of the backup data at the current intervening data center;
obtaining network parameters of the terminal data center and one or more other intervening data centers located between the current intervening data center and the terminal data center, the network parameters including at least one of bandwidth restrictions and network latencies;
calculating, based on the network parameters, a transmission time of the backup data via a direct route from the current intervening data center to the terminal data center;
calculating, based on the network parameters, transmission times of the backup data via one or more different indirect routes from the current intervening data center to the terminal data center via one or more of the other intervening data centers;
determining, from all calculated transmission times, the fastest direct or indirect route for transmitting the backup data from the current intervening data center to the terminal data center;
determining a bandwidth restriction that a next data center associated with the fastest direct or indirect route imposes for an individual data session on a network of the next data center, wherein the bandwidth restriction is less than an available bandwidth at the next data center;
re-splitting the backup data into a plurality of data streams based on the bandwidth restriction;
transmitting each of the plurality of data streams in a plurality of independent data sessions to next data center; and
repeating the above steps at the next data center if it is not the terminal data center.