High speed parallel data exchange with transfer recovery
View Patent ↗Systems and methods for transfer of data including establishing two separate connections, the two separate connections including a high speed connection and a high integrity connection. Blocks of data are exchanged over the high speed connection while the high integrity connection facilitates communication of descriptor data regarding data received over the high speed connection. As such, the data transfer speed of the high speed connection is utilized while communication via the high integrity connection allows for data reliability features not provided by the high speed connection.
1. A method of transferring data between computing devices operable to communicate over an electronic network, comprising:
establishing a high reliability connection between a source device and a destination device;
establishing a high speed connection between the source device and the destination device;
exchanging a descriptor between the source device and the destination device, the descriptor including information regarding blocks of data in a superblock of data to be transferred from the source device to the destination device, wherein the descriptor is exchanged via the high reliability connection and the blocks of data are transferred via the high speed connection, wherein the descriptor is at least two bits, wherein a first bit, of the descriptor, represents a first block of data of at least a portion of the superblock of data to be sent, and wherein a second bit, of the descriptor, represents a second block of data of at least the portion of the superblock of data to be sent;
maintaining recovery information regarding the data transfer, the recovery information including an indication of the blocks of data that have been received at the destination device;
wherein the recovery information is useable after an interruption of communication between the source device and the destination device to resume the data transfer such that blocks of data that have been received are retained at the destination device and the data transfer resumes such that only non-received blocks of data are transferred after the resumption of the data transfer.
2. The method according to claim 1 , wherein the interruption of communication occurs due to at least one of network unavailability, network unreliability, system failure, and system power failure.
3. The method according to claim 1 , wherein the interruption of communication occurs due to a predefined condition of the transfer.
4. The method according to claim 3 , wherein the predefined condition includes at least one of a predetermined number of cycles of sending a block of data without confirmation of the receipt of the block of data, a predetermined amount of time passing without receiving confirmation of the receipt of a block, and the bitrate of the transfer falling below a threshold bitrate.
5. The method of according to claim 1 , wherein resumption of the data transfer occurs based on at least one of a predetermined amount of time passing, a detection of a changed condition, and a resumption command received from a user.
6. The method according to claim 1 , wherein a file to be transferred is divided into said blocks of data, and said blocks of data are organized into superblocks.
7. The method according to claim 6 , wherein each superblock is fully processed prior to initiating a transfer of another superblock.
8. The method according to claim 7 , wherein the recovery information further includes data related to the identity of the superblock currently being transferred.
9. A method for receiving data at a destination device from a source device, comprising:
establishing at least one high integrity connection between the source device and the destination device;
establishing at least one high speed connection between the source device and the destination device;
receiving data at the destination device from the source device over at least one high speed connection;
acquiring a descriptor at the destination device regarding the data from the source device over at least one high integrity connection, wherein the descriptor is at least two bits, wherein a first bit of the descriptor represents a first block of data of at least a portion of a superblock of data to be sent, and wherein a second bit, of the descriptor, represents a second block of data of at least the portion of the superblock of data to be sent;
correlating, at the destination device, data received from the source device to the descriptor;
modifying the descriptor based on data received at the destination device over the high speed connection;
sending to the source device the modified descriptor;
receiving data at the destination device that has been resent by the source device using the at least one high speed connection;
maintaining recovery information regarding the transfer, the recovery information including an indication of the data that have been received at the destination device;
wherein the recovery information is useable after an interruption of communication between the source device and the destination device to resume the data transfer such that only non-received blocks are transferred after the recovery and any data received by the destination device prior to the interruption is retained by the source device.