Opportunistic content delivery using delta coding
Systems and methods are described for avoiding redundant data transfers using delta coding techniques when reliably and opportunistically communicating data to multiple user systems. According to embodiments, user systems track received block sequences for locally stored content blocks. An intermediate server intercepts content requests between user systems and target hosts, and deterministically chucks and fingerprints content data received in response to those requests. A fingerprint of a received content block is communicated to the requesting user system, and the user system determines based on the fingerprint whether the corresponding content block matches a content block that is already locally stored. If so, the user system returns a set of fingerprints representing a sequence of next content blocks that were previously stored after the matching content block. The intermediate server can then send only those content data blocks that are not already locally stored at the user system according to the returned set of fingerprints.
1. A method for communication of a content object between a proxy server and a client device, the method comprising:
receiving at the client device a fingerprint of an Nth data block from the proxy server, the fingerprint of the Nth data block having been generated by the proxy server after receiving at the proxy server from a content server a content object comprising the N th data block and M additional data blocks, where M is greater than one;
determining the Nth data block is stored at the client device;
generating fingerprints of X additional data blocks also stored at the client device;
sending to the proxy server from the client device an indication that the N th data block is stored at the client device and the fingerprints of the X additional data blocks that are also stored at the client device; and
receiving at least one of the M additional data blocks at the client device, the at least one of the M additional data blocks having been determined by the proxy server to not correspond to the fingerprints of the X additional data blocks stored at the client device.
2. The method of claim 1 , wherein the M additional data blocks are sequential data blocks that are received at the proxy server after the Nth data block.
3. The method of claim 2 , wherein the X additional data blocks stored at the client device follow in sequence the Nth data block in the content object.
4. The method of claim 1 , further comprising
requesting the content object at the client device, wherein the content object is received from the content server in response to the request from the client device for the content object.
5. The method of claim 1 , wherein the content object received from the content server is received as a data stream, and the proxy server chunks the data stream into the N th data block and the M additional data blocks.
6. The method of claim 1 , wherein the client device is one of a plurality of client devices that share a common communications channel and that receive the fingerprint of the Nth data block that is multicast from the proxy server.
7. The method of claim 1 , wherein:
the client device comprises a user terminal and a client optimizer, and
the client optimizer is disposed between the user terminal and the proxy server.
8. A client-side system comprising:
an object processing subsystem operable to:
receive a fingerprint of an N th data block from a server-side system across a communications network, the fingerprint of the Nth data block having been generated by the server-side system after receiving from a content server a content object comprising the N th data block and M additional data blocks, where M is greater than one;
determine the Nth data block is stored locally at the client-side system by comparing the fingerprint of the Nth data block to client fingerprints of locally stored content blocks,
generate fingerprints of X additional data blocks also stored locally at the client-side system; and
generate a response package comprising the fingerprints of the X additional data blocks; and
a communications subsystem, in communication with the object processing subsystem, and operable to communicate the response package from the client-side system to the server-side system in response to receiving the fingerprint of the Nth data block.
9. A tangible, non-transitory medium storing computer-readable instructions, which, when executed, cause a processor to perform steps comprising:
receiving at a client device a fingerprint of an Nth data block from a proxy server, the fingerprint of the Nth data block having been generated by the proxy server after receiving at the proxy server from a content server a content object comprising the N th data block and M additional data blocks, where M is greater than one;
determining the Nth data block is stored at the client device;
generating fingerprints of X additional data blocks also stored at the client device;
sending to the proxy server from the client device an indication that the N th data block is stored at the client device and the fingerprints of the X additional data blocks that are also stored at the client device; and
receiving at least one of the M additional data blocks at the client device, the at least one of the M additional data blocks having been determined by the proxy server to not correspond to the fingerprints of the X additional data blocks stored at the client device.
10. The medium of claim 9 , wherein the M additional data blocks are sequential data blocks that are received at the proxy server after the Nth data block.
11. The medium of claim 10 , wherein the X additional data blocks stored at the client device follow in sequence the Nth data block in the content object.
12. The medium of claim 9 , wherein the steps further comprise:
requesting the content object at the client device, wherein the content object is received from the content server in response to the request from the client device for the content object.
13. The medium of claim 9 , wherein the content object received from the content server is received as a data stream, and the proxy server chunks the data stream into the N th data block and the M additional data blocks.
14. The medium of claim 9 , wherein the client device is one of a plurality of client devices that share a common communications channel and that receive the fingerprint of the Nth data block that is multicast from the proxy server.
15. The medium of claim 9 , wherein:
the client device comprises a user terminal and a client optimizer; and
the client optimizer is disposed between the user terminal and the proxy server.