Data fingerprint strengthening
View Patent ↗Techniques and mechanisms described herein facilitate the strengthening of a data fingerprint. According to various embodiments, a data stream may be parsed into a plurality of data chunks. A hash function may be applied to each of the data chunks to determine a respective data chunk hash value. A respective data chunk fingerprint for each of the plurality of data chunks may be transmitted to a remote server. Each data chunk fingerprint may include the respective data chunk hash value and one or more respective intermediate hash values.
1. A computing device comprising:
memory configured to store a data stream, the data stream being parsed into a plurality of data chunks;
a processor configured to:
determine a first data chunk hash value for a first portion of each of the plurality of data chunks by applying a hash function to the first portion of each of the plurality of data chunks;
generate a first data chunk fingerprint based on the first data chunk hash value and one or more respective intermediate hash values by applying a mathematical function to the first data chunk hash value and the one or more respective intermediate hash values;
determine a second data chunk hash value for a successive portion of each of the plurality of data chunks by applying the hash function to the successive portion of each of the plurality of data chunk; and
generate a new respective intermediate hash value by applying the mathematical function to the second data chunk hash value and a previous intermediate hash value, the new respective intermediate hash value dependent upon the first portion and the second portion;
and
a communications interface configured to transmit the respective data chunk fingerprint to a remote server for each of the plurality of data chunks, each data chunk fingerprint including the respective data chunk hash value and the new respective intermediate hash values.
2. The computing device recited in claim 1 , wherein the one or more respective intermediate hash values are generated during the parsing of the data stream or during the determining of the data chunk hash values.
3. The computing device recited in claim 1 , wherein a first data chunk checksum value is generated during the parsing of the data stream.
4. The computing device recited in claim 3 , wherein the one or more respective intermediate hash values includes the first data chunk checksum value.
5. The computing device recited in claim 1 , wherein the hash function is an additive hash function which yields the data chunk hash value by summing a plurality of results produced by applying the mathematical function to a respective data chunk and a plurality of intermediate results.
6. The computing device recited in claim 5 , wherein the one or more respective intermediate hash values includes one or more of the intermediate results.
7. The computing device recited in claim 1 , wherein the first data chunk fingerprint is determined by concatenating the respective data chunk hash value and the new respective intermediate hash value.
8. The computing device recited in claim 1 , wherein the communications interface is further configured to receive a data chunk status message, the data chunk status message indicating whether a data chunk associated with one of the data chunk fingerprints is stored at the remote server.
9. The computing device recited in claim 8 , wherein the communications interface is further configured to transmit the data chunk to the remote server for storage when it is determined that the data chunk is not stored at the remote server.
10. A method, comprising:
parsing a data stream into a plurality of data chunks, the data stream being stored in memory;
generating one or more respective intermediate hash values during the parsing of the data stream;
via a processor, determining a first data chunk hash value for a first portion of each of the plurality of data chunks by applying a hash function to the first portion of each of the plurality of data chunks; generating a first data chunk fingerprint based on the first data chunk hash value and the one or more respective intermediate hash values by applying a mathematical function to the first data chunk hash value and the one or more respective intermediate hash values; determining a second data chunk hash value for a successive portion of each of the plurality of data chunks by applying the hash function to the successive portion of each of the plurality of data chunk; and generating a new respective intermediate hash value by applying the mathematical function to the second data chunk hash value and a previous intermediate hash value, the new respective intermediate hash value dependent upon the first portion and the second portion; and
via a communications interface, transmitting the respective data chunk fingerprint to a remote server for each of the plurality of data chunks, each data chunk fingerprint including the respective data chunk hash value and the new respective intermediate hash values.
11. The method recited in claim 10 , wherein the one or more respective intermediate hash values are generated during the parsing of the data stream or during the determining of the data chunk hash values.
12. The method recited in claim 10 , wherein a first data chunk checksum value is generated during the parsing of the data stream.
13. The method recited in claim 12 , wherein the one or more respective intermediate hash values includes the first data chunk checksum value.
14. The method recited in claim 10 , wherein the hash function is an additive hash function which yields the data chunk hash value by summing a plurality of results produced by applying the mathematical function to a respective data chunk and a plurality of intermediate results, and wherein the one or more respective intermediate hash values includes one or more of the intermediate results.
15. The method recited in claim 10 , wherein the first data chunk fingerprint is determined by concatenating the respective data chunk hash value and the new respective intermediate hash value.
16. The method recited in claim 10 , wherein the communications interface is further configured to receive a data chunk status message, the data chunk status message indicating whether a data chunk associated with one of the data chunk fingerprints is stored at the remote server.
17. One or more non-transitory computer readable media having instructions stored thereon for causing a system to perform operations comprising:
parsing a data stream into a plurality of data chunks, the data stream being stored in memory;
generating one or more respective intermediate hash values during the parsing of the data stream;
via a processor, determining a first data chunk hash value for a first portion of each of the plurality of data chunks by applying a hash function to the first portion of each of the plurality of data chunks; generating a first data chunk fingerprint based on the first data chunk hash value and the one or more respective intermediate hash values by applying a mathematical function to the first data chunk hash value and the one or more respective intermediate hash values; determining a second data chunk hash value for a successive portion of each of the plurality of data chunks by applying the hash function to the successive portion of each of the plurality of data chunk; and generating a new respective intermediate hash value by applying the mathematical function to the second data chunk hash value and a previous intermediate hash value, the new respective intermediate hash value dependent upon the first portion and the second portion; and
via a communications interface, transmitting a respective data chunk fingerprint to a remote server for each of the plurality of data chunks, each data chunk fingerprint including the respective data chunk hash value and the one or more respective intermediate hash values.
18. The one or more non-transitory computer readable media recited in claim 17 , wherein the one or more respective intermediate hash values are generated during the parsing of the data stream or during the determining of the data chunk hash values.