Pre-fetching data into a memory
Systems and methods for pre-fetching of data in a memory are provided. By pre-fetching stored data from a slower memory into a faster memory, the amount of time required for data retrieval and/or processing may be reduced. First, data is received and pre-scanned to generate a sample fingerprint. Fingerprints stored in a faster memory that are similar to the sample fingerprint are identified. Data stored in the slower memory associated with the identified stored fingerprints is copied into the faster memory. The copied data may be compared to the received data. Various embodiments may be included in a network memory architecture to allow for faster data matching and instruction generation in a central appliance.
1. A method for copying stored data from a slower memory into a faster memory, the method comprising:
receiving at a first device data from a network;
pre-scanning the received data to generate a sample fingerprint;
copying stored data associated with the sample fingerprint from the slower memory into the faster memory;
comparing first data bytes in the received data with first memory bytes within the faster memory;
determining a mismatch between one of the first data bytes and one of the first memory bytes;
accessing second memory bytes in the faster memory that are non-consecutive with the first memory bytes to determine second data bytes that match the second memory bytes; and
sending instructions to a second device to reconstruct the received data.
2. The method of claim 1 , further comprising retrieving the stored data associated with the sample fingerprint according to a prioritization algorithm.
3. The method of claim 1 , wherein the faster memory comprises random access memory and a slower memory comprises a hard disk.
4. The method of claim 1 , further comprising determining whether the received data represented by the sample fingerprint is similar to positioned data in the faster memory.
5. The method of claim 1 , wherein the faster memory is associated with a network memory appliance.
6. The method of claim 1 , wherein the fingerprint comprises a hash value.
7. The method of claim 1 , wherein the stored data corresponds to other stored data in a slower memory in the second digital device.
8. The method of claim 1 , further comprising:
determining whether the received data represented by the sample fingerprint is in the slower memory of the second digital device; and
transmitting at least a portion of the received data to the second digital device.
9. A system for copying stored data from a slower memory into a faster memory, the system comprising:
a scan module configured to pre-scan received data to generate a sample fingerprint, the received data being received from a network;
a search module configured to identify stored fingerprints similar to the sample fingerprint;
a duplication module configured to copy stored data associated with the identified stored fingerprints from the slower memory into the faster memory; and
a slip search module configured to compare a first data byte in the received data with a first memory byte within the faster memory, determine a mismatch between the first data byte and the first memory byte, access a memory location containing a second memory byte that is non-consecutive with the first memory byte to determine a second data byte that matches the second memory byte, and send instructions to a second device to reconstruct the received data.
10. The system of claim 9 , further comprising a priority module configured to retrieve the stored data associated with the identified stored fingerprints according to a prioritization algorithm.
11. The system of claim 9 , wherein the faster memory comprises random access memory and a slower memory comprises a hard disk.
12. The system of claim 9 , further comprising a comparison module configured to determine whether the received data represented by the sample fingerprint is similar to positioned data in a faster memory.
13. The system of claim 9 , wherein the faster memory is associated with a network memory appliance.
14. The system of claim 9 , wherein the fingerprint comprises a hash value.
15. The system of claim 9 , wherein the stored data corresponds to other stored data in a slower memory in the second digital device.
16. The system of claim 9 , further comprising a comparison module, wherein the comparison module is configured to determine whether the received data represented by the sample fingerprint is in the slower memory of the second digital device and transmit at least a portion of the received data to the second digital device.
17. A non-transitory computer readable medium having embodied thereon a program, the program being executable by a processor for performing a method for copying stored data from a slower memory into a faster memory, the method comprising:
receiving at a first device data from a network;
pre-scanning the received data to generate a sample fingerprint;
copying stored data associated with the sample fingerprint from the slower memory into the faster memory;
comparing first data bytes in the received data with first memory bytes within the faster memory;
determining a mismatch between one of the first data bytes and one of the first memory bytes;
accessing second memory bytes in the faster memory that are non-consecutive with the first memory bytes to determine second data bytes that match the second memory bytes; and
sending instructions to a second device to reconstruct the received data.
18. The non-transitory computer readable medium of claim 17 , wherein the method further comprises retrieving the stored data associated with the sample fingerprint according to a prioritization algorithm.
19. The non-transitory computer readable medium of claim 17 , wherein the faster memory comprises random access memory and a slower memory comprises a hard disk.
20. The non-transitory computer readable medium of claim 17 , wherein the method further comprises determining whether the received data represented by the sample fingerprint is similar to positioned data in the faster memory.
21. The non-transitory computer readable medium of claim 17 , wherein the faster memory is associated with a network memory appliance.
22. The non-transitory computer readable medium of claim 17 , wherein the fingerprint comprises a hash value.
23. The non-transitory computer readable medium of claim 17 , wherein the stored data corresponds to other stored data in a slower memory in a second digital device.
24. The non-transitory computer readable medium of claim 17 , wherein the method further comprises:
determining whether the received data represented by the sample fingerprint is in the slower memory; and
transmitting at least a portion of the received data to the second digital device.