Identification of non-sequential data stored in memory
Systems and methods for identification of data stored in memory are provided. A data packet is received and a first packet byte within the data packet is compared to a first memory byte within a memory. A mismatch is determined between the first packet byte and the first memory byte. A memory location is accessed that contains a second memory byte that is non-consecutive with the first memory byte. A packet location accessed that contains a second packet byte that is non-consecutive with the first packet byte. The second packet byte is compared to the second memory byte. A retrieval instruction is generated based at least in part on a result of the comparison between the second packet byte and the second memory byte. Various embodiments may be included in a network memory architecture to allow for faster data matching and instruction generation in a central appliance.
1. A method comprising:
receiving packet data;
comparing a first packet byte within the packet data to a first memory byte within memory data;
determining a mismatch between the first packet byte and the first memory byte;
after determining the mismatch between the first packet byte and the first memory byte, accessing a memory location containing a second memory byte, the second memory byte being non-consecutive-with the first memory byte;
accessing a packet location containing a second packet byte, the second packet byte being non-consecutive with any other compared packet byte;
determining a match between the second packet byte and the second memory byte;
determining that a search end criteria has been met, the search end criteria being based on at least one of a number of jumps, a percentage of a memory searched, and a total number of memory bytes searched;
generating instructions to reassemble the packet data from the memory data; and
processing the instructions to reassemble the packet data based at least in part on a result of a comparison between the packet bytes and the memory bytes.
2. The method of claim 1 , wherein a number of packet bytes between the first packet byte and the second packet byte is equal to a number of memory bytes between the first memory byte and the second memory byte.
3. The method of claim 1 , further comprising determining that a match threshold between the matched packet bytes and the matched memory bytes is met.
4. The method of claim 3 , wherein the match threshold is based on at least one of a number of bytes having a match and a percentage of the bytes having a match.
5. The method of claim 1 , further comprising generating an insertion command to insert missing packet bytes into the memory data.
6. The method of claim 1 , further comprising generating a deletion command to delete memory bytes not matching the packet data.
7. The method of claim 1 , further comprising generating an overwrite command to replace memory bytes with packet bytes.
8. The method of claim 1 , wherein the memory data is stored in a faster memory in a network memory appliance.
9. The method of claim 1 , further comprising:
pre-scanning the packet data to generate a sample fingerprint;
identifying stored fingerprints identical to the sample fingerprint; and
copying the first memory byte associated with the identified stored fingerprints into the memory.
10. A system comprising:
a comparison module configured to compare a first packet byte within a packet data to a first memory byte within a memory data, determine a mismatch between the first packet byte and the first memory byte, access a memory location containing a second memory byte that is non-consecutive with the first memory byte after determining the mismatch between the first packet byte and the first memory byte, access a packet location containing a second packet byte that is non-consecutive with any other compared packet byte, determine a match between the second packet byte and the second memory byte, and determine that a search's end criteria has been met, the search end criteria being based on at least one of a number of jumps, a percentage of a memory searched, and a total number of memory bytes searched; and
a data module that generates instructions to reassemble the packet data from the memory data based at least in part on a result of a comparison between the packet bytes and the memory bytes.
11. The system of claim 10 , wherein a number of packet bytes between the first packet byte and the second packet byte is equal to a number of memory bytes between the first memory byte and the second memory byte.
12. The system of claim 10 , wherein the comparison module is further configured to determine that a match threshold between the matched packet bytes and the matched memory bytes is met.
13. The system of claim 12 , wherein the match threshold is based on at least one of a threshold number of bytes having a match, and a percentage of the bytes having a match.
14. The system of claim 10 , wherein the data module is further configured to generate an insertion command to insert missing packet bytes into the memory data.
15. The system of claim 10 , wherein the data module is further configured to generate a deletion command to delete memory bytes not matching the packet data.
16. The system of claim 10 , wherein the data module is further configured to generate an overwrite command to replace memory bytes with packet bytes.
17. The system of claim 10 , wherein the memory data is stored in a faster memory in a network memory appliance.
18. The system of claim 10 , further comprising a prescan module configured to pre-scan the packet data to generate a sample fingerprint, identify stored fingerprints identical to the sample fingerprint, and copy the first memory byte associated with the identified stored fingerprints into the memory.
19. A non-transitory computer readable medium having embodied thereon a program, the program being executable by a processor for performing a method for generating a retrieval instruction based on matched data, the method comprising:
receiving packet data;
comparing a first packet byte within the packet data to a first memory byte within memory data;
determining a mismatch between the first packet byte and the first memory byte;
after determining the mismatch between the first packet byte and the first memory byte, accessing a memory location containing a second memory byte, the second memory byte being non-consecutive-with the first memory byte;
accessing a packet location containing a second packet byte, the second packet byte being non-consecutive with any other compared packet byte;
determining a match between the second packet byte and the second memory byte;
determining that a search end criteria for a memory has been met, the search end criteria being based on at least one of a number of jumps, a percentage of the memory searched, and a total number of memory bytes searched;
generating instructions to reassemble the packet data from the memory data; and
processing the instructions to reassemble the packet data based at least in part on a result of a comparison between the packet bytes and the memory bytes.
20. The non-transitory computer readable medium of claim 19 , wherein a number of packet bytes between the first packet byte and the second packet byte is equal to a number of memory bytes between the first memory byte and the second memory byte.
21. The non-transitory computer readable medium of claim 19 , further comprising determining that a match threshold between the matched packet bytes and the memory bytes is met.
22. The non-transitory computer readable medium of claim 21 , wherein the match threshold is based on at least one of a number of bytes having a match and a percentage of the bytes having a match.
23. The non-transitory computer readable medium of claim 19 , wherein the method further comprises generating an insertion command to insert missing packet bytes into the memory data.
24. The non-transitory computer readable medium of claim 19 , wherein the method further comprises generating a deletion command to delete memory bytes not matching the packet data.
25. The non-transitory computer readable medium of claim 19 , wherein the method further comprises generating an overwrite command to replace memory bytes with packet bytes.
26. The non-transitory computer readable medium of claim 19 , wherein the memory data is stored in a faster memory in a network memory appliance.
27. The non-transitory computer readable medium of claim 19 , wherein the method further comprises:
pre-scanning the packet data to generate a sample fingerprint;
identifying stored fingerprints identical to the sample fingerprint; and
copying the first memory byte associated with the identified stored fingerprints into the memory.