Substring search algorithm optimized for hardware acceleration
Techniques are provided for generating a hash value for searching for substrings in a data stream without reading more than one element (e.g. one byte) at a time. According to one technique, a before a next element is added to an old hash value, the old hash value is circularly shifted one or more bits. The first original element is shifted a number of bits and XOR'ed against the old hash value. The next element is added to the old hash value. In one embodiment, an entry value is retrieved for each element from an index table and the XOR and shift operations are performed on the entry values. According to another technique, each Linear Feedback Shift Register (LFSR) of a plurality of LFSRs read in one element at a time beginning at different offsets. Each LFSR uses the same state machine. The result of reading a number of elements into an LFSR is used as the hash value.
1. A circuit, comprising:
a byte shift register array comprising a hash key output and a plurality of linear feedback shift registers (LFSRs), wherein each LFSR of the plurality of LFSRs implements a same finite state machine and is initialized to an initial state;
a memory having a hash key input coupled to the hash key output and a match output, and the memory comprising a plurality of stored digital signatures each keyed to a particular hash key; and
logic coupled to the byte shift register array which, when executed, is operable for performing the steps of:
reading a subset of input elements, of a plurality of input elements, into each LFSR of the plurality of LFSRs;
(a) reading a next input element, of the plurality of input elements, into each LFSR of the plurality of LFSRs;
(b) determining, based on a hash key indicated by one of the LFSRs of the plurality of LFSRs and the memory, whether said indicated hash key matches a hash key of a stored digital signature of the plurality of digital signatures, wherein the indicated hash key corresponds to multiple elements of the plurality of input elements;
(c) resetting said one of the LFSRs to the initial state without resetting any of the other LFSRs of the plurality of LFSRs; and
repeating (a)-(c) for each next input element in the plurality of input elements.
2. The circuit of claim 1 , wherein the stored digital signatures correspond to possible software viruses.
3. The circuit of claim 1 , wherein the logic, when executed, is operable to notify a software or hardware component associated with the circuit when the new hash key matches the hash key of a stored digital signature of the plurality of stored digital signatures.
4. The circuit of claim 1 , wherein each input element of the plurality of input elements corresponds to a byte in size.
5. The circuit of claim 1 , wherein each input element of the plurality of input elements does not correspond to a byte in size.
6. A method, comprising:
reading a subset of input elements, of a plurality of input elements, into each linear feedback shift register (LFSR) of a plurality of LFSRs, wherein each LFSR of the plurality of LFSRs implements a same state machine and is initialized to an initial state;
a memory having a hash key input and a match output, and the memory comprising a plurality of stored digital signatures each keyed to a particular hash key;
(a) reading a next input element, of the plurality of input elements, into each LFSR of the plurality of LFSRs;
(b) determining, based on a hash key indicated by one of the LFSRs of the plurality of LFSRs and the memory, whether said indicated hash key matches a hash key of a stored digital signature of the plurality of digital signatures, wherein the indicated hash key corresponds to multiple elements of the plurality of input elements;
(c) resetting said one of the LFSRs to the initial state without resetting any of the other LFSRs of the plurality of LFSRs; and
repeating (a)-(c) for each next input element in the plurality of input elements.
7. The method of claim 6 , wherein the stored digital signatures correspond to possible software viruses.
8. The method of claim 6 , further comprising notifying a software or hardware component associated with the circuit when the new hash key matches the hash key of a stored digital signature of the plurality of stored digital signatures.
9. The method of claim 6 , wherein each input element of the plurality of input elements corresponds to a byte in size.
10. The method of claim 6 , wherein each input element of the plurality of input elements does not correspond to a byte in size.
11. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:
reading a subset of input elements, of a plurality of input elements, into each linear feedback shift register (LFSR) of a plurality of LFSRs, wherein each LFSR of the plurality of LFSRs implements a same state machine and is initialized to an initial state;
a memory having a hash key input and a match output, and the memory comprising a plurality of stored digital signatures each keyed to a particular hash key;
(a) reading a next input element, of the plurality of input elements, into each LFSR of the plurality of LFSRs;
(b) determining, based on a hash key indicated by one of the LFSRs of the plurality of LFSRs and the memory, whether said indicated hash key matches a hash key of a stored digital signature of the plurality of digital signatures, wherein the indicated hash key corresponds to multiple elements of the plurality of input elements;
(c) resetting said one of the LFSRs to the initial state without resetting any of the other LFSRs of the plurality of LFSRs; and
repeating (a)-(c) for each next input element in the plurality of input elements.
12. The one or more non-transitory computer-readable media of claim 11 , wherein the stored digital signatures correspond to possible software viruses.
13. The one or more non-transitory computer-readable media of claim 11 , wherein the instructions, when executed by the one or more processors, further cause notifying a software or hardware component associated with the circuit when the new hash key matches the hash key of a stored digital signature of the plurality of stored digital signatures.
14. The one or more non-transitory computer-readable media of claim 11 , wherein each input element of the plurality of input elements corresponds to a byte in size.
15. The one or more non-transitory computer-readable media of claim 11 , wherein each input element of the plurality of input elements does not correspond to a byte in size.