Method and apparatus of duplicate packet detection and discard

1. A method for duplicate packet detection and discard, comprising:

for multiple incoming packet data, determining a first hash value of at least a portion of a packet and a timestamp and determining a second hash value of at least the portion of the packet; and

employing the determined first hash value as an index to a table and comparing the determined second hash value to a determined third hash value of data stored at a table entry at a table location determined by the index;

if no match is found, storing the determined second hash value of at least the portion of the packet and the timestamp as data at the table entry at the table location determined by the index, but if a match is found, comparing the timestamp of the table entry and the timestamp of the incoming packet data, and if the timestamps are within a time value of each other, determining that the packet data is a duplicate, otherwise, if the timestamps are not within the time value, determining that the packet data is not a duplicate, and storing the timestamp at the table entry and passing the incoming packet data on for further use.

2. The method according to claim 1 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a MAC frame.

3. The method according to claim 1 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a CRC portion of the packet.

4. The method according to claim 1 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a MAC frame and a CRC portion of the packet.

5. The method according to claim 1 , wherein at least one of said hash values comprise a hash value of a portion of the packet sufficient to provide a unique identification of the packet.

6. The method according to claim 1 , wherein at least one of said hash values comprise a hash value of a sufficient number of octets of the packet sufficient to provide a unique identification of the packet.

7. An apparatus for duplicate packet detection and discard, comprising:

a network interface for connecting to and observing traffic on a network;

a processor;

a storage medium for storing data;

wherein said processor observes multiple incoming packet data via said network interface, determines a first hash value of a packet, a second hash value of the packet, and a timestamp and stores said first hash value, said second hash value, said timestamp and said packet data; and

a device driver, wherein, in response to a request for packet data, said device driver obtains a stored packet data from said storage medium, employs at least a portion of the first hash value as an index to a table and comparing a third hash value of any data stored at a table entry stored at a table location addressed by the index with said second hash value;

if no match is found, storing the second hash value and said timestamp and packet data as the table entry at the table location addressed by the index, but if a match is found, comparing the timestamp stored as the table entry at the table location addressed by the index and the timestamp of the requested packet data, and if the timestamps are within a time value of each other, determining that the packet data is a duplicate, otherwise, if the timestamps are not within the time value, determining that the packet data is not a duplicate, and storing the timestamp of the requested packet data as the table entry at a table location addressed by the index and passing the requested packet data on for further use.

8. The apparatus according to claim 7 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a MAC frame.

9. The apparatus according to claim 7 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a CRC portion of the packet.

10. The apparatus according to claim 7 , wherein at least one of said hash values comprise a hash value of a portion of the packet other than a MAC frame and a CRC portion of the packet.

11. The apparatus according to claim 7 , wherein at least one of said hash values comprise a hash value of a portion of the packet sufficient to provide a unique identification of the packet.

12. The apparatus according to claim 7 , wherein at least one of said hash values comprise a hash value of a sufficient number of octets of the packet sufficient to provide a unique identification of the packet.

13. The apparatus according to claim 7 , wherein said apparatus is implemented as a network monitoring card.

14. The apparatus according to claim 13 , wherein network monitoring card comprises a PCIe interface.

15. An apparatus for duplicate packet detection and discard, comprising:

a server computer, said server comprising

a network interface for connecting to and observing traffic on a network;

a processor;

a storage medium for storing data;

wherein said processor observes multiple incoming packet data via said network interface, determines a first hash value of a packet, a second hash value of the packet data, and an incoming timestamp and stores said first hash value, said second hash value, said incoming timestamp and packet data; and

a device driver, wherein, in response to a request for packet data, said device driver obtains a packet from said storage medium, employs at least a portion of the first hash value as an index to a table and comparing a third hash value of any data stored at a table entry stored at a table location addressed by the index with the second hash value;

if no match is found, storing the second hash value and incoming timestamp and packet data at the table location addressed by the index, but if a match is found, comparing a stored timestamp stored at the table at the table location addressed by the index and the incoming timestamp, and if the incoming and stored timestamps are within a time value of each other, determining that the packet data is a duplicate, otherwise, if the incoming and stored timestamps are not within the time value, determining that the packet data is not a duplicate, and storing the incoming timestamp as the stored timestamp table entry at the table location addressed by the index and passing the requested packet data on for further use.

16. The apparatus according to claim 15 , wherein said hash values comprise a hash value of a portion of the packet other than a MAC frame.

17. The apparatus according to claim 15 , wherein said hash values comprise a hash value of a portion of the packet other than a CRC portion of the packet.

18. The apparatus according to claim 15 , wherein said hash values comprise a hash value of a portion of the packet other than a MAC frame and a CRC portion of the packet.

19. The apparatus according to claim 15 , wherein said hash values comprise a hash value of a portion of the packet sufficient to provide a unique identification of the packet.

20. The apparatus according to claim 15 , wherein said hash values comprise a hash value of a sufficient number of octets of the packet sufficient to provide a unique identification of the packet.