Method and system for generating durable host identifiers using network artifacts
A host identification engine receives network traffic from a network and uses one or more artifact extractors to extract artifact data items that can identify a host. The artifact data items can be stored in a host signature database. Network addresses to which the hosts correspond can be stored in a network address database. A mapping table can be implemented to match the data in the signature database and network database to generate durable host identification data that can accurately track hosts as they use different identification data and/or move between hosts.
1. A computer-implemented method for generating network host identification data, comprising:
extracting one or more artifact data items from network traffic, the network traffic corresponding to communications between one or more hosts;
generating durable host identification data, the durable host identification data corresponding to a collection of the one or more artifact data items that identify a host across different network addresses;
storing the durable host identification data in a signature database; and
storing one or more network addresses and time periods in a network address database, the one or more network addresses corresponding to the durable host identification data, the time periods corresponding to the one or more network addresses that were in use by the one or more hosts.
2. The computer-implemented method of claim 1 , further comprising
implementing a table to match data from the signature database and the network address database, the table comprising a mapping between the durable host identification data and the one or more network addresses to track the one or more hosts.
3. The computer-implemented method of claim 1 , wherein the the one or more artifact data items are multi-valued artifacts or single-valued artifacts.
4. The computer-implemented method of claim 3 , wherein multi-valued artifacts are appended to matching multi-valued artifacts.
5. The computer-implemented method of claim 3 , wherein new single-valued artifacts overwrite matching existing single-valued artifacts.
6. The computer-implemented method of claim 1 , wherein artifacts are unambiguous artifacts or ambiguous artifacts.
7. The computer-implemented method of claim 6 , wherein ambiguous artifacts for a host are added to a matching collection of artifacts for the host in the signature database and wherein unambiguous artifacts are used to detect one or more duplicate collection of artifacts.
8. The computer-implemented method of claim 1 , further comprising:
receiving network traffic through a network component by creating a copy of the network traffic for analysis, the network traffic being parsed into one or more session datasets comprising unidirectional flows generated by one or more hosts.
9. The computer-implemented method of claim 8 , further comprising:
merging one or more duplicate collection of artifacts.
10. The computer-implemented method of claim 1 , further comprising:
responding to requests for network address to durable host identification mapping data using a query agent.
11. The computer-implemented method of claim 1 , further comprising:
distributing the signature database across a plurality of different networks, the plurality of different networks comprising a first network and a second network that is different from the first network; and
identifying a host on the first network using durable host identification data generated on the second network.
12. The computer-implemented method of claim 1 , further comprising:
organizing the one or more artifact data items into clusters that corresponds to the network address for the host, wherein the one or more artifact data items corresponds to the host.
13. A system for generating network host identification data, comprising:
a computer processor to execute a set of program code instructions;
a memory to hold the set of program code instructions, in which program code instructions comprises program code to perform:
extracting one or more artifact data items from network traffic, the network traffic corresponding to communications between one or more hosts;
generating durable host identification data, the durable host identification data corresponding to a collection of the one or more artifact data items that identify a host across different network addresses;
storing the durable host identification data in a signature database; and
storing one or more network addresses and time periods in a network address database, the one or more network addresses corresponding to the durable host identification data, the time periods corresponding to the one or more network addresses that were in use by the one or more hosts.
14. The system of claim 13 , further comprising
implementing a table to match data from the signature database and the network address database, the table comprising a mapping between the durable host identification data and the one or more network addresses to track the one or more hosts.
15. The system of claim 13 , wherein the one or more artifact data items are multi-valued artifacts or single-valued artifacts.
16. The system of claim 15 , wherein multi-valued artifacts are appended to matching multi-valued artifacts.
17. The system of claim 15 , wherein new single-valued artifacts overwrite matching existing single-valued artifacts.
18. The system of claim 13 , wherein artifacts are unambiguous artifacts or ambiguous artifacts.
19. The system of claim 18 , wherein ambiguous artifacts for a host are added to a matching collection of artifacts for the host in the signature database.
20. The system of claim 18 , wherein unambiguous artifacts are used to detect one or more duplicate collection of artifacts.
21. The system of claim 20 , in which the program code instructions further comprises program code to merge one or more duplicate collection of artifacts.
22. The system of claim 13 , in which the program code instructions further comprises program code to respond to requests for network address to durable host identification mapping data using a query agent.
23. The system of claim 13 , in which the program code instructions further comprises program code to distribute the signature database across a plurality of different networks, the plurality of different networks comprising a first network and a second network that is different from the first network; and identify a host on the first network using durable host identification data generated on the second network.
24. The system of claim 13 , further comprising:
organizing the one or more artifact data items into clusters that corresponds to the network address for the host, wherein the one or more artifact data items corresponds to the host.
25. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for generating network host identification data, the method comprising:
extracting one or more artifact data items from network traffic, the network traffic corresponding to communications between one or more hosts;
generating durable host identification data, the durable host identification data corresponding to a collection of the one or more artifact data items that identify a host across different network addresses;
storing the durable host identification data in a signature database; and
storing one or more network addresses and time periods in a network address database, the one or more network addresses corresponding to the durable host identification data, the time periods corresponding to the one or more network addresses that were in use by the one or more hosts.
26. The computer program product of claim 25 , further comprising
implementing a table to match data from the signature database and the network address database, the table comprising a mapping between the durable host identification data and the one or more network addresses to track the one or more hosts.
27. The computer program product of claim 25 , wherein the one or more artifact data items are multi-valued artifacts or single-valued artifacts.
28. The computer program product of claim 25 , wherein the non-transitory computer readable medium further comprises instructions which, when executed by the processor, causes the processor to execute the method further comprising:
responding to requests for network address to durable host identification mapping data using a query agent.
29. The computer program product of claim 25 , wherein the non-transitory computer readable medium further comprises instructions which, when executed by the processor, causes the processor to execute the method further comprising:
distributing the signature database across a plurality of different networks, the plurality of different networks comprising a first network and a second network that is different from the first network; and
identifying a host on the first network using durable host identification data generated on the second network.
30. The computer program product of claim 25 , further comprising:
organizing the one or more artifact data items into clusters that corresponds to the network address for the host, wherein the one or more artifact data items corresponds to the host.