Anonymized network data collection and network threat assessment and monitoring systems and methods
Systems and methods for data collection and processing in a network, including one or more sensors disposed in a network interface and configured to collect raw signal traffic data where each sensor is further configured to parse the raw signal traffic data into network protocols; split the network protocols into content data and metadata; derive contextual metadata from the content data; compile the metadata and the derived metadata to produce anonymized metadata; encrypt the anonymized metadata; and transmit to the encrypted anonymized metadata to a unified data server.
1. A computer implemented method comprising:
obtaining a plurality of encrypted anonymized network communications from a first network, the plurality of encrypted anonymized network communications including encrypted anonymized network metadata and contextual metadata derived from network communications traffic associated with the first network;
decrypting the first plurality of encrypted anonymized network communications to obtain a plurality of anonymized network communications from the first network, including a first anonymized network communication; and
generating a network security assessment associated with the first network based at least partially on the first anonymized network communication.
2. The method of claim 1 , further comprising generating a network security assessment of a second network based at least partially on the first anonymized network communication.
3. The method of claim 1 , comprising:
obtaining a security alert relating to an attacking network element and the first network;
generating a network entity having a graph representation of the attacking network element, the network entity being at least partially based on information related to the first anonymized network communication;
providing a displayable version the network entity;
generating a threat entity using information in the data store, the threat entity: being at least partially based on information related to the first anonymized network communication, having a graph representation, and describing a customer
context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the threat entity.
4. The method of claim 3 wherein the customer context includes information regarding a compromised network element in the first network and network communications between the compromised network element and the attacking network element.
5. The method of claim 3 , comprising:
obtaining a plurality of encrypted anonymized network communications from a second network, decrypting the plurality of encrypted anonymized network communications from the second network, thereby obtaining a plurality of anonymized network communications from the second network, including a second anonymized network communication; providing the second anonymized network communication to the data store; and generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on information related to the first anonymized network communication and the second anonymized network communication, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.
6. The method of claim 3 , comprising:
applying analytics to the threat entity and information in the data store, identifying a network relationship between the attacking network element and an unknown network element;
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the network relationship, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element and the unknown network element; and
providing a displayable version of the updated threat entity.
7. The method of claim 3 , comprising:
obtaining external network security information from an external source, providing the network security information to the data store; and
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the external network security information, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.
8. A non-transitory computer-readable medium have contents, execution of which in a processing system cause the processing system to perform a method comprising:
identifying a communication protocol related to a first network communication of a plurality of network communications of a network;
identifying a network metadata portion of the first network communication, based at least partially on the communication protocol;
identifying a data portion of the first network communication based at least partially on the communication protocol;
generating anonymized contextual metadata based on the data portion; and
combining the network metadata and the anonymized contextual metadata into a first anonymized network communication.
9. The non-transitory computer-readable medium of claim 8 wherein the method includes generating an encrypted version of the first anonymized network communications, and providing the plurality of encrypted anonymized network communications to a remote server.
10. The non-transitory computer-readable medium of claim 9 wherein the contextual metadata is derived from characteristics of the first network communication including authorization traffic, HTTP flow, encryption, VPN activity, encryption certificates, passive fingerprints and application traffic.
11. A non-transitory computer-readable medium have contents that configure a processing system to perform a process comprising:
obtaining a plurality of encrypted anonymized network communications from a first network, the plurality of encrypted anonymized network communications including encrypted anonymized network metadata and contextual metadata derived from network communications traffic associated with the first network;
decrypting the first plurality of encrypted anonymized network communications to obtain a plurality of anonymized network communications from the first network, including a first anonymized network communication; and
generating a network security assessment associated with the first network based at least partially on the first anonymized network communication.
12. The non-transitory computer-readable medium of claim 11 , the process further comprising generating a network security assessment of a second network based at least partially on the first anonymized network communication.
13. The non-transitory computer-readable medium of claim 11 , the process comprising:
obtaining a security alert relating to an attacking network element and the first network;
generating a network entity having a graph representation of the attacking network element using information in the data store, the network entity being at least partially based on information related to the first anonymized network communication;
providing a displayable version the network entity;
generating a threat entity using information in the data store, the threat entity: being at least partially based on information related to the first anonymized network communication, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the threat entity.
14. The non-transitory computer-readable medium of claim 13 wherein the customer context includes information regarding a compromised network element in the first network and network communications between the compromised network element and the attacking network element.
15. The non-transitory computer-readable medium of claim 13 , the process comprising:
obtaining a plurality of encrypted anonymized network communications from a second network, decrypting the plurality of encrypted anonymized network communications from the second network, thereby obtaining a plurality of anonymized network communications from the second network, including a second anonymized network communication; providing the second anonymized network communication to the data store; and generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on information related to the first anonymized network communication and the second anonymized network communication, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.
16. The non-transitory computer-readable medium of claim 13 , the process comprising:
applying analytics to the threat entity and information in the data store, identifying a network relationship between the attacking network element and an unknown network element;
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the network relationship, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element and the unknown network element; and
providing a displayable version of the updated threat entity.
17. The non-transitory computer-readable medium of claim 13 , the process comprising:
obtaining external network security information from an external source, providing the network security information to the data store; and
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the external network security information, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.
18. A system comprising:
a memory; and
processing circuitry coupled to the memory, wherein the processing circuitry, when in operation, controls a process that includes:
obtaining a plurality of encrypted anonymized network communications from a first network, the plurality of encrypted anonymized network communications including encrypted anonymized network metadata and contextual metadata derived from network communications traffic associated with the first network;
decrypting the first plurality of encrypted anonymized network communications, to obtain a plurality of anonymized network communications from the first network, including a first anonymized network communication; and
generating a network security assessment associated with the first network based at least partially on the first anonymized network communication.
19. The system of claim 18 , the process further comprising generating a network security assessment of a second network based at least partially on the first anonymized network communication.
20. The system of claim 18 , the process comprising:
obtaining a security alert relating to an attacking network element and the first network;
generating a network entity having a graph representation of the attacking network element using information in the data store, the network entity being at least partially based on information related to the first anonymized network communication;
providing a displayable version the network entity;
generating a threat entity using information in the data store, the threat entity: being at least partially based on information related to the first anonymized network communication, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the threat entity.
21. The system of claim 20 wherein the customer context includes information regarding a compromised network element in the first network and network communications between the compromised network element and the attacking network element.
22. The system of claim 20 , the process comprising:
obtaining a plurality of encrypted anonymized network communications from a second network, decrypting the plurality of encrypted anonymized network communications from the second network, thereby obtaining a plurality of anonymized network communications from the second network, including a second anonymized network communication; providing the second anonymized network communication to the data store; and generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on information related to the first anonymized network communication and the second anonymized network communication, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.
23. The system of claim 20 , the process comprising:
applying analytics to the threat entity and information in the data store, identifying a network relationship between the attacking network element and an unknown network element;
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the network relationship, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element and the unknown network element; and
providing a displayable version of the updated threat entity.
24. The system of claim 20 , the process comprising:
obtaining external network security information from an external source, providing the network security information to the data store; and
generating an updated threat entity using information in the data store, the updated threat entity:
being at least partially based on the external network security information, having a graph representation, and describing a customer context, a global context, and a network context related to the attacking network element; and
providing a displayable version of the updated threat entity.