Method and system for anonymizing activity records

1. A method for processing activity records, the method comprising:

establishing, by a computing system, an anonymization dictionary based on user activity encountered by the computing system, by:

obtaining, by an endpoint agent executing on the computing system, an activity record, wherein the activity record comprises metadata documenting the user activity;

generating, by an activity monitoring engine of the computing system that is operatively connected to the endpoint agent, the anonymization dictionary, wherein generating the anonymization dictionary comprises:

detecting, in the activity record, a plurality of target entities, wherein each of target entities is metadata related to the user activity and requires anonymization;

making a determination that a resource is associated with a subset of the target entities of the plurality of target entities;

after making the determination:

assigning an anonymized identity to the subset of target entities;

generating an anonymization identifier for each target entity in the subset of target entities to obtain a plurality of anonymization identifiers each comprising the anonymized identity; and

anonymizing user activity, based on the anonymization dictionary previously generated by the computing system by:

processing, by an anonymization engine executing on the computing system, the activity record using the anonymization dictionary to obtain an anonymized activity record, by:

replacing, in the activity record, target entities with their corresponding anonymized identifiers, specified in the anonymization dictionary; and

storing the anonymized activity record.

2. The method of claim 1 ,

wherein generating the anonymization dictionary further comprises, for each target entity in the subset of target entities: assigning an entity type of the target entity,

wherein generating the anonymization identifier for each target entity comprises storing the entity type in the anonymized identifier.

3. The method of claim 2 , wherein the entity type is at least one selected from the group consisting of a user name, an email address, a domain name, an IP address, a port number, a host name, a company name, and a partner name.

4. The method of claim 2 ,

wherein, for at least two target entities in the subset of target entities, the entity type is identical, and

wherein a unique instance identifier is assigned to each of the at least two target entities, and

wherein the unique instance identifiers are included in the anonymized identifiers for the at least two target entities.

5. The method of claim 4 , wherein the unique instance identifiers are globally unique identifiers.

6. The method of claim 1 , further comprising:

receiving an additional activity record;

processing the additional activity record to obtain a second plurality of target entities;

making a second determination that the anonymization dictionary does not have an entry to anonymize at least one of the second plurality of target entities;

based on the second determination, updating the anonymization dictionary to include at least one additional entry in order to obtain an updated anonymization dictionary; and

processing the additional activity record using the updated anonymization dictionary to obtain a second anonymized activity record.

7. The method of claim 1 further comprising:

receiving an additional activity record;

processing the additional activity record to obtain a second plurality of target entities;

making a second determination that the anonymization dictionary has an entry to anonymize at least one of the second plurality of target entities;

based on the second determination, processing the additional activity record using the anonymization dictionary to obtain a second anonymized activity record.

8. The method of claim 1 , wherein each of the plurality of target entities comprises at least one selected from a group consisting of an email address, a domain name, an IP address, a port number, a host name, a company name, and a partner name.

9. The method of claim 1 , wherein the anonymization dictionary is an access-protected database.

10. The method of claim 1 , further comprising:

analyzing the anonymized activity record using a threat detection algorithm.

11. The method of claim 1 , wherein the resource is a user.

12. A method for processing activity records, the method comprising:

establishing, by a computing system, an anonymization dictionary based on user activity encountered by the computing system, by:

obtaining, by an endpoint agent executing on the computing system, an activity record, wherein the activity record comprises metadata documenting the user activity;

generating, by an activity monitoring engine of the computing system that is operatively connected to the endpoint agent, the anonymization dictionary, wherein generating the anonymization dictionary comprises:

detecting, in the activity record, a plurality of target entities, wherein each of the plurality of target entities is metadata related to the user activity and requires anonymization;

assigning an anonymized identity to each unique target entity of the plurality of target entities;

generating dictionary entries for the plurality of target entities, wherein each dictionary entry comprises a target entity and a corresponding anonymized identifier comprising the anonymized identity for the target entity;

generating, by the computing system, an equivalence map, by:

making a determination that a resource is associated with a subset of the target entities of the plurality of target entities;

storing, in the equivalence map, an identity relationship specifying that the subset of the target entities is associated with the resource;

anonymizing user activity, based on the anonymization dictionary previously generated by the computing system, by:

processing, by an anonymization engine executing on the computing system, the activity record using the anonymization dictionary to obtain an anonymized activity record, by:

replacing, in the activity record, target entities with their corresponding anonymized identifiers, specified in the anonymization dictionary; and

storing the anonymized record.

13. The method of claim 12 , wherein generating the equivalence map may be performed prior to storing the anonymized record.

14. The method of claim 12 , wherein generating the equivalence map may be performed after storing the anonymized record.

15. A computing system, comprising:

at least one computer processor;

an endpoint agent comprising software instructions executing on the at least one computer processor, and enabling the computing system to:

obtain an activity record, wherein the activity record comprises metadata documenting the user activity;

an activity monitoring engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to establish an anonymization dictionary based on user activity encountered by the computing system, by:

generating an anonymization dictionary, comprising:

detecting, in the activity record, a plurality of target entities, wherein each of the target entities is metadata related to user activity and requires anonymization;

making a determination that a resource is associated with a subset of the target entities of the plurality of target entities;

assigning an anonymized identifier to the subset of target entities;

an anonymization engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to

anonymize user activity, based on the anonymization dictionary previously generated by the activity monitoring engine by:

processing the activity record using the anonymization dictionary to obtain an anonymized activity record by:

replacing, in the activity record, target entities with their corresponding anonymized identifiers, specified in the anonymization dictionary; and

a repository configured to store the anonymized activity record.

16. The system of claim 15 , wherein the anonymization engine further comprises software instructions executing on the at least one computer processor, that enable the computing system to:

receive a request from a third-party system for the activity record;

in response to the request:

identify the anonymized activity record;

detect in the anonymized activity record a second plurality of target entities to anonymize;

replace the second plurality of target entities with corresponding anonymized identifiers from the anonymization dictionary in order to obtain a second anonymized activity record; and

provide the second anonymized activity record to the third-party system.

17. The system of claim 15 , wherein the anonymization engine further comprises software instructions executing on the at least one computer processor, that enable the computing system to:

receive a request from a third-party system for the anonymized activity record;

in response to the request:

detect in the anonymized activity record a second plurality of target entities to anonymize;

replace the second plurality of target entities with corresponding anonymized identifiers from the anonymization dictionary in order to obtain a second anonymized activity record; and

provide the second anonymized activity record to the third-party system.

18. The system of claim 15 , further comprising:

a threat analysis engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to analyze the at least one anonymized activity record using a threat detection algorithm.

19. The system of claim 15 , further comprising:

a threat analysis engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to analyze the at least one anonymized activity record using a threat detection algorithm and the equivalence map.

20. A computing system, comprising:

at least one computer processor;

an endpoint agent comprising software instructions executing on the at least one computer processor, and enabling the computing system to:

obtain an activity record, wherein the activity record comprises metadata documenting the user activity;

an activity monitoring engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to establish an anonymization dictionary based on user activity encountered by the computing system by:

generating an anonymization dictionary, comprising:

detecting in the activity record, a plurality of target entities, wherein each of the plurality of target entities is metadata related to the user activity and requires anonymization;

assigning an anonymized identity to each unique target entity of the plurality of target entities;

generating dictionary entries for the plurality of target entities, wherein each dictionary entry comprises a target entity and a corresponding anonymized identifier comprising the anonymized identity for the target entity;

generating an equivalence map, by:

making a determination that a resource is associated with a subset of the target entities of the plurality of target entities;

storing, in the equivalence map, an identity relationship specifying that the subset of the target entities is associated with the resource;

an anonymization engine comprising software instructions executing on the at least one computer processor, and enabling the computing system to:

anonymize user activity, based on the anonymization dictionary previously generated by the activity monitoring engine, by:

processing the activity record using the anonymization dictionary to obtain an anonymized activity record, by:

replacing, in the activity record, target entities with their corresponding anonymized identifiers, specified in the anonymization dictionary; and

a repository configured to store the anonymized activity record.

21. A method for processing activity records, the method comprising:

obtaining a plurality of activity records, wherein each activity record comprises metadata documenting user activity;

providing at least one of the plurality of activity records to each of a plurality of workers;

receiving, from each of the plurality of workers, a set of target entities detected in the at least one activity record, wherein each target entity is metadata related to the user activity and requires anonymization;

generating, using the sets of target entities, an anonymization dictionary, by:

making a determination that a resource is associated with a plurality of the target entities, wherein a first target entity is obtained from a first worker of the plurality of workers and the second target entity is obtained from a second worker of the plurality of workers; and

assigning anonymized identifiers to the plurality of target entities;

providing a copy of the anonymization dictionary to each of the plurality of workers;

receiving, from each of the plurality of workers, at least one anonymized activity record generated using the copy of the anonymization dictionary, wherein in the at least one anonymized activity record, target entities are replaced by the corresponding anonymized identifiers, specified in the anonymization dictionary;

storing the at least one anonymized activity record.

22. The method of claim 21 , further comprising:

analyzing the at least one anonymized activity record using a threat detection algorithm.

23. The method of claim 21 , wherein the plurality of activity records are obtained from a plurality of endpoint agents.

24. A method for processing activity records, the method comprising:

obtaining a first plurality of anonymized activity records and a first local anonymization dictionary from a first endpoint agent, wherein each of the anonymized activity records comprises at least one anonymized identifier that replaces a target entity, wherein that target entity is metadata related to user activity and requires anonymization;

obtaining a second plurality of anonymized activity records and a second local anonymization dictionary from a second endpoint agent;

storing the first plurality of anonymized activity records, the first local anonymization dictionary, the second plurality of anonymized activity records, and the second local anonymization dictionary; and

performing a threat analysis using the first plurality of anonymized activity records, the first local anonymization dictionary, the second plurality of anonymized activity records, and the second local anonymization dictionary.