IP Library Granted Patent US 10,965,696
Granted Patent B1
US 10,965,696 · App. 15/797,065 · Granted Mar 30, 2021

Evaluation of anomaly detection algorithms using impersonation data derived from user data

Inventors: Shay Amram (Holon, IL); Alex Zaslavsky (Brookline, MA)
Assignee: EMC IP Holding Company LLC
H04L63/1425G06N20/00H04L63/1416G06Q10/0635
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,965,696
App. No.
15/797,065
Granted
Mar 30, 2021
Kind
B1
Abstract

Techniques are provided for evaluating anomaly detection algorithms using impersonation data derived from user transaction data. An exemplary method comprises obtaining transaction data of a given enterprise organization comprising transactions of a plurality of users; generating impersonation data by modifying one or more features of a subset of the transaction data of the given enterprise organization; classifying (i) at least a portion of the transaction data of the given enterprise organization, and (ii) at least a portion of the impersonation data using the anomaly detection algorithm of the given enterprise organization, wherein records of the impersonation data comprise a known classification; and evaluating a performance of the anomaly detection algorithm of the given enterprise organization by comparing the classification of records of the impersonation data by the anomaly detection algorithm with the known classification.

Claims (40)

1. A method, comprising:

obtaining transaction data of a given enterprise organization comprising transactions of a plurality of users, wherein the transaction data comprises one or more of online transactions, login communications and attempts to access protected resources;

extracting one or more features from at least a subset of the transaction data of the given enterprise organization;

generating impersonation data by modifying one or more of the extracted features of the subset of the transaction data of the given enterprise organization from a first value for a given feature to a second value for the given feature to simulate a plurality of attack classes, wherein the extracted features of the transaction data that are modified are defined for each simulated attack class, wherein the generating the impersonation data comprises modifying a feature value of one or more of a user identifier feature and a location feature of a given transaction of a given user from the transaction data to a different feature value from the transaction data;

classifying, using at least one processing device, (i) at least a portion of the transaction data of the given enterprise organization, and (ii) at least a portion of the impersonation data, using an anomaly detection algorithm of the given enterprise organization, wherein records of the impersonation data comprise a known classification;

evaluating, using the at least one processing device, a performance of the anomaly detection algorithm of the given enterprise organization to identify at least one anomaly by comparing the classification of records of the impersonation data by the anomaly detection algorithm with the known classification for at least one of the plurality of simulated attack classes; and

initiating one or more remedial actions in response to the identified at least one anomaly.

2. The method of claim 1 , wherein the features of the transaction data that are modified are selected to simulate a given predefined attack scenario.

3. The method of claim 1 , wherein one or more modified features of a given impersonation record are based on the transaction data of one or more of the plurality of users of the given enterprise organization.

4. The method of claim 1 , wherein the evaluating comprises determining a risk score for one or more records in the impersonation data.

5. The method of claim 1 , further comprising the step of training the anomaly detection algorithm of the given enterprise organization using a different subset of the transaction data of the given enterprise organization.

6. The method of claim 1 , wherein the anomaly detection algorithm further comprises a supervised learning method that uses the known classification of the impersonation data to detect one or more of at least one feature and at least one pattern that corresponds to an impersonation attempt.

7. The method of claim 1 , wherein the known classification in the impersonation data provides an indication that the impersonation data corresponds to a false event.

8. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:

obtaining transaction data of a given enterprise organization comprising transactions of a plurality of users, wherein the transaction data comprises one or more of online transactions, login communications and attempts to access protected resources;

extracting one or more features from at least a subset of the transaction data of the given enterprise organization;

generating impersonation data by modifying one or more of the extracted features of the subset of the transaction data of the given enterprise organization from a first value for a given feature to a second value for the given feature to simulate a plurality of attack classes, wherein the extracted features of the transaction data that are modified are defined for each simulated attack class, wherein the generating the impersonation data comprises modifying a feature value of one or more of a user identifier feature and a location feature of a given transaction of a given user from the transaction data to a different feature value from the transaction data;

classifying, using the at least one processing device, (i) at least a portion of the transaction data of the given enterprise organization, and (ii) at least a portion of the impersonation data, using an anomaly detection algorithm of the given enterprise organization, wherein records of the impersonation data comprise a known classification;

evaluating, using the at least one processing device, a performance of the anomaly detection algorithm of the given enterprise organization to identify at least one anomaly by comparing the classification of records of the impersonation data by the anomaly detection algorithm with the known classification for at least one of the plurality of simulated attack classes; and

initiating one or more remedial actions in response to the identified at least one anomaly.

9. The computer program product of claim 8 , wherein the features of the transaction data that are modified are selected to simulate a given predefined attack scenario.

10. The computer program product of claim 8 , wherein one or more modified features of a given impersonation record are based on the transaction data of one or more of the plurality of users of the given enterprise organization.

11. The computer program product of claim 8 , wherein the anomaly detection algorithm further comprises a supervised learning method that uses the known classification of the impersonation data to detect one or more of at least one feature and at least one pattern that corresponds to an impersonation attempt.

12. The computer program product of claim 8 , wherein the known classification in the impersonation data provides an indication that the impersonation data corresponds to a false event.

13. An apparatus, comprising:

a memory; and

at least one processing device, coupled to the memory, operative to implement the following steps:

obtaining transaction data of a given enterprise organization comprising transactions of a plurality of users, wherein the transaction data comprises one or more of online transactions, login communications and attempts to access protected resources;

extracting one or more features from at least a subset of the transaction data of the given enterprise organization;

generating impersonation data by modifying one or more of the extracted features of the subset of the transaction data of the given enterprise organization from a first value for a given feature to a second value for the given feature to simulate a plurality of attack classes, wherein the extracted features of the transaction data that are modified are defined for each simulated attack class, wherein the generating the impersonation data comprises modifying a feature value of one or more of a user identifier feature and a location feature of a given transaction of a given user from the transaction data to a different feature value from the transaction data;

classifying, using the at least one processing device, (i) at least a portion of the transaction data of the given enterprise organization, and (ii) at least a portion of the impersonation data, using an anomaly detection algorithm of the given enterprise organization, wherein records of the impersonation data comprise a known classification;

evaluating, using the at least one processing device, a performance of the anomaly detection algorithm of the given enterprise organization to identify at least one anomaly by comparing the classification of records of the impersonation data by the anomaly detection algorithm with the known classification for at least one of the plurality of simulated attack classes; and

initiating one or more remedial actions in response to the identified at least one anomaly.

14. The apparatus of claim 13 , wherein the features of the transaction data that are modified are selected to simulate a given predefined attack scenario.

15. The apparatus of claim 13 , wherein one or more modified features of a given impersonation record are based on the transaction data of one or more of the plurality of users of the given enterprise organization.

16. The apparatus of claim 13 , wherein the evaluating comprises determining a risk score for one or more records in the impersonation data.

17. The apparatus of claim 13 , further comprising the step of training the anomaly detection algorithm of the given enterprise organization using a different subset of the transaction data of the given enterprise organization.

18. The apparatus of claim 13 , wherein the anomaly detection algorithm further comprises a supervised learning method that uses the known classification of the impersonation data to detect one or more of at least one feature and at least one pattern that corresponds to an impersonation attempt.

19. The computer program product of claim 8 , further comprising training the anomaly detection algorithm of the given enterprise organization using a different subset of the transaction data of the given enterprise organization.

20. The apparatus of claim 13 , wherein the known classification in the impersonation data provides an indication that the impersonation data corresponds to a false event.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (044535/0109) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO WYSE TECHNOLOGY L.L.C.)
Reel/Frame 060753/0414 →
RELEASE OF SECURITY INTEREST AT REEL 044535 FRAME 0001 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058298/0475 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
PATENT SECURITY AGREEMENT (CREDIT) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 044535/0001 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 044535/0109 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 30, 2017
From: AMRAM, SHAY; ZASLAVSKY, ALEX
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 043980/0580 →
Cited By (1)
US 12,388,869