IP Library Granted Patent US 10,503,906
Granted Patent B2
US 10,503,906 · App. 14/956,464 · Granted Dec 10, 2019

Determining a risk indicator based on classifying documents using a classifier

Inventors: Jake Seigel (Halifax, CA); Robert MacIntosh (Halifax, CA)
Assignee: QUEST SOFTWARE INC.
G06F21/577G06F21/552G06F21/554H04L63/1425G06F2221/2113H04L63/1416
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,503,906
App. No.
14/956,464
Granted
Dec 10, 2019
Kind
B2
Abstract

Systems and techniques for determining and displaying risk indicators are described. A set of event logs occurring in a time interval and associated with a user account retrieving may be retrieved from an event log database. For individual event logs in the set of event logs, a context may be determined. A resource associated with the individual log may have an associated classification. An activity risk associated with the individual event log may be determined based at least in part on the first context and, if applicable, on the associated classification. For individual event logs in the set of event logs, a risk indicator may be determined based at least in part on the activity risk. In some cases, a cumulative risk indicator may be determined for the particular time interval based on the risk indicators associated with the individual event logs.

Claims (117)

1. A computer-implemented method, comprising:

generating a classifier based on a classifier algorithm;

training, using electronic training data, the classifier to classify data within a predetermined accuracy, wherein training the classifier to classify data within the predetermined accuracy comprises:

training the classifier with training data;

classifying, by the classifier, test data to generate classified test data;

determining that the test data is not within the predetermined accuracy based on a comparison of the classified test data with a pre-classified set of test data;

tuning the classifier;

retraining the classifier with the training data;

reclassifying, by the classifier, the test data to generate reclassified test data;

determining that the reclassified test data is within the predetermined accuracy based on a comparison of the reclassified test data with the pre-classified set of test data;

receiving, via a network, a first event log from an agent that is configured to monitor data activities at a data storage, the first event log including information pertaining to a first access to data stored in the data storage by an electronic device associated with a user account;

causing the first event log to be stored in an event log database;

retrieving, from the event log database, a set of event logs associated with a user account, each event log of the set of event logs occurring in a time interval;

selecting the first event log from the set of event logs, the first event log associated with a resource having a classification assigned to the resource by the classifier based at least in part on a taxonomy;

determining a first context associated with the first event log;

determining a first activity risk associated with the first event log based at least in part on the first context;

determining a first risk indicator associated with the user account based at least in part on the first activity risk;

receiving, via the network, a second event log from the agent during the time interval, the second event log including information pertaining to a second access to data stored in the data storage by the electronic device associated with the user account;

determining a second risk indicator associated with the user account based at least in part on the second event log; and

determining a cumulative risk indicator for the time interval based on the first risk indicator and the second risk indicator.

2. The computer-implemented method of claim 1 , further comprising:

selecting a third event log from the set of event logs;

determining a second context associated with the third event log;

determining a second activity risk associated with the third event log based at least in part on the second context; and

determining a third risk indicator associated with the user account based at least in part on the second activity risk.

3. The computer-implemented method of claim 2 , further comprising:

displaying, for the time interval, a graph that includes a plurality of risk indicators including at least the first risk indicator the second risk indicator, and the third risk indicator.

4. The computer-implemented method of claim 2 , further comprising:

determining a first cumulative risk indicator associated with the user account during a first time interval based at least in part on the first risk indicator and the second risk indicator;

determining a second cumulative risk indicator associated with the user account during a second time interval based at least in part on a second set of event logs; and

displaying a graph that includes a plurality of cumulative risk indicators including at least the first cumulative risk indicator and the second cumulative risk indicator.

5. The computer-implemented method of claim 1 , further comprising:

determining the resource identified in the first event log as being accessed by an activity;

determining the classification associated with the resource; and

associating a resource risk associated with the resource based at least in part on the classification.

6. The computer-implemented method of claim 5 , determining the first activity risk associated with the first event log based at least in part on the first context comprises:

determining the first activity risk based at least in part on the first context and the resource risk associated with the resource.

7. The computer-implemented method of claim 5 , wherein:

the classification associated with the resource comprises one of public, internal, confidential, or restricted.

8. One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to perform operations comprising:

generate a classifier based on a classifier algorithm;

train, using electronic training data, the classifier to classify data within a predetermined accuracy, wherein the operation to train the classifier to classify data within the predetermined accuracy further comprises:

train the classifier with training data;

classify, by the classifier, test data to generate classified test data;

determine that the test data is not within the predetermined accuracy based on a comparison of the classified test data with a pre-classified set of test data;

tune the classifier;

retrain the classifier with the training data;

reclassify, by the classifier, the test data to generate reclassified test data;

determine that the reclassified test data is within the predetermined accuracy based on a comparison of the reclassified test data with the pre-classified set of test data;

receive, via a network, a first event log from an agent that is configured to monitor data activities at a data storage, the first event log including information pertaining to a first access to data stored in the data storage by an electronic device associated with a user account;

cause the first event log to be stored in an event log database;

retrieve, from the event log database, a set of event logs associated with a user account, each event log of the set of event logs occurring in a time interval;

select the first event log from the set of event logs, the first event log associated with a resource having a classification assigned to the resource by the classifier based at least in part on a taxonomy;

determine a first context associated with the first event log;

determine a first activity risk associated with the first event log based at least in part on the first context;

determine a first risk indicator associated with the user account based at least in part on the first activity risk;

receive, via the network, a second event log from the agent during the time interval, the second event log including information pertaining to a second access to data stored in the data storage by the electronic device associated with the user account;

determine a second risk indicator associated with the user account based at least in part on the second event log; and

determine a cumulative risk indicator for the time interval based on the first risk indicator and the second risk indicator.

9. The one or more non-transitory computer-readable media of claim 8 , the operations further comprising:

select a third event log from the set of event logs;

determine a second context associated with the third event log;

determine a second activity risk associated with the third event log based at least in part on the second context; and

determine a third risk indicator associated with the user account based at least in part on the second activity risk.

10. The one or more non-transitory computer-readable media of claim 9 , the operations further comprising:

display, for the time interval, a graph that includes a plurality of risk indicators including at least the first risk indicator, the second risk indicator, and the third risk indicator.

11. The one or more non-transitory computer-readable media of claim 9 , the operations further comprising:

determine a first cumulative risk indicator associated with the user account during a first time interval based at least in part on the first risk indicator and the second risk indicator;

determine a second cumulative risk indicator associated with the user account during a second time interval based at least in part on a second set of event logs; and

display a graph that includes a plurality of cumulative risk indicators including at least the first cumulative risk indicator and the second cumulative risk indicator.

12. The one or more non-transitory computer-readable media of claim 8 , the operations further comprising:

determine the resource identified in the first event log as being accessed by an activity;

determine the classification associated with the resource; and

associate a resource risk associated with the resource based at least in part on the classification.

13. The one or more non-transitory computer-readable media of claim 8 , determining the first activity risk associated with the first event log based at least in part on the first context comprises:

determine the first activity risk based at least in part on the first context and a resource risk associated with the resource, wherein the classification associated with the resource comprises one of public, internal, confidential, or restricted.

14. A server, comprising:

one or more processors; and

one or more non-transitory computer-readable media storing instructions that are executable by the one or more processors to perform operations comprising:

generate a classifier based on a classifier algorithm;

train, using electronic training data, the classifier to classify data within a predetermined accuracy, wherein the operation to train the classifier to classify data within the predetermined accuracy further comprises:

train the classifier with training data;

classify, by the classifier, test data to generate classified test data;

determine that the test data is not within the predetermined accuracy based on a comparison of the classified test data with a pre-classified set of test data;

tune the classifier;

retrain the classifier with the training data;

reclassify, by the classifier, the test data to generate reclassified test data;

determine that the reclassified test data is within the predetermined accuracy based on a comparison of the reclassified test data with the pre-classified set of test data;

receive, via a network, a first event log from an agent that is configured to monitor data activities at a data storage, the first event log including information pertaining to a first access to data stored in the data storage by an electronic device associated with a user account;

cause the first event log to be stored in an event log database;

retrieve, from the event log database, a set of event logs associated with a user account, each event log of the set of event logs occurring in a time interval;

select the first event log from the set of event logs, the first event log associated with a resource having a classification assigned to the resource by the classifier based at least in part on a taxonomy;

determine a first context associated with the first event log;

determine a first activity risk associated with the first event log based at least in part on the first context;

determine a first risk indicator associated with the user account based at least in part on the first activity risk;

receive, via the network, a second event log from the agent during the time interval, the second event log including information pertaining to a second access to data stored in the data storage by the electronic device associated with the user account;

determine a second risk indicator associated with the user account based at least in part on the second event log; and

determine a cumulative risk indicator for the time interval based on the first risk indicator and the second risk indicator.

15. The server of claim 14 , the operations further comprising:

select a third event log from the set of event logs;

determine a second context associated with the third event log;

determine a second activity risk associated with the third event log based at least in part on the second context; and

determine a third risk indicator associated with the user account based at least in part on the second activity risk.

16. The server of claim 15 , the operations further comprising:

display, for the time interval, a graph that includes a plurality of risk indicators including at least the first risk indicator, the second risk indicator, and the third risk indicator.

17. The server of claim 15 , the operations further comprising:

determine a first cumulative risk indicator associated with the user account during a first time interval based at least in part on the first risk indicator and the second risk indicator;

determine a second cumulative risk indicator associated with the user account during a second time interval based at least in part on a second set of event logs; and

display a graph that includes a plurality of cumulative risk indicators including at least the first cumulative risk indicator and the second cumulative risk indicator.

18. The server of claim 14 , the operations further comprising:

determine the resource identified in the first event log as being accessed by an activity;

determine the classification associated with the resource; and

associate a resource risk associated with the resource based at least in part on the classification.

19. The server of claim 18 , wherein determining the first activity risk associated with the first event log based at least in part on the first context comprises:

determine the first activity risk based at least in part on the first context and the resource risk associated with the resource.

20. The server of claim 18 , wherein:

the classification associated with the resource comprises one of public, internal, confidential, or restricted.

Assignments (26)
RELEASE OF SECURITY INTEREST Recorded Nov 19, 2025
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; BINARYTREE.COM LLC; ERWIN, INC.
Reel/Frame 073606/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 18, 2025
From: GOLDMAN SACHS BANK USA, AS COLLATERAL AGENT
To: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; BINARYTREE.COM LLC; ERWIN, INC.
Reel/Frame 073613/0326 →
SECURITY INTEREST Recorded Jun 8, 2025
From: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; ERWIN, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 071527/0649 →
SECURITY INTEREST Recorded Jun 8, 2025
From: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; ERWIN, INC.
To: ALTER DOMUS (US) LLC
Reel/Frame 071527/0001 →
SECOND LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Feb 2, 2022
From: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; BINARYTREE.COM LLC; ERWIN, INC.; ONE IDENTITY LLC; ONELOGIN, INC.; ONE IDENTITY SOFTWARE INTERNATIONAL DESIGNATED ACTIVITY COMPANY
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 058952/0279 →
FIRST LIEN INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Feb 2, 2022
From: QUEST SOFTWARE INC.; ANALYTIX DATA SERVICES INC.; BINARYTREE.COM LLC; ERWIN, INC.; ONE IDENTITY LLC; ONELOGIN, INC.; ONE IDENTITY SOFTWARE INTERNATIONAL DESIGNATED ACTIVITY COMPANY
To: GOLDMAN SACHS BANK USA
Reel/Frame 058945/0778 →
RELEASE OF SECOND LIEN SECURITY INTEREST IN PATENTS Recorded Feb 2, 2022
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
To: QUEST SOFTWARE INC.
Reel/Frame 059096/0683 →
RELEASE OF FIRST LIEN SECURITY INTEREST IN PATENTS Recorded Feb 2, 2022
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
To: QUEST SOFTWARE INC.
Reel/Frame 059105/0479 →
CHANGE OF NAME Recorded Jun 19, 2018
From: DELL SOFTWARE INC.
To: QUEST SOFTWARE INC.
Reel/Frame 046393/0009 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Jun 7, 2018
From: QUEST SOFTWARE INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 046327/0347 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Jun 7, 2018
From: QUEST SOFTWARE INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 046327/0486 →
RELEASE OF FIRST LIEN SECURITY INTEREST IN PATENTS RECORDED AT R/F 040581/0850 Recorded May 22, 2018
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
To: QUEST SOFTWARE INC. (F/K/A DELL SOFTWARE INC.); AVENTAIL LLC
Reel/Frame 046211/0735 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE PREVIOUSLY RECORDED AT REEL: 040587 FRAME: 0624. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Nov 28, 2017
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: QUEST SOFTWARE INC. (F/K/A DELL SOFTWARE INC.); AVENTAIL LLC
Reel/Frame 044811/0598 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Nov 10, 2016
From: DELL SOFTWARE INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040587/0624 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Nov 9, 2016
From: DELL SOFTWARE INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040581/0850 →
RELEASE OF SECURITY INTEREST Recorded Oct 31, 2016
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: AVENTAIL LLC; DELL PRODUCTS, L.P.; DELL SOFTWARE INC.
Reel/Frame 040521/0467 →
RELEASE OF SECURITY INTEREST IN CERTAIN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040039/0642) Recorded Oct 31, 2016
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
To: AVENTAIL LLC; DELL PRODUCTS L.P.; DELL SOFTWARE INC.
Reel/Frame 040521/0016 →
RELEASE OF REEL 037848 FRAME 0210 (NOTE) Recorded Sep 14, 2016
From: BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040031/0725 →
SECURITY AGREEMENT Recorded Sep 14, 2016
From: AVENTAIL LLC; DELL PRODUCTS, L.P.; DELL SOFTWARE INC.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040030/0187 →
SECURITY AGREEMENT Recorded Sep 14, 2016
From: AVENTAIL LLC; DELL PRODUCTS L.P.; DELL SOFTWARE INC.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040039/0642 →
RELEASE OF REEL 037848 FRAME 0001 (TL) Recorded Sep 14, 2016
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040028/0152 →
RELEASE OF REEL 037847 FRAME 0843 (ABL) Recorded Sep 13, 2016
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
Reel/Frame 040017/0366 →
SUPPLEMENT TO PATENT SECURITY AGREEMENT (ABL) Recorded Feb 18, 2016
From: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 037847/0843 →
SUPPLEMENT TO PATENT SECURITY AGREEMENT (TERM LOAN) Recorded Feb 18, 2016
From: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037848/0001 →
SUPPLEMENT TO PATENT SECURITY AGREEMENT (NOTES) Recorded Feb 18, 2016
From: DELL SOFTWARE INC.; DELL PRODUCTS L.P.; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 037848/0210 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2015
From: SEIGEL, JAKE; MACINTOSH, ROBERT
To: DELL SOFTWARE, INC.
Reel/Frame 037186/0544 →
Continuity (1)
Related Publication 20170161503A1 · Jun 8, 2017
Cited By (1)
US 12,602,294