Anomalous user account detection systems and methods
Autoencoder-based anomaly detection methods have been used in identifying anomalous users from large-scale enterprise logs with the assumption that adversarial activities do not follow past habitual patterns. Most existing approaches typically build models by reconstructing single-day and individual-user behaviors. However, without capturing long-term signals and group-correlation signals, the models cannot identify low-signal yet long-lasting threats, and will incorrectly report many normal users as anomalies on busy days, which, in turn, leads to a high false positive rate. A method is provided based on compound behavior, which takes into consideration long-term patterns and group behaviors. The provided method leverages a novel behavior representation and an ensemble of deep autoencoders and produces an ordered investigation list.
1. A system for detecting anomalous cyber activity comprising: a memory; and a processor in communication with the memory, the processor configured to:
receive information on each of a plurality of cyber activities for each of a first plurality of user IDs, wherein the information is selected from: (1) a quantity of files locally opened by the respective user ID that the respective user ID had never locally opened before, (2) a quantity of files remotely opened by the respective user ID that the respective user ID had never remotely opened before, (3) a quantity of files written locally by the respective user ID that the respective user ID had never written locally before, (4) a quantity of files written remotely by the respective user ID that the respective user ID had never written remotely before, (5) a quantity of local files copied to a remote device that the respective user ID had never copied to a remote device before, and (6) a quantity of remote files copied to a local device that the respective user ID had never copied to a local device before;
determine, for each of the plurality of cyber activities of each of the first plurality of user IDs, a first deviation between the received information on a respective cyber activity for a respective user ID and previously stored information on the respective cyber activity for the respective user ID;
determine, for each of the plurality of cyber activities of each of the first plurality of user IDs, a second deviation between the received information on the respective cyber activity for the respective user ID and previously stored group information on the respective cyber activity for a group including a second plurality of user IDs;
determine, using at least one autoencoder, an anomaly score for each of the plurality of cyber activities of each of the first plurality of user IDs based on the determined first and second deviations for a respective cyber activity of a respective user ID; and
generate an ordered list of user IDs from the first plurality of user IDs based on the respective anomaly scores of each of the first plurality of user IDs such that the user ID with the most anomalous cyber activities is listed first in the ordered list.
2. The system of claim 1 , wherein the received information on each respective cyber activity for each respective user ID includes information on the respective cyber activity for the respective user ID over a first timeframe and information on the respective cyber activity for the respective user ID over a second timeframe.
3. The system of claim 2 , wherein the first timeframe corresponds to working hours of the first plurality of user IDs, and
wherein the second timeframe corresponds to non-working hours of the first plurality of user IDs.
4. The system of claim 2 , wherein the first and second timeframes are each less than twenty-four hours, and
wherein the received information on each respective cyber activity for each respective user ID includes the information over the first and second timeframes for a period of one or more days.
5. The system of claim 3 , wherein the period is equal to multiple days.
6. The system of claim 1 , wherein the previously stored group information for a respective cyber activity is an average of information on the respective cyber activity for each user ID in the group.
7. The system of claim 1 , wherein the first deviation D1 for a respective cyber activity of a respective user ID is calculated according to the below equations:
D
1
=
σ
1
,
t
,
d
+
σ
2
,
t
,
d
…
+
σ
f
,
t
,
d
σ
f
,
t
,
d
=
{
Δ
,
if
δ
f
,
t
,
d
≥
Δ
-
Δ
,
if
δ
f
,
t
,
d
≤
-
Δ
δ
f
,
t
,
d
,
otherwise
δ
f
,
t
,
d
=
m
f
,
t
,
d
-
mean
(
h
→
f
,
t
,
d
)
s
t
d
(
h
→
f
,
t
,
d
)
wherein of,t, d is a sigma for a respective user ID of a cyber activity f in a timeframe t, on a day d, 8f,t, d is a variance for a respective user ID of a cyber activity f in a timeframe t, on a day d,mf,t, d is numeric measurements for the respective user ID of the cyber activity fin the timeframe t, on the day d, mean(hf,t, d ) is a mean of hf,t, d ,
wherein hf,t, d is a vector of the previously stored information for the respective user ID of the cyber activity f in the timeframe t, on the day d, and std(hf,t, d ) is a standard deviation of hf,t, d .
8. The system of claim 7 , wherein the sigma of each respective cyber activity of a respective user ID is weighted with a weight, when determining the first deviation DI, according to the below equations:
D
1
=
ω
1
,
t
,
d
σ
1
,
t
,
d
+
ω
2
,
t
,
d
σ
2
,
t
,
d
…
+
ω
f
,
t
,
d
σ
f
,
t
,
d
w
f
,
t
,
d
=
1
log
2
(
max
(
s
t
d
(
h
→
f
,
t
,
d
)
,
2
)
)
wherein wf,t, d is a weight of a cyber activity f in a timeframe t, on a day d, and std(hf,t, d ) is a mean of hf,t, d ,
wherein hf,t, d is a vector of the previously stored information of the cyber activity f in the timeframe t, on the day d.
9. The system of claim 1 , wherein the second deviation D2 for a respective cyber activity of a respective user ID is calculated according to the below equation:
D
2
=
σ
1
,
t
,
d
+
σ
2
,
t
,
d
…
+
σ
f
,
t
,
d
σ
f
,
t
,
d
=
{
Δ
,
if
δ
f
,
t
,
d
≥
Δ
-
Δ
,
if
δ
f
,
t
,
d
≤
-
Δ
δ
f
,
t
,
d
,
otherwise
δ
f
,
t
,
d
=
m
f
,
t
,
d
-
mean
(
g
→
f
,
t
,
d
)
s
t
d
(
g
→
f
,
t
,
d
)
wherein of,t, d is a sigma for a respective user ID of a cyber activity f in a timeframe t, on a day d, 6 f,t, d is a variance for a respective user ID of a cyber activity f in a timeframe t, on a day d,mf,t, d is numeric measurements for the respective user ID of the cyber activity f in the timeframe t, on the day d, mean(ft, d ) is a mean of 4 ftd,
wherein gf,t, d is a vector of the previously stored information on the group for the cyber activity f in the timeframe t, on the day d, and std(′ft, d ) is a standard deviation of ′ft, d .
10. The system of claim 1 , wherein the information on a respective cyber activity for a respective user ID includes: (1) a quantity of connections of an external storage device by the respective user ID to a device in communication with the processor, and (2) a quantity of connections of an external storage device by the respective user ID to a device that the respective user ID had never connected to before.
11. The system of claim 1 , wherein the information on a respective cyber activity for a respective user ID includes a quantity of file uploads to a domain address that the respective user ID had never before uploaded to the domain address.
12. A method for detecting anomalous cyber activity comprising:
receiving information on each of a plurality of cyber activities for each of a first plurality of user IDs, wherein the information is selected from: (1) a quantity of files locally opened by the respective user ID that the respective user ID had never locally opened before, (2) a quantity of files remotely opened by the respective user ID that the respective user ID had never remotely opened before, (3) a quantity of files written locally by the respective user ID that the respective user ID had never written locally before, (4) a quantity of files written remotely by the respective user ID that the respective user ID had never written remotely before, (5) a quantity of local files copied to a remote device that the respective user ID had never copied to a remote device before, and (6) a quantity of remote files copied to a local device that the respective user ID had never copied to a local device before;
determining, for each of the plurality of cyber activities of each of the first plurality of user IDs, a first deviation between the received information on a respective cyber activity for a respective user ID and previously stored information on the respective cyber activity for the respective user ID;
determining, for each of the plurality of cyber activities of each of the first plurality of user IDs, a second deviation between the received information on the respective cyber activity for the respective user ID and previously stored group information on the respective cyber activity for a group including a second plurality of user IDs;
determining, using at least one autoencoder, an anomaly score for each of the plurality of cyber activities of each of the first plurality of user IDs based on the determined first and second deviations for a respective cyber activity of a respective user ID; and
generating an ordered list of user IDs from the first plurality of user IDs based on the respective anomaly scores of each of the first plurality of user IDs such that the user ID with the most anomalous cyber activities is listed first in the ordered list.
13. The system of claim 12 , wherein the information on a respective cyber activity for a respective user ID includes: (1) a quantity of document file uploads to a domain address that the respective user ID had never before uploaded to the domain address, (2) a quantity of .exe file uploads to a domain address that the respective user ID had never before uploaded to the domain address, (3) a quantity of .jpg file uploads to a domain address that the respective user ID had never before uploaded to the domain address, (4) a quantity of .pdf file uploads to a domain address that the respective user ID had never before uploaded to the domain address, (5) a quantity of .txt file uploads to a domain address that the respective user ID had never before uploaded to the domain address, (6) a quantity of .zip file uploads to a domain address that the respective user ID had never before uploaded to the domain address, and (7) a quantity of new http requests that the respective user ID had never before requested.
14. The method of claim 12 , wherein the ordered list is generated based on the N-th highest rank of the respective user IDs in each of the plurality of cyber activities.
15. The method of claim 12 , wherein the first plurality of user IDs include the second plurality of user IDs.
16. The method of claim 12 , wherein the first plurality of user IDs consist of the second plurality of user IDs.
17. The method of claim 12 , further comprising displaying the generated ordered list on a display.
18. A non-transitory, computer-readable medium storing instructions, which when executed by a processor, cause the processor to:
receive information on each of a plurality of cyber activities for each of a first plurality of user IDs, wherein the information is selected from: (1) a quantity of files locally opened by the respective user ID that the respective user ID had never locally opened before, (2) a quantity of files remotely opened by the respective user ID that the respective user ID had never remotely opened before, (3) a quantity of files written locally by the respective user ID that the respective user ID had never written locally before, (4) a quantity of files written remotely by the respective user ID that the respective user ID had never written remotely before, (5) a quantity of local files copied to a remote device that the respective user ID had never copied to a remote device before, and (6) a quantity of remote files copied to a local device that the respective user ID had never copied to a local device before;
determine, for each of the plurality of cyber activities of each of the first plurality of user IDs, a first deviation between the received information on a respective cyber activity for a respective user ID and previously stored information on the respective cyber activity for the respective user ID;
determine, for each of the plurality of cyber activities of each of the first plurality of user IDs, a second deviation between the received information on the respective cyber activity for the respective user ID and previously stored group information on the respective cyber activity for a group including a second plurality of user IDs;
determine, using at least one autoencoder, an anomaly score for each of the plurality of cyber activities of each of the first plurality of user IDs based on the determined first and second deviations for a respective cyber activity of a respective user ID; and
generate an ordered list of user IDs from the first plurality of user IDs based on the respective anomaly scores of each of the first plurality of user IDs such that the user ID with the most anomalous cyber activities is listed first in the ordered list.
19. The non-transitory, computer-readable medium of claim 18 , wherein the information on a respective cyber activity for a respective user ID includes information on connections of an external storage device by the respective user ID to a device in communication with the processor, information on files accessed by the respective user ID, and information on files uploaded by the respective user ID to a domain address.