Methods and apparatus to identify malicious activity in a network
Example network monitoring methods disclosed herein include iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity. For example, the iterative adjusting is to (1) reduce a first distance calculated between a first pair of reference devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device. Disclosed example network monitoring methods also include determining whether a second unclassified device is associated with malicious network activity based on the output set of weights.
1. A network monitor comprising:
memory including computer readable instructions; and
a processor to execute the computer readable instructions to perform operations including:
iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and
determining whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.
2. The network monitor of claim 1 , wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.
3. The network monitor of claim 2 , wherein the operations further include:
processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and
processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.
4. The network monitor of claim 1 , wherein the iterative adjusting further includes:
evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and
evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.
5. The network monitor of claim 4 , wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.
6. The network monitor of claim 1 , wherein the determining of whether the second unclassified device is associated with malicious network activity includes:
determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and
comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.
7. A network monitoring method comprising:
iteratively adjusting, by executing an instruction with a processor, respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and
determining, by executing an instruction with the processor, whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.
8. The method of claim 7 , wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.
9. The method of claim 8 , further including:
processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and
processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.
10. The method of claim 7 , wherein the iterative adjusting further includes:
evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and
evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.
11. The method of claim 10 , wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.
12. The method of claim 7 , wherein the determining of whether the second unclassified device is associated with malicious network activity includes:
determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and
comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.
13. The method of claim 7 , further including selecting the pair of the reference devices and the first unclassified device based on a random number generator.
14. A tangible computer readable storage medium including computer readable instructions which, when executed, cause a processor to perform operations comprising:
iteratively adjusting respective weights assigned to respective types of network activity features for devices monitored in a network, the iterative adjusting to determine an output set of weights corresponding to ones of the types of network activity features indicative of malicious network activity, the iterative adjusting to (1) reduce a first distance calculated between a first pair of reference devices selected from a first set of the devices previously classified as being associated with malicious network activity, and (2) increase a second distance calculated between a first one of the pair of the reference devices and a first unclassified device selected from a second set of the devices that are unclassified; and
determining whether a second unclassified device selected from the second set of the devices is associated with malicious network activity based on the output set of weights.
15. The tangible computer readable storage medium of claim 14 , wherein the types of network activity features include a first tier of network activity features generated from network log records obtained for the devices monitored in the network, a second tier of network activity features generated from the first tier of network activity features and the network log records, and a third tier of features generated from the second tier of network activity features.
16. The tangible computer readable storage medium of claim 15 , wherein the operations further include:
processing the network log records to determine a first set of network activity features for a first one of the devices monitored in the network, the first set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features; and
processing the network log records to determine a second set of network activity features for a second one of the devices monitored in the network, the second set of network activity features including respective values for the first tier of network activity features, the second tier of network activity features and the third tier of network activity features.
17. The tangible computer readable storage medium of claim 14 , wherein the iterative adjusting further includes:
evaluating a squared weighted Euclidean distance function based on the weights, a first set of network activity features determined for the first one of the pair of the reference devices and a second set of network activity features determined for the second one of the pair of the reference devices to calculate the first distance; and
evaluating the squared weighted Euclidean distance function based on the weights, the first set of network activity features determined for the first one of the pair of the reference device and a third set of network activity features determined for the first unclassified device to calculate the second distance.
18. The tangible computer readable storage medium of claim 17 , wherein the iterative adjusting of the respective weights includes adjusting the respective weights according to a stochastic gradient descent algorithm.
19. The tangible computer readable storage medium of claim 14 , wherein the determining of whether the second unclassified device is associated with malicious network activity includes:
determining a third distance based on the output set of weights and a respective set of network activity features determined for the second unclassified device; and
comparing the third distance to a threshold to determine whether the second unclassified device is associated with malicious network activity.
20. The tangible computer readable storage medium of claim 14 , wherein the operations further include selecting the pair of the reference devices and the first unclassified device based on a random number generator.