Systems and methods for zero-footprint large-scale user-entity behavior modeling

1. A method for reducing storage space used in tracking behavior of a plurality of network endpoints by modeling the behavior with a behavior model, the method comprising:

receiving a plurality of records, each respective record of the plurality of records corresponding to a respective network endpoint of the plurality of network endpoints;

determining the respective network endpoint, of the plurality of network endpoints, to which each respective record of the plurality of records corresponds;

assigning a respective dedicated queue for each respective network endpoint;

transmitting to each respective dedicated queue, each record of the plurality of records that corresponds to the respective network endpoint to which the respective dedicated queue is assigned;

generating, for each respective network endpoint, using each record of the respective dedicated queue originating from the respective network endpoint, a respective vector representing a respective behavior model, wherein the generating the respective vector further comprises:

identifying a module of a plurality of modules that is idle, wherein the plurality of modules are programmed to generate the respective vectors representing the respective behavior models;

commanding the idle module to generate the respective vector representing the respective behavior model by:

encoding data of each respective record within the respective dedicated queue as a floating point value in the respective vector, wherein the encoding the data further comprises extracting the data from a field of the respective record, and concatenating the data into a string; and

feeding the string into a Document to Vector (doc2vec) algorithm, thereby

outputting the respective vector;

storing each respective vector to a memory; and

determining an anomalous behavior state for a network endpoint in the plurality of network endpoints by comparing the respective vector of the network endpoint to a normalcy threshold in a multidimensional space,

wherein the plurality of records is of a first data size, wherein a sum of a data size of each respective behavior model is of a second data size, and wherein the second data size is two or more orders of magnitude smaller than the first data size.

2. The method of claim 1 , wherein each respective record identifies a respective single network flow originating from the respective network endpoint that corresponds to the respective record.

3. The method of claim 1 , wherein the tracking is performed using a Kalman filter.

4. The method of claim 1 , further comprising:

tracking behavior of the network endpoint in the plurality of network endpoints by deriving a multivariate Gaussian distribution to determine a current position of the respective vector of the network endpoint in the multidimensional space.

5. The method of claim 1 , wherein the generating further comprises:

forming a document from the string; and

wherein the feeding the string into doc2vec algorithm further comprises analyzing, using the doc2vec algorithm, the document using a shallow neural network.

6. A system for reducing storage space used in tracking behavior of a plurality of network endpoints by modeling the behavior with a behavior model, the system comprising:

storage circuitry;

communications circuitry; and

control circuitry configured to:

receive, by the communications circuitry, a plurality of records, each respective record of the plurality of records corresponding to a respective network endpoint of the plurality of network endpoints;

determine the respective network endpoint, of the plurality of network endpoints, to which each respective record of the plurality of records corresponds;

assign a respective dedicated queue for each respective network endpoint;

transmit, to each respective dedicated queue, each record of the plurality of records that corresponds to the respective network endpoint to which the respective dedicated queue is assigned;

generate, for each respective network endpoint, using each record of the respective dedicated queue corresponding to the respective network endpoint, a respective vector representing a respective behavior model, wherein to generate the respective vector, the control circuitry is further configured to:

identify one or more modules of a plurality of modules that are idle, wherein the plurality of modules are programmed to generate the respective vectors representing the respective behavior models; and

command an idle module of the one or more identified idle modules to generate the respective vector representing the respective behavior model by:

encoding data of each respective record within the respective dedicated queue as a floating point value in the respective vector, wherein to encode the data the control circuitry is further configured to extract the data from a field of the respective record, and concatenate the data into a string; and

feeding the string into a Document to Vector (doc 2 vec) algorithm, thereby outputting the respective vector;

store, by the storage circuitry, each respective vector in a memory;

determine an anomalous behavior state for a network endpoint in the plurality of network endpoints by determining a current position of the respective vector of the network endpoint is in a region of a multidimensional space, the region having a probability value less than a threshold value; and

track behavior of the network endpoint over time by comparing a current position of the respective vector of the network endpoint in the multidimensional space to a previous position of a previous version of the respective vector of the network endpoint in the multidimensional space,

wherein the plurality of records is of a first data size, wherein a sum of a data size of each respective behavior model is of a second data size, and wherein the second data size is two or more orders of magnitude smaller than the first data size.

7. The system of claim 6 , wherein the control circuitry is further configured to:

in response to determining the anomalous behavior state for the network endpoint, alert a network administrator.

8. The system of claim 6 , wherein each respective record identifies a respective single network flow originating from the respective network endpoint that corresponds to the respective record.

9. The system of claim 6 , wherein the control circuitry is further configured to track the behavior of each respective network endpoint over time using a Kalman filter.

10. The system of claim 6 , wherein the control circuitry is further configured to:

track behavior of the network endpoint in the plurality of network endpoints by deriving a multivariate Gaussian distribution to determine the current position of the respective vector of the network endpoint in the multidimensional space.

11. The system of claim 6 , wherein to generate the respective vector, the control circuity is further configured to:

form a document from the string; and

wherein to feed the string into doc2vec algorithm, the control circuitry is further configured to analyze, using the doc2vec algorithm, the document using a shallow neural network.

12. The system of claim 6 , wherein to generate the respective vectors representing the respective behavior models, the control circuitry is further configured to:

determine first dedicated queues each having a number of records that exceed a threshold value; and

assign a first portion of the identified idle modules to the first dedicated queues.

13. The system of claim 12 , wherein to generate the respective vectors representing the respective behavior models, the control circuitry is further configured to:

assign a second portion of the identified idle modules to second dedicated queues based on a load balancing scheme.

14. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a plurality of records, each respective record of the plurality of records corresponding to a respective network endpoint of a plurality of network endpoints;

determining the respective network endpoint, of the plurality of network endpoints, to which each respective record of the plurality of records corresponds;

assigning a respective dedicated queue for each respective network endpoint;

transmitting, to each respective dedicated queue, each record of the plurality of records that corresponds to the respective network endpoint to which the respective dedicated queue is assigned;

generating, for each respective network endpoint, using each record of the respective dedicated queue corresponding to the respective network endpoint, a respective vector representing a respective behavior model, wherein the generating the respective vector further comprises:

identifying one or more modules of a plurality of modules that are idle, wherein the plurality of modules are programmed to generate the respective vectors representing the respective behavior models; and

commanding an idle module of the one or more identified idle modules to generate the respective vector representing the respective behavior model by:

encoding data of each respective record within the respective dedicated queue as a floating point value in the respective vector, wherein the encoding the data further comprises extracting the data from a field of the respective record, and concatenating the data into a string; and

feeding the string into a Document to Vector (doc2vec) algorithm, thereby outputting the respective vector;

storing each respective vector to a memory; and

determining an anomalous behavior state for a network endpoint in the plurality of network endpoints by comparing the respective vector of the network endpoint to a normalcy threshold in a multidimensional space,

wherein the plurality of records is of a first data size, wherein a sum of a data size of each respective behavior model is of a second data size, and wherein the second data size is smaller than the first data size.

15. The non-transitory computer-readable medium of claim 14 , wherein the generating the respective vectors representing the respective behavior models further comprises:

determining first dedicated queues each having a number of records that exceed a threshold value; and assigning a first portion of the identified idle modules to the first dedicated queues.

16. The non-transitory computer-readable medium of claim 15 , wherein the generating the respective vectors representing the respective behavior models further comprises:

assigning a second portion of the identified idle modules to second dedicated queues based on a load balancing scheme.

17. The non-transitory computer-readable medium of claim 15 , wherein the generating the respective vectors representing the respective behavior models further comprises:

randomly assigning a second portion of the identified idle modules to second dedicated queues.

18. The non-transitory computer-readable medium of claim 14 , wherein the tracking is performed using a Kalman filter.

19. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise:

tracking behavior of the network endpoint in the plurality of network endpoints by deriving a multivariate Gaussian distribution to determine a current position of the respective vector of the network endpoint in the multidimensional space.

20. The non-transitory computer-readable medium of claim 14 , wherein the generating further comprises:

forming a document from the string; and

wherein the feeding the string into doc2vec algorithm further comprises analyzing, using the doc2vec algorithm, the document using a shallow neural network.