Customizable load balancing in a user behavior analytics deployment
A deployment manager executing in a distributed computing environment generates a user behavior analytics (UBA) deployment to process structured event data. The deployment manager configures a streaming cluster to perform streaming processing on real-time data and configures a batch cluster to perform batch processing on aggregated data. A configuration manager executing in the distributed computing environment interoperates with the deployment manager to update the UBA deployment with user-provided code and configurations that define streaming and batch models, among other things. In this manner, the deployment manager provides a scalable UBA deployment that can be customized, via the configuration manager, by a user.
1. A computer-implemented method, comprising:
generating a container image based on both of (i) a default container image defined in a deployment configuration and (ii) a batch model defined in a content package received from a client;
instantiating, according to a schedule defined in the deployment configuration, a container based on the container image;
identifying, based on the deployment configuration, event data stored in one or more data stores; and
distributing the event data to the container for batch processing via execution of the batch model.
2. The computer-implemented method of claim 1 , wherein the execution of the batch model identifies one or more anomalies in the event data.
3. The computer-implemented method of claim 1 , wherein the container is co-located within the one or more data stores that store the event data.
4. The computer-implemented method of claim 1 , wherein the deployment configuration specifies one or more tables within the one or more data stores, and the event data is included within the one or more tables.
5. The computer-implemented method of claim 1 , wherein the schedule specifies when the event data is to be processed via execution of the batch model.
6. The computer-implemented method of claim 1 , further comprising destroying the container after the batch processing is complete.
7. The computer-implemented method of claim 1 , wherein a first entity causes at least a portion of a first subset of event data to be generated, and a second entity causes at least a portion of a second subset of event data to be generated.
8. The computer-implemented method of claim 1 , further comprising:
receiving raw event data from a data source, wherein the raw event data includes machine data indexed with timestamps;
queueing the raw event data in a load balancer;
initiating execution of a second container that is defined in the deployment configuration; and
configuring the second container to process the raw event data to generate the event data.
9. The computer-implemented method of claim 1 , further comprising initiating execution of a second container to perform at least one of an extract-transform-load operation, a tokenization operation, a field value extraction operation, and an indexing operation to process the event data.
10. The computer-implemented method of claim 1 , wherein the event data comprises a plurality of time-stamped, searchable events, each event in the plurality of time-stamped, searchable events including a portion of raw machine data reflecting activity in an information technology environment.
11. The computer-implemented method of claim 1 , wherein instantiating the container comprises causing a first virtual machine to execute an instance of the container image.
12. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
generating a container image based on both of (i) a default container image defined in a deployment configuration and (ii) a batch model defined in a content package received from a client;
instantiating, according to a schedule defined in the deployment configuration, a container based on the container image;
identifying, based on the deployment configuration, event data stored in one or more data stores; and
distributing the event data to the container for batch processing via execution of the batch model.
13. The one or more non-transitory computer-readable media of claim 12 , wherein the execution of the batch model identifies one or more anomalies in the event data.
14. The one or more non-transitory computer-readable media of claim 12 , wherein the container is co-located within the one or more data stores that store the event data.
15. The one or more non-transitory computer-readable media of claim 12 , wherein the deployment configuration specifies one or more tables within the one or more data stores, and the event data is included within the one or more tables.
16. The one or more non-transitory computer-readable media of claim 12 , wherein the schedule specifies when the event data is to be processed via execution of the batch model.
17. The one or more non-transitory computer-readable media of claim 12 , further comprising destroying the container after the batch processing is complete.
18. The one or more non-transitory computer-readable media of claim 12 , further comprising:
receiving raw event data from a data source, wherein the raw event data includes machine data indexed with timestamps;
queueing the raw event data in a load balancer;
initiating execution of a second container that is defined in the deployment configuration; and
configuring the second container to process the raw event data to generate the event data.
19. A system, comprising:
a memory that includes instructions; and
a processor that is coupled to the memory and performs the steps of:
generating a container image based on a default container image defined in a deployment configuration and a batch model defined in a content package received from a client;
generating a container image based on both of (i) a default container image defined in a deployment configuration and (ii) a batch model defined in a content package received from a client;
instantiating, according to a schedule defined in the deployment configuration, a container based on the container image;
identifying, based on the deployment configuration, event data stored in one or more data stores; and
distributing the event data to the container for batch processing via execution of the batch model.
20. The system of claim 19 , wherein the schedule specifies when the event data is to be processed via execution of the batch model.