Capacity tracking and forecast modeling across multiple platforms
Systems and methods track capacity and model forecasts across multiple platforms. A flow of data traffic from a first device to a second device is monitored. A future flow of data traffic is forecast based on the monitored flow, the future flow of data traffic including additional data traffic in addition to the monitored flow of data traffic. A threshold is anticipated to be reached in the forecasted future flow of data traffic. A corrective action is determined to mitigate the anticipated threshold being reached and then implemented.
1 . A computer-implemented method comprising:
monitoring a flow of data traffic from a first device comprising a data source to a second device comprising a processing environment;
forecasting a future flow of data traffic based on the monitored flow by mapping the monitored flow of the monitored data traffic to a communication path between the first device and the second device,
the communication path comprising a first component and a second component communicatively coupled to the first device and the second device, the first component and the second component each having a mapped usage to process the data traffic in the monitored flow of the data traffic between the first device and the second device,
each of the first component and the second component comprising an application programming interface (API), a firewall, a load balancer, an application container, a shared service, a database, a gateway, a virtual machine, or a dependent system,
the future flow of data traffic including additional data traffic in addition to the monitored flow of data traffic, and forecasting the future flow of data traffic further comprising using the mapping to model a future capacity usage of the first component and the second component;
based on the modeled future capacity usage of the first component and the second component, anticipating that a threshold is to be reached by at least one of the first component or the second component in the forecasted future flow of data traffic;
determining a corrective action to mitigate the anticipated threshold being reached by at least one of the first component or the second component;
causing the determined corrective action to be implemented, the corrective action proactively preventing the anticipated threshold from being reached by at least one of the first component or the second component; and
based on implementing the determined corrective action, stabilizing the first device and the second device with respect to the flow of data traffic during a surge in the data traffic.
2 . The computer-implemented method of claim 1 , wherein the corrective action includes one or more of:
adding additional capacity to at least one of the first component or the second component, the adding comprising scaling an available capacity of the first component or the second component,
adjusting the flow of data traffic,
limiting a rate of the flow of data traffic, and
pausing the flow of data traffic.
3 . The computer-implemented method of claim 1 , wherein the threshold includes a first threshold indicating a warning threshold level and a second threshold indicating a critical threshold level, wherein the critical threshold level is a higher threshold level than the warning threshold level.
4 . The computer-implemented method of claim 3 , wherein:
the critical threshold level is an indication that a resource usage of a resource associated with the first component or the second component exceeds a peak capacity for the first component or the second component in the forecasted future flow of data traffic, the resource comprising a processor, a memory, a network bandwidth, a computing processing resource, or a processing resource; and
the warning threshold level is an indication that the resource usage of the resource associated with the first component or the second component reaches a predetermined percentage of the peak capacity for the first component or the second component in the forecasted future flow of data traffic.
5 . The computer-implemented method of claim 3 , wherein:
anticipating that the first threshold is to be reached further comprises anticipating that a resource usage for the first component or the second component in the flow of data traffic, based on the forecasted future flow of data traffic, reaches a predetermined percentage of peak capacity for the first component or the second component; and
anticipating that the second threshold is to be reached further comprises anticipating that the resource usage for the first component or the second component in the flow of data traffic, based on the forecasted future flow of data traffic, exceeds the peak capacity for the first component or the second component.
6 . The computer-implemented method of claim 5 , wherein causing the corrective action to be implemented comprises:
causing a first corrective action to be implemented based on the anticipation that the first threshold is to be reached in the forecasted future flow of data traffic by the first component or the second component;
causing a second corrective action to be implemented based on the anticipation that the second threshold is to be reached in the forecasted future flow of data traffic by the first component or the second component; and
wherein the first corrective action is different than the second corrective action.
7 . The computer-implemented method of claim 1 , wherein the flow of data traffic includes traversing the application container on the second device.
8 . The computer-implemented method of claim 1 , wherein the corrective action is implemented prior to the threshold being reached by the first component or the second component.
9 . The computer-implemented method of claim 1 , wherein monitoring the flow of data traffic further comprises identifying a subtype usage involved in or frequency of units traversed in the monitored flow of data traffic and forecasting the future flow of the data traffic is further based on the identified subtype usage or the identified frequency of units.
10 . The computer-implemented method of claim 1 , wherein monitoring the flow of data traffic further comprises identifying a customer identifier associated with a received request that initiates the monitored flow of data traffic.
11 . The computer-implemented method of claim 1 , wherein monitoring the flow of data traffic further comprises identifying a downstream request that is processed in the flow of data traffic in response to a request received by the first device or the second device.
12 . The computer-implemented method of claim 1 , further comprising:
defining a measurement of a workload based on the monitored flow of data traffic; and
the determined corrective action being an optimal corrective action, the optimal corrective action determined to provide a greatest net benefit to the flow of data traffic.
13 . The computer-implemented method of claim 1 , further comprising:
defining an interface that maps the flow of data traffic; and
forecasting the future flow of the data traffic further comprising extrapolating the monitored flow of data traffic through the flow of data traffic mapped by the defined interface.
14 . The computer-implemented method of claim 1 , further comprising:
receiving, by the second device a query from the first device;
responding, by the second device, to the query from the first device, the responding initiating the monitored flow of data traffic; and
forecasting the future flow of data traffic for the received query based on the monitored flow.
15 . The computer-implemented method of claim 1 , further comprising:
a capacity of the first component or the second component having a scaling limit; and
determining the corrective action being based in part on the scaling limit of the first component or the second component.
16 . A system comprising:
a processor;
a memory communicatively coupled to the processor; and
a capacity modeler, implemented on the processor, that:
monitors a flow of data traffic from a first device comprising a data source to a second device comprising a processing environment;
forecasts a future flow of data traffic based on the monitored flow, by mapping the monitored flow of the monitored data traffic to a communication path between the first device and the second device,
the communication path comprising a first component and a second component communicatively coupled to the first device and the second device, the first component and the second component each having a mapped usage to process the data traffic in the monitored flow of the data traffic between the first device and the second device,
each of the first component or the second component comprising an application programming interface (API), a firewall, a load balancer, an application container, a shared service, a database, a gateway, a virtual machine, or a dependent system,
the future flow of data traffic including additional data traffic in addition to the monitored flow of data traffic, and forecasting the future flow of data traffic further comprising using the mapping to model a future capacity usage of the first component or the second component;
based on the modeled future capacity usage of the first component or the second component, anticipates that a threshold is to be reached by the first component or the second component in the forecasted future flow of data traffic;
determines a corrective action to mitigate the anticipated threshold being reached by the first component or the second component;
initiates the determined corrective action, the corrective action proactively preventing the anticipated threshold from being reached; and
based on initiating the determined corrective action, stabilizing the first device and the second device with respect to the flow of data traffic during a surge in the data traffic.
17 . The system of claim 16 ,
the threshold further comprising a first threshold indicating a warning threshold level and a second threshold indicating a critical threshold level,
the critical threshold level being a higher threshold level than the warning threshold level.
18 . The system of claim 17 , wherein:
to anticipate that the first threshold is to be reached, the capacity modeler further anticipates that a resource usage for the first component or the second component in the flow of data traffic, based on the forecasted future flow of data traffic, reaches a predetermined percentage of peak capacity for the first component or the second component; and
to anticipate that the second threshold is to be reached, the capacity modeler further anticipates that the resource usage for the first component or the second component in the flow of data traffic, based on the forecasted future flow of data traffic, exceeds the peak capacity for the first component or the second component.
19 . The system of claim 18 , wherein, to initiate the corrective action, the capacity modeler further:
identifies a first corrective action based on the anticipation that the first threshold is to be reached in the forecasted future flow of data traffic by the first component or the second component;
identifies a second corrective action based on the anticipation that the second threshold is to be reached in the forecasted future flow of data traffic by the first component or the second component; and
wherein the first corrective action is different than the second corrective action.
20 . A non-transitory computer-storage medium storing instructions that, when executed by a processor, cause the processor to:
monitor a flow of data traffic from a first device comprising a data source to a second device comprising a processing environment;
forecast a future flow of data traffic based on the monitored flow by mapping the monitored flow of the monitored data traffic to a communication path between the first device and the second device,
the communication path comprising a first component and a second component communicatively coupled to the first device and the second device, each of the first component and the second component having a mapped usage to process the data traffic in the monitored flow of the data traffic between the first device and the second device,
the first component or the second component comprising an application programming interface (API), a firewall, a load balancer, an application container, a shared service, a database, a gateway, a virtual machine, or a dependent system,
the future flow of data traffic including additional data traffic in addition to the monitored flow of data traffic, and forecasting the future flow of data traffic further comprising using the mapping to model future capacity usage of the first component or the second component;
based on the modeled future capacity usage of the first component or the second component, anticipate that a threshold is to be reached by the first component or the second component in the forecasted future flow of data traffic;
determine a corrective action to mitigate the anticipated threshold being reached by the first component or the second component;
initiate the determined corrective action, the corrective action proactively preventing the anticipated threshold from being reached; and
based on initiating the determined corrective action, stabilizing the first device and the second device with respect to the flow of data traffic during a surge in the data traffic.