Load shedding in a distributed system
A load shedding system provides improved fault tolerance and resilience in message communications. The requesting service application may be configured to send data request(s) to a responding service application. A load shedding manager is programmed or configured to receive the data request(s) and determine, based on one or more configurable criteria and status information whether to allow the data request(s) to proceed or not. The criteria for making the determination may include various configurable settings, including error rate time window, and threshold values. The status information may include various information that describes the performance or health of the responding service, including, but not limited to: error rate for the responding service application's processing of incoming requests, latency of communication between the requesting service application and the responding service application, number of concurrent operations or requests being processed by the responding service application, CPU usage of the responding service application, disk usage of the responding service application, and other similar status information.
1. A method, comprising:
receiving an incoming data request intended to be sent to a service application for further processing;
receiving status information that describes performance of the service application;
using two or more rules to determine, based on the status information, whether to allow the incoming data request to be sent to the service application for further processing;
wherein a first rule of the two or more rules comprises:
generating a random number; and
comparing the status information regarding the performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing;
wherein a second rule of the two or more rules comprises:
counting a total number of prior incoming data requests for a time window having a duration of a time window value;
comparing the total number of prior incoming data requests to a minimum request count threshold to determine whether to allow the incoming data request to be sent to the service application for further processing;
wherein the method is performed using one or more processors.
2. The method of claim 1 , wherein the two or more rules comprises at least a third rule, wherein the third rule comprises:
determining, using the status information, an error rate for the time window;
comparing the error rate for the time window to a lower error rate threshold to determine whether to allow the incoming data request to be sent to the service application for further processing;
comparing the error rate for the time window to an upper error rate threshold to determine whether to allow the incoming data request to be sent to the service application for further processing.
3. The method of claim 2 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing the error rate for the time window to the random number.
4. The method of claim 1 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing central processing unit (CPU) usage of the service application to the random number.
5. The method of claim 1 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing disk usage of the service application to the random number.
6. The method of claim 1 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing network latency of the service application to the random number.
7. The method of claim 1 , further comprising:
logging a log entry for the incoming data request comprising the status information.
8. The method of claim 1 , wherein:
the two or more rules comprises at least a third rule, wherein the second rule comprises applying a step function to the status information and determining whether to allow the incoming data request to be sent to the service application is based on a result of the step function;
the status information comprises status information received over a period of time.
9. The method of claim 1 , wherein:
the two or more rules comprises at least a third rule, wherein the second rule comprises applying a linear function to the status information and determining whether to allow the incoming data request to be sent to the service application is based on a result of the linear function;
the status information comprises status information received over a period of time.
10. One or more non-transitory computer-readable storage media storing instructions, which when executed by one or more processors, cause:
receiving an incoming data request intended to be sent to a service application for further processing;
receiving status information that describes performance of the service application;
using two or more rules to determine, based on the status information, whether to allow the incoming data request to be sent to the service application for further processing;
wherein a first rule of the two or more rules comprises:
generating a random number; and
comparing the status information regarding the performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing;
wherein a second rule of the two or more rules comprises:
counting a total number of prior incoming data requests for a time window having a duration of a time window value;
comparing the total number of prior incoming data requests to a minimum request count threshold to determine whether to allow the incoming data request to be sent to the service application for further processing.
11. The one or more non-transitory computer-readable storage media of claim 10 , wherein the two or more rules comprises at least a third rule, wherein the third rule comprises:
determining, using the status information, an error rate for the time window;
comparing the error rate for the time window to a lower error rate threshold to determine whether to allow the incoming data request to be sent to the service application for further processing;
comparing the error rate for the time window to an upper error rate threshold to determine whether to allow the incoming data request to be sent to the service application for further processing.
12. The one or more non-transitory computer-readable storage media of claim 11 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing the error rate for the time window to the random number.
13. The one or more non-transitory computer-readable storage media of claim 10 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing central processing unit (CPU) usage of the service application to the random number.
14. The one or more non-transitory computer-readable storage media of claim 10 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing disk usage of the service application to the random number.
15. The one or more non-transitory computer-readable storage media of claim 10 , wherein comparing the status information regarding performance of the service application to the random number to determine whether to allow the incoming data request to be sent to the service application for further processing comprises comparing network latency of the service application to the random number.
16. The one or more non-transitory computer-readable storage media of claim 10 , further comprising:
logging a log entry for the incoming data request comprising the status information.
17. The one or more non-transitory computer-readable storage media of claim 10 , wherein:
the two or more rules comprises at least a third rule, wherein the second rule comprises applying a step function to the status information and determining whether to allow the incoming data request to be sent to the service application is based on a result of the step function;
the status information comprises status information received over a period of time.
18. The one or more non-transitory computer-readable storage media of claim 10 , wherein:
the two or more rules comprises at least a third rule, wherein the second rule comprises applying a linear function to the status information and determining whether to allow the incoming data request to be sent to the service application is based on a result of the linear function;
the status information comprises status information received over a period of time.