Resource allocation using distributed segment processing credits
Systems and methods for allocating resources are disclosed. Resources as processing time, writes or reads are allocated. Credits are issued to the clients in a manner that ensure the system is operating in a safe allocation state. The credits can be used not only to allocate resources but also to throttle clients where necessary. Credits can be granted fully, partially, and in a number greater than requested. Zero or negative credits can also be issued to throttle clients. Segment credits are associated with identifying unique fingerprints or segments and may be allocated by determining how many credits a CPU/cores can support. This maximum number may be divided amongst clients connected with the server.
1. A method for allocating resources of a cluster of servers to connected clients, the method comprising:
receiving a request for credits from a client, wherein each credit corresponds to resources of the cluster;
determining, by the cluster, whether the credits requested by the client result in a safe allocation state for all of the clients, wherein the cluster determines whether the allocation state is safe by finding a hypothetical set of credit requests by all the clients that allows each of the clients to request and use credits, wherein the allocation state is safe when the set of credit requests is found and wherein the allocation state is not safe when the set of credit requests is not found; and
issuing the credits to the client such that the allocation state is safe.
2. The method of claim 1 , further comprising determining that the allocation state is safe when the set of credit requests allows each of the clients to request and use their maximum allowed credits.
3. The method of claim 1 , further comprising determining that the allocation state is unsafe and issuing the credits in a number that is one of:
less than a number of the credits requested by the client;
zero; or
negative.
4. The method of claim 1 , wherein the cluster maintains matrices including a total availability matrix identifying a maximum number of credits available from each server in the cluster, a current availability matric identifying a number of credits available from each of the servers, a current demand matrix identifying the maximum allowed credits of each of the clients, and a current need matrix identifying credit needs for each of the clients, further comprising determining whether the allocation state is safe based on the matrices.
5. The method of claim 1 , further comprising determining whether the allocation state is safe by considering whether each of the clients can request and use their maximum allowed credits in succession.
6. The method of claim 5 , further comprising determining whether a first client can request and use their maximum allowed credits and then determining whether a second client can request and use their maximum allowed credits.
7. The method of claim 1 , wherein the credits are one of segment credits for segment processing operations, stream credits for streaming operations, or restore credits for restoring operations.
8. The method of claim 1 , further comprising throttling the client when the number of credits issued is zero or negative.
9. The method of claim 1 , further comprising releasing credits by the client.
10. A non-transitory computer readable medium including computer executable instructions configured to be executed by a processor for performing a method, the method comprising:
receiving a request for credits from a client, wherein each credit corresponds to resources of a cluster of servers connected to clients including the client;
determining, by the cluster, whether the credits requested by the client result in a safe allocation state for all of the clients, wherein the cluster determines whether the allocation state is safe by finding a hypothetical set of credit requests by all the clients that allows each of the clients to request and use credits, wherein the allocation state is safe when the set of credit requests is found and wherein the allocation state is not safe when the set of credit requests is not found; and
issuing the credits to the client such that the allocation state is safe.
11. The non-transitory computer readable medium of claim 10 , further comprising determining that the allocation state is safe when the set of credit requests allows each of the clients to request and use their maximum allowed credits.
12. The non-transitory computer readable medium of claim 10 , further comprising determining that the allocation state is unsafe and issuing the credits in a number that is one of:
less than a number of the credits requested by the client;
zero; or
negative.
13. The non-transitory computer readable medium of claim 10 , wherein the cluster maintains matrices including a total availability matrix identifies a maximum number of credits available from each server in the cluster, a current availability matric identifies a number of credits available from each of the servers, a current demand matrix identifies the maximum allowed credits of each of the clients, and a current need matrix identifies credit needs for each of the clients, further comprising determining whether the allocation state is safe based on the matrices.
14. The non-transitory computer readable medium of claim 10 , further comprising determining whether the allocation state is safe by considering whether each of the clients can request and use their maximum allowed credits in succession.
15. The non-transitory computer readable medium of claim 14 , further comprising determining whether a first client can request and use their maximum allowed credits and then determining whether a second client can request and use their maximum allowed credits.
16. The non-transitory computer readable medium of claim 10 , wherein the credits are one of segment credits for segment processing operations, stream credits for streaming operations, or restore credits for restoring operations.
17. The non-transitory computer readable medium of claim 10 , further comprising throttling the client when the number of credits issued is zero or negative.
18. The non-transitory computer readable medium of claim 10 , further comprising releasing credits by the client.