Systems and methods of managing resource utilization on a threaded computer system
View Patent ↗Embodiments of the invention relate generally to incremental computing. Specifically, embodiments of the invention include systems and methods for the concurrent processing of multiple, incremental changes to a data value while at the same time monitoring and/or enforcing threshold values for that data value. Embodiments of the invention also include systems and methods of managing utilization of a resource of a computer system having a number of threads.
1. A method of managing utilization of a resource of a computer system having a number of threads, the method comprising:
by a computer system comprising one or more physical processors,
receiving a usage threshold for a resource on the computer system, the usage threshold associated with a trigger event, the resource comprising storage capacity of a data domain stored on a storage device;
determining a usage for the resource on the computer system, wherein the usage comprises an amount of the storage capacity of the data domain that is being used, and the usage threshold comprises a limit on the amount of the storage capacity of the data domain allowed to be used on the storage device;
organizing the computer system into a number of subsystems, wherein the number of subsystems is two or more, and wherein the number is determined based at least in part on factors including the number of threads, the usage threshold, and the usage, wherein each of the subsystems is configured to track a different portion of the storage capacity of the data domain;
allocating the subsystems among the threads, wherein each thread is allocated one or more subsystems and wherein each subsystem is configured to access the portion of the resource available and which the subsystem is configured to track;
dividing the usage threshold into a set of subsystem thresholds based on the number of subsystems;
tracking resource usage for each subsystem to determine whether the trigger event occurs, wherein the trigger event is based on a subsystem usage for each subsystem and a subsystem threshold corresponding to the subsystem; and
in response to an occurrence of the trigger event associated with the usage threshold for the resource:
distributing a request to modify resource usage to at least one subsystem, and
initiating an enforcement action.
2. The method of claim 1 , wherein the computer system comprises a distributed system comprising one or more nodes.
3. The method of claim 2 , wherein at least one node of the distributed system is allocated one or more subsystems.
4. The method of claim 2 , wherein the distributed system comprises at least one of a distributed storage system and a distributed file system.
5. The method of claim 1 , wherein the resource comprises physical space on the storage device.
6. The method of claim 1 , wherein the resource comprises quantity of files stored on the storage device.
7. The method of claim 6 , wherein the quantity of files comprises a count associated with the number of files.
8. The method of claim 6 , wherein the quantity of files comprises physical space associated with the files.
9. The method of claim 1 , wherein the resource comprises logical space on the storage device.
10. The method of claim 9 , wherein the logical space comprises physical space less space relating to metadata associated with a protection level for the resource.
11. The method of claim 1 , wherein the sum of the subsystem usage thresholds equals the usage threshold.
12. The method of claim 1 , wherein the sum of the subsystem usages equals the usage.
13. The method of claim 1 , wherein the request to modify resource usage comprises a plurality of delta requests for the resource, wherein each delta request comprises a request to make an incremental change to the amount of the storage capacity of the data domain stored on the storage device, the incremental change comprising an incremental value and an associated operation type corresponding to increment or decrement, and
wherein the method further comprises randomly distributing the plurality of delta requests to the subsystems.
14. The method of claim 13 , wherein the trigger event comprises a determination that the delta request could cause the subsystem usage to pass the subsystem usage threshold in combination with any subset of delta requests pending on the subsystem.
15. The method of claim 1 , wherein allocating the subsystems among the threads comprises allocating at most one subsystem to any thread.
16. The method of claim 1 , wherein the factors for determining the number of subsystems further include a protection level for the resource.
17. The method of claim 16 , wherein allocating the subsystems among the threads comprises mirroring the subsystems based at least in part on the protection level.
18. The method of claim 1 , further comprising reorganizing the computer system into one or more subsystems based on occurrence of an event.
19. The method of claim 18 , wherein the event comprises a subsystem usage level passing a subsystem usage threshold.
20. The method of claim 18 , wherein the event comprises adding a new thread to the computer system.
21. The method of claim 18 , wherein the event comprises updating a usage threshold for the computer system or a subsystem.
22. The method of claim 18 , wherein the event comprises a possible value range boundary associated with subsystem usage passing a subsystem usage threshold.
23. The method of claim 18 , wherein said reorganizing comprises determining a number of the subsystems for reorganization, wherein the number is one or more.
24. The method of claim 23 , wherein the number is one.
25. The method of claim 23 , wherein the number is based at least in part on factors including the number of threads, the usage threshold, and the usage.
26. The method of claim 25 , wherein the factors further include a protection level for the resource.
27. The method of claim 18 , wherein the trigger event comprises a determination that the event has occurred.
28. The method of claim 1 , wherein the enforcement action is based on a type of the usage threshold.
29. The method of claim 1 , wherein the enforcement action comprises preventing execution of an action that would cause resource usage to pass the usage threshold.
30. A non-transitory computer-readable medium having instructions stored thereon for managing, when the instructions are executed, utilization of a resource of a computer system having a number of threads, the instructions comprising:
receiving a usage threshold for a resource on the computer system, the usage threshold associated with a trigger event, the resource comprising storage capacity of a data domain stored on a storage device;
determining a usage for the resource on the computer system, wherein the usage comprises an amount of the storage capacity of the data domain that is being used, and the usage threshold comprises a limit on the amount of the storage capacity of the data domain allowed to be used on the storage device;
organizing the computer system into a number of subsystems, wherein the number of subsystems is two or more, and wherein the number is determined based at least in part on factors including the number of threads, the usage threshold, and the usage, wherein each of the subsystems is configured to track a different portion of the storage capacity of the data domain;
allocating the subsystems among the threads, wherein each thread is allocated one or more subsystems and wherein each subsystem is configured to access the portion of the resource available and which the subsystem is configured to track;
dividing the usage threshold into a set of subsystem thresholds based on the number of subsystems;
tracking resource usage for each subsystem to determine whether the trigger event occurs, wherein the trigger event is based on a subsystem usage for each subsystem and a subsystem threshold corresponding to the subsystem; and
in response to an occurrence of the trigger event associated with the usage threshold for the resource:
distributing a request to modify resource usage to at least one subsystem, and
initiating an enforcement action.
31. The non-transitory computer-readable medium of claim 30 , wherein the enforcement action comprises preventing execution of an action that would cause resource usage to pass the usage threshold.
32. The non-transitory computer-readable medium of claim 30 , wherein the enforcement action comprises informing a user of an action that causes the occurrence of the trigger event.
33. A system for managing utilization of a resource of a computer system having a number of threads, the system comprising:
an accounting system comprising a physical computer processor, the accounting system configured to receive a usage threshold and to determine usage for a resource on the computer system, the usage threshold associated with a trigger event, the resource comprising storage capacity of a data domain stored on a storage device of the computer system, and wherein the usage comprises an amount of the storage capacity of the data domain that is being used and the usage threshold comprises a limit on the amount of the storage capacity of the data domain allowed to be used on the storage device;
wherein the accounting system is further configured to organize the computer system into a number of subsystems, wherein the number is two or more and depends at least in part on factors including the number of threads, the usage threshold, and the usage, wherein each of the subsystems is configured to track a different portion of the storage capacity of the data domain;
wherein the accounting system is further configured to allocate the subsystems among the threads for tracking resource usage for each subsystem to determine whether the trigger event occurs;
wherein each thread is allocated one or more subsystems and wherein each subsystem is configured the portion of the resource available and which the subsystem is configured to track;
wherein the accounting system is further configured to divide the usage threshold into a set of subsystem thresholds based on the number of subsystems;
wherein the trigger event is based on a subsystem usage for each subsystem and a subsystem threshold corresponding to the subsystem; and
wherein, in response to an occurrence of the trigger event associated with the usage threshold for the resource, the accounting system is further configured to distribute a request to modify resource usage to at least one subsystem and initiate an enforcement action.
34. The system of claim 33 , wherein the computer system comprises a distributed system comprising one or more nodes.
35. The system of claim 34 , wherein at least one node of the distributed system is allocated one or more subsystems.
36. The system of claim 33 , wherein the distributed system comprises at least one of a distributed storage system and a distributed file system.
37. The system of claim 33 , wherein the resource comprises physical space on the storage device.
38. The system of claim 33 , wherein the resource comprises quantity of files stored on the storage device.
39. The system of claim 38 , wherein the quantity of files comprises a count associated with the number of files.
40. The system of claim 38 , wherein the quantity of files comprises physical space associated with the files.
41. The system of claim 33 , where the resource comprises logical space on the storage device.
42. The system of claim 41 , wherein the logical space comprises physical space less space relating to metadata associated with a protection level for the resource.
43. The system of claim 33 , wherein the sum of all the subsystem usage thresholds equals the usage threshold.
44. The system of claim 33 , wherein the sum of all the subsystem usage levels equals the usage.
45. The system of claim 33 , wherein the request to modify resource usage comprises a plurality of delta requests for the resource, wherein each delta request comprises a request to make an incremental change to the amount of the storage capacity of the data domain stored on the storage device, the incremental change comprising an incremental value and an associated operation type corresponding to increment or decrement, and
wherein the accounting system is further configured to randomly distribute the plurality of delta requests to the subsystems.
46. The system of claim 45 , wherein the trigger event comprises a determination that the delta request could cause the subsystem usage to pass the subsystem usage threshold in combination with any subset of delta requests pending on the subsystem.
47. The system of claim 33 , wherein the accounting system is configured to allocate at most one subsystem to any thread.
48. The system of claim 33 , wherein the factors for determining the number of subsystems further include a protection level for the resource.
49. The system of claim 48 , wherein the accounting system is further configured to mirror the subsystems based at least in part on the protection level.
50. The system of claim 33 , wherein the accounting system is further configured to reorganize the computer system into one or more subsystems based on occurrence of an event.
51. The system of claim 50 , wherein the event comprises a subsystem usage level passing a subsystem usage threshold.
52. The system of claim 50 , wherein the event comprises adding a new thread to the computer system.
53. The system of claim 50 , wherein the event comprises updating a usage threshold for the computer system or a subsystem.
54. The system of claim 50 , wherein the event comprises a possible value range boundary associated with the subsystem usage level passing a subsystem usage threshold.
55. The system of claim 50 , wherein to reorganize the computer system into one or more subsystems, the accounting system is configured to determine a number of the subsystems for reorganization, wherein the number is one or more.
56. The system of claim 55 , wherein the number is one.
57. The system of claim 55 , wherein the number is based at least in part on factors including the number of threads, the usage threshold, and the usage.
58. The system of claim 57 , wherein the factors further include a protection level for the resource.
59. The method of claim 1 , wherein the enforcement action comprises informing a user of an action that causes the occurrence of the trigger event.
60. The non-transitory computer-readable medium of claim 30 , wherein the enforcement action is based on a type of the usage threshold.
61. The system of claim 50 , wherein the trigger event comprises a determination that the event has occurred.
62. The system of claim 33 , wherein the enforcement action is based on a type of the usage threshold.
63. The system of claim 33 , wherein the enforcement action comprises preventing execution of an action that would cause resource usage to pass the usage threshold.
64. The system of claim 33 , wherein the enforcement action comprises informing a user of an action that causes the occurrence of the trigger event.