Systems and methods of quota accounting
View Patent ↗Embodiments of the invention relate generally to incremental computing. Specifically, embodiments of the invention include systems and methods that provide for the concurrent processing of multiple, incremental changes to a data value while at the same time monitoring and/or enforcing threshold values for that data value. For example, a method is provided that implements domain quotas within a data storage system.
1. A system for implementing domain quotas in a data storage system, the system comprising:
a memory configured to store a domain data structure, the domain data structure comprising:
a domain size field, the domain size field comprising a value that reflects a size of a data domain comprising committed transactions;
a bounded size field, the bounded size field comprising a value that reflects a maximum possible size or a minimum possible size of the data domain based on a plurality of pending data transactions that have not committed or aborted;
an incremental value field, the incremental value field comprising a value that reflects a change in the size of the data domain caused by a data transaction;
an operation type field, the operation type field comprising a value that indicates whether the change in the size of the data domain caused by the data transaction is either an increment or a decrement; and
a quota field, the quota field comprising a value that indicates a size threshold for either a minimum or maximum size for the size of the data domain to be within a quota defined for the data domain; and
a processor configured to communicate with the memory, the processor further configured to execute a quota module configured to:
access the domain data structure for a pending data transaction;
determine, based at least in part on the accessed domain data structure, whether a temporary domain size field passes the quota field if the pending data transaction is applied to the data domain;
if the temporary domain size field does not pass the quota field, apply the pending data transaction to the data domain; and
if the temporary domain size field passes the quota field, suspend the pending data transaction.
2. The system of claim 1 , wherein if the processor determines the temporary domain size field passes the quota field, the processor is configured to suspend the pending data transaction until the pending data transaction acquires an exclusive lock.
3. The system of claim 2 , wherein after the exclusive lock is acquired, the processor is configured to commit the pending transaction and to update the domain data structure.
4. The system of claim 1 , wherein if the processor determines the temporary domain size field passes the quota field, the processor is further configured to permit the pending data transaction and to communicate a notification that the quota field has been passed.
5. The system of claim 1 , wherein if the processor determines the temporary domain size field passes the quota field, the processor is further configured to abort the pending transaction.
6. The system of claim 1 , wherein if the processor determines the temporary domain size field passes the quota field, the processor is further configured to permit the pending data transaction and to monitor a condition associated with the domain size field being past the quota field.
7. The system of claim 6 , wherein the condition is an amount of time that the domain size field is past the quota field.
8. The system of claim 1 , further comprising:
a persistent storage;
the processor further configured to execute a journal module configured to store in the persistent storage the pending data transaction until the pending data transaction either is applied to the data domain or is aborted.
9. The system of claim 1 , wherein the data storage system is associated with at least one of the following: a distributed storage system, a file system, and a distributed file system.
10. A method for implementing domain quotas in a data storage system, the method comprising:
accessing a domain data structure for a pending data transaction, the domain data structure comprising:
a domain size field, the domain size field comprising a value that reflects a size of a data domain comprising committed transactions;
a bounded size field, the bounded size field comprising a value that reflects a maximum possible size or a minimum possible size of the data domain based on a plurality of pending data transactions that have not committed or aborted;
an incremental value field, the incremental value field comprising a value that reflects a change in the size of the data domain caused by the pending data transaction;
an operation type field, the operation type field comprising a value that indicates whether the change in the size of the data domain caused by the pending data transaction is either an increment or a decrement; and
a quota field, the quota field comprising a value that indicates a size threshold for either a minimum or maximum size for the size of the data domain to be within a quota defined for the data domain;
determining, based at least in part on the accessed domain data structure, whether a temporary domain size field passes the quota field if the pending data transaction is applied to the data domain, said determining performed via execution of instructions by at least one processor;
if the temporary domain size field does not pass the quota field, applying the pending data transaction to the data domain; and
if the temporary domain size field passes the quota field, suspending the pending data transaction.
11. The method of claim 10 , wherein suspending the pending data transaction comprises suspending the pending data transaction until the pending data transaction acquires an exclusive lock.
12. The method of claim 11 , wherein after the exclusive lock is acquired, the method further comprises:
committing the pending transaction; and
updating the domain data structure.
13. The method of claim 10 , wherein if it is determined that the temporary domain size field passes the quota field, the method further comprises:
permitting the pending data transaction; and
communicating a notification that the quota field has been passed.
14. The method of claim 10 , wherein if it is determined that the temporary domain size field passes the quota field, the method further comprises aborting the pending transaction.
15. The method of claim 10 , wherein if it is determined that the temporary domain size field passes the quota field, the method further comprises:
permitting the pending data transaction; and
monitoring a condition associated with the domain size field being past the quota field.
16. The method of claim 15 , wherein the condition is an amount of time that the domain size field is past the quota field.
17. The method of claim 10 , further comprising storing in a persistent storage the pending data transaction until the pending data transaction either is applied to the data domain or is aborted.
18. The method of claim 10 , wherein the data storage system is associated with at least one of the following: a distributed storage system, a file system, and a distributed file system.