Aggregates index
The disclosed technology addresses the need in the art for a content management system that can be highly flexible to the needs of its subjects. The present technology permits any object to be shared by providing a robust and flexible access control list mechanism. The present technology utilizes a data structure that is highly efficient that both minimizes the amount of information that needs to be written into any database, but also allows for fast reads and writes of information from authoritative tables that are a source of truth for the content management system, while allowing for maintenance of indexes containing more refined data that allow for efficient retrieval of certain information that would normally need to be calculated when it is needed.
1. A method comprising:
maintaining an aggregates index including fields for an object identifier (“ID”) associated with an object included in a directory structure and an aggregate value;
receiving an action on a first object;
creating a new row in the aggregates index, the new row including data for a first object ID of the first object, and a timestamp indicating a time of the action on the first object;
determining the first object has at least one ancestor;
based on the action on the first object, recursively performing the following steps:
determining a next ancestor object of the first object in the new row in the aggregates index by moving up directory structure including the first object;
for each ancestor object, creating a second new row in the aggregates index, the timestamp indicating the time of the action on the first object and identifying the first object as being the object ID on which the action was taken that resulted in the new row being created;
receiving a query identifying at least a second object ID and a time value or time range; and
when the second object ID exists in the aggregates index and is associated with the time value or time range, return at least one of data associated with the aggregate value or data associated with the second object ID from a row associated with the second object ID in the aggregates index.
2. The method of claim 1 , wherein each row in the aggregates index includes the aggregated value for aggregated data for all descendants of an object ID that is a subject of the row.
3. The method of claim 2 , wherein the aggregates index includes a plurality of immutable rows, whereby any row in the aggregates index represents the aggregated data for the object ID represented in that row during a time period starting at the timestamp represented in that row and ending at a later timestamp represented in another, later written, row for the object ID.
4. The method of claim 2 , wherein the aggregated data is an aggregated value for at least one category including a number of bytes occupied by all objects located under the object in the directory structure, or a number of objects under the object in the directory structure.
5. The method of claim 2 comprising:
receiving a request to provide aggregate data from the aggregates index, wherein the request specifies a search object ID and a timestamp value or range;
obtaining the aggregate data for the search object ID and the timestamp from a single row in the aggregates index; and
returning the aggregate data in response to the request.
6. The method of claim 1 comprising:
receiving a request to identify changes made under a search object ID, wherein the request specifies the search object ID and a timestamp value or a timestamp range;
obtaining the object ID on which the action was taken from any row matching the request; and
returning the object ID on which the action was taken at the timestamp value or in the timestamp range.
7. The method of claim 1 , wherein the creating the new row in the aggregates index is a synchronous process that is performed in coordination with an update of at least one authoritative table.
8. A non-transitory computer readable medium comprising instructions, the instructions, when executed by a computing system, cause the computing system to:
maintain an aggregates index including fields for an object identifier (“ID”) associated with an object included in a directory structure and an aggregate value;
receiving an action on a first object;
create a new row in the aggregates index, the new row including data for a first object ID of the first object, and a timestamp indicating a time of the action on the first object;
determining the first object has at least one ancestor;
based on the action on the first object, recursively perform the following steps:
determine a next ancestor object of the first object in the new row in the aggregates index by moving up directory structure including the first object;
for each ancestor object, create a second new row in the aggregates index, the timestamp indicating the time of the action on the first object and identifying the first object as being the object ID on which the action was taken that resulted in the new row being created;
receive a query identifying at least a second object ID and a time value or time range; and
when the second object ID exists in the aggregates index and is associated with the time value or time range, return at least one of data associated with the aggregate value or data associated with the second object ID from a row associated with the second object ID in the aggregates index.
9. The non-transitory computer readable medium of claim 8 , wherein the query is for the aggregated value under the object ID.
10. The non-transitory computer readable medium of claim 8 , wherein the instructions further cause the computing system to:
receive, at the aggregates index, a query for aggregated value under the object ID and a time range;
determine that the object ID does not exist in the aggregates index during the time range;
identify the object ID in the aggregates index a most recent time prior to the time range; and
return data from a row in the aggregates index that includes the object ID at the most recent time prior to the time range.
11. The non-transitory computer readable medium of claim 8 , wherein the query is to determine whether any change occurred under the object ID during the time range, the instructions further cause the computing system to:
lookup the object ID during the time range in the aggregates index; and
when the data is returned from the aggregates index, send a response to the query indicating that a change has occurred under the object ID during the time range and identifying the object causing the change, when no data is returned from the aggregates index, send a response to the query indicating that no change has occurred under the object ID during the time range.
12. The non-transitory computer readable medium of claim 8 , wherein the aggregates index includes a plurality of immutable rows, whereby any row in the aggregates index represents aggregated data for the object ID represented in that row during a time period starting at the timestamp represented in that row and ending at a later timestamp represented in another, later written, row for the object ID.
13. The non-transitory computer readable medium of claim 8 , wherein the aggregated data that includes the aggregate value for at least one of a category including a number of bytes occupied by all objects located under the object in the directory structure, or a number of objects under the object in the directory structure.
14. A system comprising:
at least one processor; and
at least one computer readable medium storing instructions, wherein when executed by the at least one processor, the instructions are effective cause the system to:
maintain an aggregates index including fields for an object identifier (“ID”) associated with an object included in a directory structure, an aggregate value, and for an action object ID at a timestamp, wherein the action object ID indicates that an action was taken on the object;
receiving an action on a first object;
create a new row in the aggregates index, the new row including data for a first action object ID of the first object and a timestamp indicating a time of first action on the first object;
determining the first object has at least one ancestor;
based on the action on the first object, recursively performing the following steps:
determining a next ancestor object of the first object in the new row in the aggregates index by moving up directory structure including the first object;
for each ancestor object, creating a second new row in the aggregates index, the timestamp indicating the time of the first action on the first object and identifying the first object as being the object ID on which the first action was taken that resulted in the new row being created;
receive a query identifying at least a second object ID and a time value or time range; and
when the second object ID exists in the aggregates index and is associated with the time value or the time range, return at least one of data associated with the aggregate value or data associated with the second object ID from a row identified by the second object ID in the aggregates index.
15. The system of claim 14 , wherein the aggregates index includes the field for an aggregated value, the aggregated value being descriptive of an aggregation of the data for all descendants of the object ID, and wherein the query is for the aggregated value under the object ID.
16. The system of claim 14 , wherein the instructions further cause the system to:
receive, at the aggregates index, a query for the aggregated value under the object ID and a time range;
determine that the object ID does not exist in the aggregates index during the time range;
identify the object ID in the aggregates index a most recent time prior to the time range; and
return data from the row in the aggregates index include the object ID at the most recent time prior to the time range.
17. The system of claim 14 , wherein the query is to determine whether any change occurred under the object ID during the time range, the instructions further cause the system to:
lookup the object ID during the time range in the aggregates index; and
when the data is returned from the aggregates index, send a response to the query indicating that a change has occurred under the object ID during the time range and identifying the action object ID causing the change.
18. The system of claim 14 , wherein the query is to determine whether any change occurred under the object ID during the time range, the instructions further cause the system to:
lookup the object ID during the time range in the aggregates index; and
when no data is returned from the aggregates index, send a response to the query indicating that no change has occurred under the object ID during the time range.