PROJECTIONS FOR BIG DATABASE SYSTEMS

1 . A system comprising:

one or more computer readable storage mediums configured to store:

program instructions;

a canonical dataset;

a first projection dataset;

a second projection dataset; and

a buffer; and

one or more processors configured to execute the program instructions to cause the system to:

receive a data chunk comprising an edit to the canonical dataset;

temporarily store the data chunk in the buffer;

update the first projection dataset based on the data chunk; and

update the second projection dataset asynchronously from the first projection dataset based on system resources that are available to update the first projection dataset and the second projection dataset, wherein updating the second projection dataset includes appending the data chunk to contents of the second projection dataset and/or sorting the contents of the second projection dataset.

2 . The system of claim 1 , wherein updating the second projection dataset includes compacting the second projection dataset.

3 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

determine to prioritize updating the first projection dataset over updating the second projection dataset; and

update the first projection dataset before updating the second projection dataset.

4 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

update the second projection dataset asynchronously from the first projection dataset when the system resources are constrained.

5 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

update the second projection dataset based on one or more projection rules associated with the second projection dataset.

6 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

flush the data chunk from the buffer.

7 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

update the second projection dataset asynchronously from the canonical dataset.

8 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

update the second projection dataset asynchronously from the buffer.

9 . The system of claim 1 , wherein the first projection dataset comprises a first subset of the canonical dataset, wherein the second projection dataset comprises a second subset of the canonical dataset.

10 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

in response to receipt of a query of the canonical dataset:

determine one or more projection rules corresponding to a requested result of the query;

determine a selected projection dataset to which the one or more projection rules have been applied;

rewrite the query to execute on the selected projection dataset instead of the canonical dataset; and

execute the query on the selected projection dataset without applying the one or more projection rules to the selected projection dataset during query execution.

11 . The system of claim 1 , wherein the one or more processors are configured to execute the program instructions to further cause the system to:

in response to receipt of a query of the canonical dataset:

combine the buffer with the first projection dataset and/or the second projection dataset to form a combined dataset;

rewrite the query to execute on the combined dataset; and

execute the query on the combined dataset.

12 . A computer-implemented method comprising:

receiving a data chunk comprising an edit to a canonical dataset;

temporarily storing the data chunk in a buffer;

updating a first projection dataset based on the data chunk; and

updating a second projection dataset asynchronously from the first projection dataset based on system resources that are available to update the first projection dataset and the second projection dataset, wherein updating the second projection dataset includes appending the data chunk to contents of the second projection dataset and/or sorting the contents of the second projection dataset.

13 . The computer-implemented method of claim 12 , wherein updating the second projection dataset includes compacting the second projection dataset.

14 . The computer-implemented method of claim 12 further comprising:

updating the second projection dataset asynchronously from the first projection dataset when the system resources are constrained.

15 . The computer-implemented method of claim 12 further comprising:

determining to prioritize updating the first projection dataset over updating the second projection dataset; and

updating the first projection dataset before updating the second projection dataset.

16 . The computer-implemented method of claim 12 further comprising:

in response to receipt of a query of the canonical dataset:

determining one or more projection rules corresponding to a requested result of the query;

determining a selected projection dataset to which the one or more projection rules have been applied;

rewriting the query to execute on the selected projection dataset instead of the canonical dataset; and

executing the query on the selected projection dataset without applying the one or more projection rules to the selected projection dataset during query execution.

17 . One or more computer-readable storage mediums or devices comprising, configured to store, and/or storing program instructions, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

receiving a data chunk comprising an edit to a canonical dataset;

temporarily storing the data chunk in a buffer;

updating a first projection dataset based on the data chunk; and

updating a second projection dataset asynchronously from the first projection dataset based on system resources that are available to update the first projection dataset and the second projection dataset, wherein updating the second projection dataset includes appending the data chunk to contents of the second projection dataset and/or sorting the contents of the second projection dataset.

18 . The one or more computer-readable storage mediums of claim 17 , wherein updating the second projection dataset includes compacting the second projection dataset.

19 . The one or more computer-readable storage mediums of claim 17 , wherein the program instructions executable by one or more processors to cause the one or more processors to perform further operations comprising:

determining to prioritize updating the first projection dataset over updating the second projection dataset; and

updating the first projection dataset before updating the second projection dataset.

20 . The one or more computer-readable storage mediums of claim 17 , wherein the program instructions executable by one or more processors to cause the one or more processors to perform further operations comprising:

in response to receipt of a query of the canonical dataset:

determining one or more projection rules corresponding to a requested result of the query;

determining a selected projection dataset to which the one or more projection rules have been applied;

rewriting the query to execute on the selected projection dataset instead of the canonical dataset; and

executing the query on the selected projection dataset without applying the one or more projection rules to the selected projection dataset during query execution.