Incremental refresh of a materialized view

1. A method comprising:

storing a source table including a set of micro-partitions in a plurality of shared storage devices;

storing, by a compute service manager independent of the plurality of shared storage devices, a materialized view based on the source table, the materialized view being a persistent query result;

updating, by one or more execution nodes assigned by the compute service manager, the source table;

receiving, by the computer service manager, a query;

merging, by the compute service manager, the updated source table and the materialized view to generate a merged table;

scanning the merged table to identify an update to the source table not reflected in the materialized view, the update including a modification to one or more of micro-partitions in the set of micro-partitions in the source table;

applying, by one or more execution nodes assigned by the compute service manager, the identified update to the materialized view to generate an updated materialized view; and

executing the query using the updated materialized view without accessing the source table.

2. The method of claim 1 , wherein applying the identified update includes updating a single micro-partition in the materialized view corresponding to the set of micro-partitions in the source table.

3. The method of claim 1 , further comprising:

redirecting a second query away from the materialized view to the source table while the update is being applied to the materialized view.

4. The method of claim 1 , wherein applying the identified update to the materialized view comprises:

in response to determining a new row has been inserted into the source table, refreshing the materialized view by inserting the new row into the materialized view; and

in response to determining a row has been deleted from the source table, compacting the materialized view by deleting the row from the materialized view.

5. The method of claim 4 , wherein refreshing the materialized view by inserting the new row into the materialized view is prioritized over compacting the materialized view by deleting the row from the materialized view.

6. The method of claim 4 , wherein compacting the materialized view occurs in response to detecting a quantity of deleted row exceeds a predetermined threshold number of deleted rows.

7. The method of claim 1 , wherein the update comprises one or more of a new micro-partition being added to the source table or one or more of micro-partitions being deleted from the source table.

8. A system comprising:

at least one hardware processor; and

at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:

storing a source table including a set of micro-partitions in a plurality of shared storage devices;

storing, by a compute service manager independent of the plurality of shared storage devices, a materialized view based on the source table, the materialized view being a persistent query result;

updating, by one or more execution nodes assigned by the compute service manager, the source table;

receiving, by the computer service manager, a query;

merging, by the compute service manager, the updated source table and the materialized view to generate a merged table;

scanning the merged table to identify an update to the source table not reflected in the materialized view, the update including a modification to one or more of micro-partitions in the set of micro-partitions in the source table;

applying, by one or more execution nodes assigned by the compute service manager, the identified update to the materialized view to generate an updated materialized view; and

executing the query using the updated materialized view without accessing the source table.

9. The system of claim 8 , wherein applying the identified update includes updating a single micro-partition in the materialized view corresponding to the set of micro-partitions in the source table.

10. The system of claim 8 , further comprising:

redirecting a second query away from the materialized view to the source table while the update is being applied to the materialized view.

11. The system of claim 8 , wherein applying the identified update to the materialized view comprises:

in response to determining a new row has been inserted into the source table, refreshing the materialized view by inserting the new row into the materialized view; and

in response to determining a row has been deleted from the source table, compacting the materialized view by deleting the row from the materialized view.

12. The system of claim 11 , wherein refreshing the materialized view by inserting the new row into the materialized view is prioritized over compacting the materialized view by deleting the row from the materialized view.

13. The system of claim 11 , wherein compacting the materialized view occurs in response to detecting a quantity of deleted row exceeds a predetermined threshold number of deleted rows.

14. The system of claim 8 , wherein the update comprises one or more of a new micro-partition being added to the source table or one or more of micro-partitions being deleted from the source table.

15. A non-transitory machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:

storing a source table including a set of micro-partitions in a plurality of shared storage devices;

storing, by a compute service manager independent of the plurality of shared storage devices, a materialized view based on the source table, the materialized view being a persistent query result;

updating, by one or more execution nodes assigned by the compute service manager, the source table;

receiving, by the computer service manager, a query;

merging, by the compute service manager, the updated source table and the materialized view to generate a merged table;

scanning the merged table to identify an update to the source table not reflected in the materialized view, the update including a modification to one or more of micro-partitions in the set of micro-partitions in the source table;

applying, by one or more execution nodes assigned by the compute service manager, the identified update to the materialized view to generate an updated materialized view; and

executing the query using the updated materialized view without accessing the source table.

16. The non-transitory machine-storage medium of claim 15 , wherein applying the identified update includes updating a single micro-partition in the materialized view corresponding to the set of micro-partitions in the source table.

17. The non-transitory machine-storage medium of claim 15 , further comprising:

redirecting a second query away from the materialized view to the source table while the update is being applied to the materialized view.

18. The non-transitory machine-storage medium of claim 15 , wherein applying the identified update to the materialized view comprises:

in response to determining a new row has been inserted into the source table, refreshing the materialized view by inserting the new row into the materialized view; and

in response to determining a row has been deleted from the source table, compacting the materialized view by deleting the row from the materialized view.

19. The non-transitory machine-storage medium of claim 18 , wherein refreshing the materialized view by inserting the new row into the materialized view is prioritized over compacting the materialized view by deleting the row from the materialized view.

20. The non-transitory machine-storage medium of claim 18 , wherein compacting the materialized view occurs in response to detecting a quantity of deleted row exceeds a predetermined threshold number of deleted rows.

21. The non-transitory machine-storage medium of claim 15 , wherein the update comprises one or more of a new micro-partition being added to the source table or one or more of micro-partitions being deleted from the source table.