Transactional streaming of change tracking data

1. A method for generating a transactional change tracking stream, the method comprising:

executing a transaction on a table of a database, the table comprising at least one micro-partition;

generating, during executing the transaction, a new micro-partition, different than the at least one micro-partition, that reflects changes made to the table by the transaction, the at least one micro-partition being unmodified by the transaction;

generating a new table in the database, the new table including the new micro-partition and the at least one micro-partition;

generating a change tracking entry in response to the transaction being fully executed, the change tracking entry comprising an indication of one or more modifications made to the table by the transaction, an indication of the new micro-partition generated during the transaction, and a timestamp indicating when the transaction was fully executed; and

entering the change tracking entry into a change tracking stream, the change tracking stream comprising one or more sequential change tracking entries that each correspond to a different transaction that has been fully executed on the table.

2. The method of claim 1 , wherein the transaction comprises one or more of an insert, a delete, an update, and a merge of database data.

3. The method of claim 1 , wherein the table comprises a plurality of micro-partitions, and the transaction is executed on one or more of the micro-partitions in the plurality of micro-partitions.

4. The method of claim 1 , wherein the change tracking entry further comprises one or more of:

a timestamp indicating when the transaction was requested;

an identifier of a user that requested the transaction;

an identifier of an account that requested the transaction; and

a minimum and maximum of data values inserted by the transaction.

5. The method of claim 1 , wherein generating the change tracking entry comprises generating one or more change tracking entries for each of one or more micro-partitions of the table that are modified by the transaction.

6. The method of claim 5 , wherein each of the one or more change tracking entries is stored as metadata in an applicable micro-partition of the one or more micro-partitions.

7. The method of claim 1 , wherein the change tracking stream comprises a change tracking micro-partition that is stored within the table as metadata, and wherein the change tracking micro-partition is separate from one or more micro-partitions of the table that comprise database data.

8. The method of claim 1 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, and wherein the method further comprises reading the change tracking stream from a first timestamp to a second timestamp to determine a delta for database data between the first timestamp and the second timestamp.

9. The method of claim 8 , wherein the first timestamp indicates a most recent time the change tracking stream was read, and the second timestamp indicates a present time, and wherein the delta for the database data between the first timestamp and the second timestamp indicates one or more modifications made to the table since the most recent time the change tracking stream was read.

10. The method of claim 1 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, and wherein the method further comprises reading the change tracking stream to determine a comprehensive change tracking summary between a first timestamp and a second timestamp, wherein the comprehensive change tracking summary comprises an indication of all incremental modifications made to the table between the first timestamp and the second timestamp.

11. A system for generating a transactional change tracking stream, the system comprising:

one or more processors; and

data storage containing instructions executable by the one or more processors to perform operations comprising:

executing a transaction on a table of a database, the table comprising at least one micro-partition;

generating, during executing the transaction, a new micro-partition, different than the at least one micro-partition, that reflects changes made to the table by the transaction, the at least one micro-partition being unmodified by the transaction;

generating a new table in the database, the new table including the new micro-partition and the at least one micro-partition;

generating a change tracking entry in response to the transaction being fully executed, the change tracking entry comprising an indication of one or more modifications made to the table by the transaction, an indication of the new micro-partition generated during the transaction, and a timestamp indicating when the transaction was fully executed; and

entering the change tracking entry into a change tracking stream, the change tracking stream comprising one or more sequential change tracking entries that each correspond to a different transaction that has been fully executed on the table.

12. The system of claim 11 , wherein the transaction comprises one or more of an insert, a delete, an update, and a merge of database data.

13. The system of claim 11 , wherein the table comprises a plurality of micro-partitions, and the transaction is executed on one or more of the micro-partitions in the plurality of micro-partitions.

14. The system of claim 11 , wherein the change tracking entry further comprises one or more of:

a timestamp indicating when the transaction was requested;

an identifier of a user that requested the transaction;

an identifier of an account that requested the transaction; and

a minimum and maximum of data values inserted by the transaction.

15. The system of claim 11 , wherein:

generating the change tracking entry comprises generating one or more change tracking entries for each of one or more micro-partitions of the table that are modified by the transaction; and

each of the one or more change tracking entries is stored as metadata in an applicable micro-partition of the one or more micro-partitions.

16. The system of claim 11 , wherein the change tracking stream comprises a change tracking micro-partition that is stored within the table as metadata, and wherein the change tracking micro-partition is separate from one or more micro-partitions of the table that comprise database data.

17. The system of claim 11 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, the operations further comprising reading the change tracking stream from a first timestamp to a second timestamp to determine a delta for database data between the first timestamp and the second timestamp.

18. The system of claim 17 , wherein the first timestamp indicates a most recent time the change tracking stream was read, and the second timestamp indicates a present time, and wherein the delta for the database data between the first timestamp and the second timestamp indicates one or more modifications made to the table since the most recent time the change tracking stream was read.

19. The system of claim 11 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, the operations further comprising reading the change tracking stream to determine a comprehensive change tracking summary between a first timestamp and a second timestamp, wherein the comprehensive change tracking summary comprises an indication of all incremental modifications made to the table between the first timestamp and the second timestamp.

20. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

executing a transaction on a table of a database, the table comprising at least one micro-partition;

generating, during executing the transaction, a new micro-partition, different than the at least one micro-partition, that reflects changes made to the table by the transaction, the at least one micro-partition being unmodified by the transaction;

generating a new table in the database, the new table including the new micro-partition and the at least one micro-partition;

generating a change tracking entry in response to the transaction being fully executed, the change tracking entry comprising an indication of one or more modifications made to the table by the transaction, an indication of the new micro-partition generated during the transaction, and a timestamp indicating when the transaction was fully executed; and

entering the change tracking entry into a change tracking stream, the change tracking stream comprising one or more sequential change tracking entries that each correspond to a different transaction that has been fully executed on the table.

21. The non-transitory computer readable storage media of claim 20 , wherein the transaction comprises one or more of an insert, a delete, an update, and a merge of database data.

22. The non-transitory computer readable storage media of claim 20 , wherein the change tracking entry further comprises one or more of:

a timestamp indicating when the transaction was requested;

an identifier of a user that requested the transaction;

an identifier of an account that requested the transaction; and

a minimum and maximum of data values inserted by the transaction.

23. The non-transitory computer readable storage media of claim 20 , wherein:

generating the change tracking entry comprises generating one or more change tracking entries for each of one or more micro-partitions of the table that are modified by the transaction; and

each of the one or more change tracking entries is stored as metadata in an applicable micro-partition of the one or more micro-partitions.

24. The non-transitory computer readable storage media of claim 20 , wherein the change tracking stream comprises a change tracking micro-partition that is stored within the table as metadata, and wherein the change tracking micro-partition is separate from one or more micro-partitions of the table that comprise database data.

25. The non-transitory computer readable storage media of claim 20 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, the operations further comprising reading the change tracking stream from a first timestamp to a second timestamp to determine a delta for database data between the first timestamp and the second timestamp.

26. The non-transitory computer readable storage media of claim 20 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, the operations further comprising reading the change tracking stream to determine a comprehensive change tracking summary between a first timestamp and a second timestamp, wherein the comprehensive change tracking summary comprises an indication of all incremental modifications made to the table between the first timestamp and the second timestamp.

27. A system comprising:

one or more processors; and

data storage containing instructions executable by the one or more processors to perform operations comprising:

executing a transaction on a table of a database, the table comprising at least one micro-partition;

generating, during executing the transaction, a new micro-partition, different than the at least one micro-partition, that reflects changes made to the table by the transaction, the at least one micro-partition being unmodified by the transaction;

generating a new table in the database, the new table including the new micro-partition and the at least one micro-partition;

generating a change tracking entry in response to the transaction being fully executed, the change tracking entry comprising an indication of one or more modifications made to the table by the transaction, an indication of the new micro-partition generated during the transaction, and a timestamp indicating when the transaction was fully executed; and

entering the change tracking entry into a change tracking stream, the change tracking stream comprising one or more sequential change tracking entries that each correspond to a different transaction that has been fully executed on the table.

28. The system of claim 27 , wherein the transaction comprises one or more of an insert, a delete, an update, and a merge of database data.

29. The system of claim 27 , wherein the table comprises a plurality of micro-partitions, and the transaction is executed on one or more of the micro-partitions in the plurality of micro-partitions.

30. The system of claim 27 , wherein the change tracking entry further comprises one or more of:

a timestamp indicating when the transaction was requested;

an identifier of a user that requested the transaction;

an identifier of an account that requested the transaction; and

a minimum and maximum of data values inserted by the transaction.

31. The system of claim 27 , wherein generating the change tracking entry comprises generating one or more change tracking entries for each of one or more micro-partitions of the table that are modified by the transaction.

32. The system of claim 31 , wherein each of the one or more change tracking entries is stored as metadata in an applicable micro-partition of the one or more micro-partitions.

33. The system of claim 27 , wherein the change tracking stream comprises a change tracking micro-partition that is stored within the table as metadata, and wherein the change tracking micro-partition is separate from one or more micro-partitions of the table that comprise database data.

34. The system of claim 27 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, and wherein the operations further comprise reading the change tracking stream from a first timestamp to a second timestamp to determine a delta for database data between the first timestamp and the second timestamp.

35. The system of claim 34 , wherein the first timestamp indicates a most recent time the change tracking stream was read, and the second timestamp indicates a present time, and wherein the delta for the database data between the first timestamp and the second timestamp indicates one or more modifications made to the table since the most recent time the change tracking stream was read.

36. The system of claim 27 , wherein each of the one or more sequential change tracking entries in the change tracking stream comprises a timestamp indicating when a corresponding transaction was executed, and wherein the operations further comprise reading the change tracking stream to determine a comprehensive change tracking summary between a first timestamp and a second timestamp, wherein the comprehensive change tracking summary comprises an indication of all incremental modifications made to the table between the first timestamp and the second timestamp.