Atomic and incremental target state definitions for database entities
Techniques for creating and modifying database entities, such as tables, tasks, etc., using declarative statements are described. Declarative statements specify a target state of the entity without specifying specific actions. The techniques described herein apply changes to the database entity atomically and incrementally.
1 . A method comprising:
receiving a statement defining an object definition of a database entity stored in a network-based data system;
parsing, by at least one hardware processor, the statement to determine a target state of the database entity;
obtaining a current state of the database entity from a metadata database in the network-based data system;
comparing the target state and the current state of the database entity to determine a set of changes;
generating a plurality of execution steps based on the set of changes, each execution step comprising a pre-execute sub-step, a commit sub-step, and a post-execute sub-step; and
executing the pre-execute sub-steps and commit sub-steps for the plurality of execution steps using a single transaction with the metadata database so that changes to the database entity corresponding to each of the plurality of execution steps become visible to a user at a same time and intermediate changes between the plurality of execution steps are not visible to the user,
the post-execute sub-steps being performed without interacting with the metadata database.
2 . The method of claim 1 , wherein the statement comprises a declarative statement defining the target state of the database entity without including a specific action.
3 . The method of claim 1 , wherein the pre-execute substep of a first execution step comprises a plurality of operations.
4 . The method of claim 1 , wherein the database entity comprises a table.
5 . The method of claim 1 , wherein database entity comprises a task.
6 . A machine-storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
receiving a statement defining an object definition of a database entity stored in a network-based data system;
parsing, by at least one hardware processor, the statement to determine a target state of the database entity;
obtaining a current state of the database entity from a metadata database in the network-based data system;
comparing the target state and the current state of the database entity to determine a set of changes;
generating a plurality of execution steps based on the set of changes, each execution step comprising a pre-execute sub-step, a commit sub-step, and a post-execute sub-step; and
executing the pre-execute sub-steps and commit sub-steps for the plurality of execution steps using a single transaction with the metadata database so that changes to the database entity corresponding to each of the plurality of execution steps become visible to a user at a same time and intermediate changes between the plurality of execution steps are not visible to the user,
the post-execute sub-steps being performed without interacting with the metadata database.
7 . The machine-storage medium of claim 6 , wherein the statement comprises a declarative statement defining the target state of the database entity without including a specific action.
8 . The machine-storage medium of claim 6 , wherein the pre-execute substep of a first execution step comprises a plurality of operations.
9 . The machine-storage medium of claim 6 , wherein the database entity comprises a table.
10 . The machine-storage medium of claim 6 , wherein database entity comprises a task.
11 . A system comprising:
at least one hardware processor; and
at least one memory storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
receiving a statement defining an object definition of a database entity stored in a network-based data system;
parsing, by at least one hardware processor, the statement to determine a target state of the database entity;
obtaining a current state of the database entity from a metadata database in the network-based data system;
comparing the target state and the current state of the database entity to determine a set of changes;
generating a plurality of execution steps based on the set of changes, each execution step comprising a pre-execute sub-step, a commit sub-step, and a post-execute sub-step; and
executing the pre-execute sub-steps and commit sub-steps for the plurality of execution steps using a single transaction with the metadata database so that changes to the database entity corresponding to each of the plurality of execution steps become visible to a user at a same time and intermediate changes between the plurality of execution steps are not visible to the user,
the post-execute sub-steps being performed without interacting with the metadata database.
12 . The system of claim 11 , wherein the statement comprises a declarative statement defining the target state of the database entity without including a specific action.
13 . The system of claim 11 , wherein the pre-execute substep of a first execution step comprises a plurality of operations.
14 . The system of claim 11 , wherein the database entity comprises a table.
15 . The system of claim 11 , wherein database entity comprises a task.