Logical Representation of Distributed Database Table Updates in an Append-Only Log File

1 . A method implemented by a computer network, comprising:

receiving a database table update to a distributed database;

loading the database table update into a database table partition on a node of a computer network executing the distributed database, wherein the database table partition does not exceed a maximum size and therefore the node holds multiple database table partitions; and

adding the database table update to an append-only log file corresponding to the database table partition, wherein adding includes representing data associated with the database table update in a logical representation that is independent of the physical representation of the data at the node to facilitate transferring the database table partition to other nodes within the computer network.

2 . The method of claim 1 wherein the database table update is selected from a new data insertion, a data load, a data modification, an old data deletion and a Data Description Language (DDL) command.

3 . The method of claim 1 wherein the database table update is selected from a new data insertion and a data load command that changes the database table in a deterministic manner.

4 . The method of claim 1 further comprising:

transferring the append-only log file to a second node; and

applying data changes from the append-only log file to a replica database table partition on the second node, wherein the replica database table partition corresponds to the database table partition on the original node.

5 . The method of claim 1 further comprising:

transferring the database table update to a second node;

loading the database table update to a replica database table partition on the second node, wherein the replica database table partition corresponds to the database table partition on the node; and

adding the database table update to a replica append-only log file corresponding to the replica database table partition on the second node.

6 . The method of claim 1 further comprising compacting the append-only log file.

7 . The method of claim 1 further comprising keeping a last written byte offset and a last applied byte offset for the append-only log file.

8 . The method of claim 1 wherein adding the database table update to the append-only log file includes calculating and adding a checksum for the database table update.

9 . The method of claim 1 wherein the maximum size of the database table partition is determined by a configuration value.

10 . The method of claim 1 wherein the database table partition has a unique identifier that is recognized across the distributed database.

11 . The method of claim 1 further comprising receiving a database query and partitioning the database query to run in parallel across the database table partitions.

12 . The method of claim 1 wherein the database table partitions are replicated across the plurality of nodes within the computer network.

13 . The method of claim 1 further comprising re-replicating a database table partition after a node becomes unavailable.

14 . The method of claim 1 wherein new database table partitions are placed randomly across the plurality of nodes in the computer network.

15 . The method of claim 1 wherein new database table partitions are positioned using information on current distribution of database table partitions.

16 . The method of claim 1 wherein new database table partitions are placed using information on availability of the plurality of nodes.

17 . The method of claim 1 wherein database table partitions are redistributed using information on current distribution of database table partitions.

18 . The method of claim 1 wherein database table partitions are redistributed using information on availability of the plurality of nodes.

19 . The method of claim 1 wherein the append-only log file is a text file.

20 . The method of claim 1 wherein the append-only log file is a binary file.