Time Stamp Bounded Addition of Data to an Append-Only Distributed Database Table
A method implemented by a computer includes receiving a segment of data that has a time dimension, where the time dimension of the segment of data is bounded by a start time stamp and an end time stamp. The segment of data is added to an append-only database table of a distributed database. The addition operation imposes an inherent data order based upon the start time stamp and end time stamp without the manual definition of a database table partition in the distributed database.
1 . A method implemented by a computer, comprising:
receiving a segment of data that has a time dimension, wherein the time dimension of the segment of data is bounded by a start time stamp and an end time stamp with intervening time stamps between the start time stamp and the end time stamp, and wherein individual time stamps have associated database record fields; and
adding the segment of data to an append-only database table of a distributed database, wherein adding automatically creates a database table partition based upon the start time stamp and end time stamp without the manual definition of a database table partition in the distributed database.
2 . The method of claim 1 wherein the append-only database table has a schema that identifies the time dimension.
3 . The method of claim 1 wherein the append-only database table is a fact table.
4 . The method of claim 1 wherein the append-only database table is a slowly changing dimension table.
5 . The method of claim 1 further comprising storing the start time stamp and the end time stamp of the segment of data.
6 . The method of claim 5 further comprising:
receiving a database query for the append-only database table, wherein the database query has a time dimension; and
using stored start time stamps and end time stamps to prune a set of segments of data not relevant to the database query.
7 . The method of claim 1 wherein the append-only database table is distributed across a plurality of nodes, wherein the nodes are connected to each other over a computer network.
8 . The method of claim 7 wherein adding the segment of data is managed by a coordinator node.
9 . The method of claim 7 wherein a plurality of segments of data are added in parallel from a plurality of nodes without interfering with one another.
10 . The method of claim 7 wherein the plurality of segments of data are replicated across the plurality of nodes.
11 . The method of claim 1 wherein adding the segment of data automatically creates a plurality of new database table partitions.
12 . The method of claim 11 further comprising using a configuration value to determine the maximum size of new database table partitions.
13 . The method of claim 11 further comprising storing the start time stamp and end time stamp for each new database table partition.
14 . The method of claim 13 further comprising:
receiving a database query for the database table, wherein the database query has a time dimension; and
using start time stamps and end time stamps of database table partitions to prune a set of database table partitions not relevant to the database query.
15 . The method of claim 11 wherein the new database table partitions are distributed across a plurality of nodes connected to each other over a computer network.
16 . The method of claim 15 wherein the plurality of new database table partitions are managed by a coordinator node.
17 . The method of claim 15 wherein the plurality of new database table partitions are added in parallel to a plurality of nodes without interfering with one another.
18 . The method of claim 15 wherein the plurality of new database table partitions are replicated across the plurality of nodes.