Synchronizable transactional database method and system
A database system that can synchronize all or a part of its contents over a limited bandwidth link is described. The lowest layer of the system, the bedrock layer, implements a transactional block store. On top of this is a B+-tree that can efficiently compute a digest (hash) of the records within any range of key values in O(log n) time. The top level is a communication protocol that directs the synchronization process such that minimization of bits communicated, rounds of communication, and local computation are simultaneously addressed.
1. A method of minimizing network traffic during the process of synchronizing a local transaction database with a remote transaction database, comprising:
receiving from the remote transaction database a single summary hash computed for database records lying in an interval in the remote transaction database;
comparing the single summary hash to a local single summary hash computed for database records lying in a same interval in the local transaction database; and
when the single summary hash does not match the local summary hash, partitioning the interval into at least a first and a second sub-interval and requesting from the remote transaction database a first sub-interval summary hash and a second sub-interval summary hash, the first sub-interval summary hash computed for database records lying in the first sub-interval in the remote transaction database and the second sub-interval summary hash computed for database records lying in the second sub-interval in the remote transaction database,
whereby the database can avoid updating database records in a sub-interval at the occurrence of a match between a local sub-interval summary hash and a sub-interval summary hash received from the remote transaction database.
2. The method of claim 1 further comprising the step of: when the local sub-interval summary hash does not match the sub-interval summary hash received from the remote transaction database, further partitioning the sub-interval into at least a first and a second sub-sub-interval and requesting from the remote transaction database a first and second sub-sub-interval summary hash, the first sub-sub-interval summary hash computed for database records lying in the first sub-sub-interval in the remote transaction database and the second sub-sub-interval summary hash computed for database records lying in the second sub-sub-interval in the remote transaction database.
3. The method of claim 1 further comprising the step of: when the local sub-interval summary hash does not match the sub-interval summary hash received from the remote transaction database, requesting from the remote transaction database a hash value for each database record lying in the sub-interval.
4. The method of claim 1 , wherein each database record has a hash value and wherein each summary hash for an interval is computed by combining the hash values for each database record in the interval.
5. The method of claim 4 , wherein the summary hashes for different intervals are stored in a free structure.
6. The method of claim 5 , wherein the free structure is a B+ tree structure.
7. The method of claim 5 , wherein the hash values are message digests and wherein the hash values are combined to compute a summary hash by an excusive or (XOR) of the hash values.
8. A computer implemented synchronizable transactional database comprising:
a database;
an interval hash value computing module coupled to the database and configured to compute a summary hash of a plurality of hash values, each hash value associated with a database record lying in an interval of the database; and
a synchronization module coupled to the database and to the interval hash value computing module, the synchronization module configured to identify database records that need synchronization by comparing a summary hash from the interval hash value computing module computed for database records lying in an interval of the database with a remote summary hash received from a remote transaction database, the synchronization module further configured to partition an interval into at least a first and a second subinterval when a summary hash for the interval in the database does not match a remote summary hash so as to seek remote summary hashes for the first and second sub-intervals from the remote transaction database,
whereby the database can avoid updating database records in a sub-interval at the occurrence of a match between a local sub-interval summary hash and a sub-interval summary hash received from the remote transaction database.
9. The system of claim 8 , wherein the database further comprises a transactional support layer configured to support a storage layer of the database, the transactional support layer further comprising shadow blocks which provides for atomized updates to the storage layer.
10. The system of claim 9 , wherein to interval hash value computing module computes a summary hash for database records lying in an interval in the database by combining the hash values associated with each database record lying in the interval.
11. The system of claim 10 , wherein the summary hashes for different intervals are stored in a tree structure.
12. The system of claim 11 , wherein the free structure is a B+ tree structure.
13. The system of claim 11 , wherein the hash values are message digests and wherein the hash values are combined to compute a summary hash by using an excusive or (XOR) of the hash values.