Coordinated hash table indexes to facilitate reducing database reconfiguration time
According to embodiments, reconfiguration of lock masters for a cluster of nodes is optimized using coordinated hash indexes to index into the master hash table and into local hash tables stored on the nodes of the cluster. A DBMS uses a hybrid hash index, a portion of which represents a corresponding master hash index, to index into both the master hash table and into a local hash table for a given lock. The hash index used to store lock metadata in a particular local hash table bucket, on a particular node, encodes the lock master index, for a master hash table, to which the locks in the local hash table bucket correspond. Only the portions of the local hash table on the lock master that correspond to the index of the master hash table bucket need to be scanned in order to perform needed tasks for lock master reconfiguration.
1. A method comprising:
calculating a hybrid hash index based on one or more hash functions and an identifier associated with a lock structure for a particular resource in a database that is managed by a multi-node system running on a cluster of nodes;
wherein a portion of the hybrid hash index comprises a master hash table index that indexes into a master hash table maintained for the multi-node system;
based, at least in part, on the portion of the hybrid hash index that comprises the master hash table index, determining that a particular node of the cluster of nodes is a lock master for the lock structure;
adding, into a local hash table at a position indicated by the hybrid hash index, lock structure reference information for the lock structure;
wherein the local hash table is stored at the particular node; and
after adding the lock structure reference information to the local hash table:
identifying the particular node as the lock master for the lock structure by retrieving, from the master hash table based on the hybrid hash index, identifying information for the particular node;
in response to identifying the particular node as the lock master for the lock structure, retrieving the lock structure reference information, from the local hash table stored at the particular node, based on the hybrid hash index; and
accessing the lock structure based on the lock structure reference information;
detecting a condition that requires reconfiguring lock masters for the cluster of nodes;
in response to detecting the condition that requires reconfiguring lock masters for the cluster of nodes:
reassigning an assigned lock master, for a set of lock structures that are mapped to a particular bucket of the master hash table, from a first node of the cluster of nodes to a second node of the cluster of nodes;
wherein the particular bucket is indexed, within the master hash table, by a particular index number represented by a set of bits;
wherein reassigning the set of lock structures comprises:
identifying a set of hash buckets, in a first local hash table on the first node, that are indexed by hash indexes whose high bits are the set of bits that represent the particular index number;
in response to identifying the set of hash buckets: storing lock structure reference information, from the set of hash buckets, in a second local hash table on the second node;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , wherein calculating the hybrid hash index comprises calculating the hybrid hash index based on a single hybrid hash function.
3. The method of claim 1 , wherein calculating the hybrid hash index comprises:
calculating a master hash index, based on the identifier associated with the lock structure and a master hash function of the one or more hash functions, that indexes into the master hash table;
calculating a local component number based on the identifier associated with the lock structure and a second hash function of the one or more hash functions;
concatenating the master hash index and at least a portion of the local component number to generate the hybrid hash index.
4. The method of claim 3 , wherein retrieving, from the master hash table based on the hybrid hash index, the identifying information for the particular node comprises retrieving the identifying information for the particular node, from the master hash table, using the master hash index as an index into the master hash table.
5. The method of claim 3 , wherein retrieving the lock structure reference information, from the local hash table stored at the particular node, based on the hybrid hash index comprises retrieving the lock structure reference information, from the local hash table, using the hybrid hash index as an index into the local hash table.
6. The method of claim 1 , wherein storing lock structure reference information, from the set of hash buckets, in the second local hash table on the second node comprises storing the lock structure reference information at hash indexes, that index into the second local hash table, that have high bits that are the set of bits that represent the particular index number.
7. The method of claim 1 , wherein the particular resource is a particular block of data.
8. The method of claim 1 , wherein the multi-node system is a distributed database management system.
9. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more processors, cause:
calculating a hybrid hash index based on one or more hash functions and an identifier associated with a lock structure for a particular resource in a database that is managed by a multi-node system running on a cluster of nodes;
wherein a portion of the hybrid hash index comprises a master hash table index that indexes into a master hash table maintained for the multi-node system;
based, at least in part, on the portion of the hybrid hash index that comprises the master hash table index, determining that a particular node of the cluster of nodes is a lock master for the lock structure;
adding, into a local hash table at a position indicated by the hybrid hash index, lock structure reference information for the lock structure;
wherein the local hash table is stored at the particular node; and
after adding the lock structure reference information to the local hash table:
identifying the particular node as the lock master for the lock structure by retrieving, from the master hash table based on the hybrid hash index, identifying information for the particular node;
in response to identifying the particular node as the lock master for the lock structure, retrieving the lock structure reference information, from the local hash table stored at the particular node, based on the hybrid hash index; and
accessing the lock structure based on the lock structure reference information;
detecting a condition that requires reconfiguring lock masters for the cluster of nodes;
in response to detecting the condition that requires reconfiguring lock masters for the cluster of nodes:
reassigning an assigned lock master, for a set of lock structures that are mapped to a particular bucket of the master hash table, from a first node of the cluster of nodes to a second node of the cluster of nodes;
wherein the particular bucket is indexed, within the master hash table, by a particular index number represented by a set of bits;
wherein reassigning the set of lock structures comprises:
identifying a set of hash buckets, in a first local hash table on the first node, that are indexed by hash indexes whose high bits are the set of bits that represent the particular index number;
in response to identifying the set of hash buckets: storing lock structure reference information, from the set of hash buckets, in a second local hash table on the second node.
10. The one or more non-transitory computer-readable media of claim 9 , wherein calculating the hybrid hash index comprises calculating the hybrid hash index based on a single hybrid hash function.
11. The one or more non-transitory computer-readable media of claim 9 , wherein calculating the hybrid hash index comprises:
calculating a master hash index, based on the identifier associated with the lock structure and a master hash function of the one or more hash functions, that indexes into the master hash table;
calculating a local component number based on the identifier associated with the lock structure and a second hash function of the one or more hash functions;
concatenating the master hash index and at least a portion of the local component number to generate the hybrid hash index.
12. The one or more non-transitory computer-readable media of claim 11 , wherein retrieving, from the master hash table based on the hybrid hash index, the identifying information for the particular node comprises retrieving the identifying information for the particular node, from the master hash table, using the master hash index as an index into the master hash table.
13. The one or more non-transitory computer-readable media of claim 11 , wherein retrieving the lock structure reference information, from the local hash table stored at the particular node, based on the hybrid hash index comprises retrieving the lock structure reference information, from the local hash table, using the hybrid hash index as an index into the local hash table.
14. The one or more non-transitory computer-readable media of claim 9 , wherein storing lock structure reference information, from the set of hash buckets, in the second local hash table on the second node comprises storing the lock structure reference information at hash indexes, that index into the second local hash table, that have high bits that are the set of bits that represent the particular index number.
15. The one or more non-transitory computer-readable media of claim 9 , wherein the particular resource is a particular block of data.
16. The one or more non-transitory computer-readable media of claim 9 , wherein the multi-node system is a distributed database management system.