Systems and methods for shard-based data indexing and retrieval in multi-key environments
Systems, computer program products, and methods are described herein for shard-based data indexing and retrieval in multi-key environments. The present disclosure is configured to streamline data management by utilizing a shard directory to assign unique shard key hashes to database records. This configuration allows for the efficient creation, addition, updating, and deletion of keys with minimal computational overhead. By leveraging a domain, key type, and key value, the system computes a compact shard key hash, facilitating targeted data retrieval within a specific shard. This eliminates the need to search across all shards, thereby enhancing performance. The system's instructions, when executed by a processing device, ensure that these operations preserve data integrity and optimize database interactions, leading to a more responsive and scalable data infrastructure.
1 . A system for shard-based data indexing and retrieval in multi-key environments, the system comprising:
at least one non-transitory storage device; and
at least one processor coupled to the at least one non-transitory storage device, wherein the at least one processor is configured to:
receive input data for the creation or addition of a key to a database record;
compute a shard key hash based on the input data;
access a shard directory within a sharded database environment to determine if the computed shard key hash exists within the shard directory;
perform a search operation by receiving a search query comprising a single domain and a single key type, locating a shard key hash in the shard directory corresponding to the single domain and the single key type, and executing a targeted search in the sharded database environment using the located shard key hash when the search query is associated with a single key;
update the database record by receiving updated data, locating an existing shard key hash from the shard directory;
modify the database record and the shard directory entry to reflect the updated data;
delete the database record by identifying the database record to be deleted, locating an associated shard key hash in the shard directory, and removing the database record and the corresponding shard directory entry; and
maintain each shard key hash as immutable with a single database record or a set of records sharing the single domain and the single key type.
2 . The system of claim 1 , wherein the shard key hash is a unique identifier derived from an integer hash of a combination of a domain, a key type, and a key value associated with the input data.
3 . The system of claim 2 , wherein the shard key hash is reduced to a predetermined number of digits to facilitate shard mapping.
4 . The system of claim 1 , wherein in response to the shard key hash not existing within the shard directory, create a new shard key hash entry in the shard directory and associating the new shard key hash with the input data.
5 . The system of claim 1 , wherein upon the shard key hash existing within the shard directory or after creating a new shard key hash entry, creating or updating the database record with the shard key hash to enable indexed storage of the database record within a corresponding shard of a sharded database environment.
6 . The system of claim 1 , further comprising reindexing the database record by receiving updated data, locating an existing shard key hash from the shard directory, and modifying the database record and the shard directory entry to reflect the reindexing.
7 . The system of claim 1 , wherein maintaining each shard key hash as immutable further comprises maintaining the sharded database environment integrity by ensuring that each shard key hash is immutable once set and is uniquely associated with the single database record or a set of records sharing the same domain and key type.
8 . A computer program product for shard-based data indexing and retrieval in multi-key environments the computer program product comprising a non-transitory computer-readable medium comprising code causing an apparatus to:
receive input data for the creation or addition of a key to a database record;
compute a shard key hash based on the input data;
access a shard directory within a sharded database environment to determine if the computed shard key hash exists within the shard directory;
perform a search operation by receiving a search query comprising a single domain and a single key type, locating a shard key hash in the shard directory corresponding to the single domain and the single key type, and executing a targeted search in the sharded database environment using the located shard key hash when the search query is associated with a single key;
update the database record by receiving updated data, locating an existing shard key hash from the shard directory;
modify the database record and the shard directory entry to reflect the updated data;
delete the database record by identifying the database record to be deleted, locating an associated shard key hash in the shard directory, and removing the database record and the corresponding shard directory entry; and
maintain each shard key hash as immutable with a single database record or a set of records sharing the single domain and the single key type.
9 . The computer program product of claim 8 , wherein the shard key hash is a unique identifier derived from an integer hash of a combination of a domain, a key type, and a key value associated with the input data.
10 . The computer program product of claim 9 , wherein the shard key hash is reduced to a predetermined number of digits to facilitate shard mapping.
11 . The computer program product of claim 8 , wherein in response to the shard key hash not existing within the shard directory, create a new shard key hash entry in the shard directory and associating the new shard key hash with the input data.
12 . The computer program product of claim 8 , wherein upon the shard key hash existing within the shard directory or after creating a new shard key hash entry, creating or updating the database record with the shard key hash to enable indexed storage of the database record within a corresponding shard of a sharded database environment.
13 . The computer program product of claim 8 , further comprising reindexing the database record by receiving updated data, locating an existing shard key hash from the shard directory, and modifying the database record and the shard directory entry to reflect the reindexing.
14 . The computer program product of claim 8 , wherein maintaining each shard key hash as immutable further comprises maintaining the sharded database environment integrity by ensuring that each shard key hash is immutable once set and is uniquely associated with the single database record or a set of records sharing the same domain and key type.
15 . A method for shard-based data indexing and retrieval in multi-key environments the method comprising:
providing a computing system comprising a computer processing device and a non-transitory computer readable medium, where the computer readable medium comprises configured computer program instruction code, such that when said instruction code is operated by said computer processing device, said computer processing device performs the following operations:
receiving input data for the creation or addition of a key to a database record;
computing a shard key hash based on the input data;
accessing a shard directory within a sharded database environment to determine if the computed shard key hash exists within the shard directory;
performing a search operation by receiving a search query comprising a single domain and a single key type, locating a shard key hash in the shard directory corresponding to the single domain and the single key type, and executing a targeted search in the sharded database environment using the located shard key hash when the search query is associated with a single key;
updating the database record by receiving updated data, locating an existing shard key hash from the shard directory;
modifying the database record and the shard directory entry to reflect the updated data;
deleting the database record by identifying the database record to be deleted, locating an associated shard key hash in the shard directory, and removing the database record and the corresponding shard directory entry; and
maintaining each shard key hash as immutable with a single database record or a set of records sharing the single domain and the single key type.
16 . The method of claim 15 , wherein the shard key hash is a unique identifier derived from an integer hash of a combination of a domain, a key type, and a key value associated with the input data.
17 . The method of claim 16 , wherein the shard key hash is reduced to a predetermined number of digits to facilitate shard mapping.
18 . The method of claim 15 , wherein in response to the shard key hash not existing within the shard directory, create a new shard key hash entry in the shard directory and associating the new shard key hash with the input data.
19 . The method of claim 15 , wherein upon the shard key hash existing within the shard directory or after creating a new shard key hash entry, creating or updating the database record with the shard key hash to enable indexed storage of the database record within a corresponding shard of a sharded database environment.
20 . The method of claim 15 , further comprising reindexing the database record by receiving updated data, locating an existing shard key hash from the shard directory, and modifying the database record and the shard directory entry to reflect the reindexing.