SYSTEM AND METHOD FOR PERFORMING JOIN OPERATIONS IN DIGITAL BACKUPS OF DATABASE APPLICATIONS
A method and system for searching joinable tables in digital backups of database applications for performing join operations. The method includes receiving a database table to perform a join operation on, wherein the received database table is included in the digital backups; searching for joinable columns across at least a set of tables included in the digital backups, wherein a joinable column shares at least a set of column values with a column in the received database table; and joining, upon a user selection, a joinable database table having a joinable column with the received database table.
1 . A method for searching joinable tables in digital backups of database applications for performing join operations, comprising:
receiving a database table to perform a join operation on, wherein the received database table is included in the digital backups;
searching for joinable columns across at least a set of tables included in the digital backups, wherein a joinable column shares at least a set of column values with a column in the received database table; and
joining, upon a user selection, a joinable database table having a joinable column with the received database table.
2 . The method of claim 1 , further comprising:
joining each table having a joinable column with the received database table.
3 . The method of claim 1 , wherein the joinable database table and the received database table are of different database applications.
4 . The method of claim 3 , wherein a database application is at least any one of: a NoSQL database application and a SQL database application.
5 . The method of claim 1 , wherein digital backups of each the joinable database table and the received database table include any one of: normalized columns values, a category of each column, and searchable values.
6 . The method of claim 5 , further comprising:
determining the category of each column by prompting a large language model (LLM) with at least a set of values of a column; and
saving original column names in the metadata of the digital backups.
7 . The method of claim 6 , further comprising:
normalizing column values of database tables stored in the digital backups, wherein the normalization includes applying a normalization function selected based on a category of a respective column;
saving the normalized column values in digital backups; and
saving a reverse normalization function in metadata of the digital backups.
8 . The method of claim 5 , further comprising:
generating searchable values by applying a MinHash function on normalized values in each column of each database table in the digital backups; and
calculating a set of ‘n’ minimal hash values on all rows of each database table, wherein the parameter ‘n’ is preconfigured.
9 . The method of claim 8 , wherein searching for joinable columns further comprises:
searching for columns having a number of ‘k’ identical searchable values; wherein ‘k’ is a number smaller than the number of ‘n’.
10 . The method of claim 8 , wherein searching for joinable columns further comprises:
identifying candidate joinable columns based on a hash map; and
determining if each candidate joinable column includes a set of ‘k’ identical searchable values; wherein ‘k’ is a number smaller than the number of ‘n’.
11 . The method of claim 1 , further comprising:
storing in the digital backups the joined table.
12 . The method of claim 1 , wherein the database applications are hosted on one or more virtual machines.
13 . The method of claim 1 , wherein the digital backups include a plurality of backup files, wherein a backup file includes a plurality of rows stored in a file having a column-oriented format.
14 . A non-transitory computer-readable medium storing a set of instructions for searching joinable tables in digital backups of database applications for performing join operations, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the device to:
receive a database table to perform a join operation on, wherein the received database table is included in the digital backups;
search for joinable columns across at least a set of tables included in the digital backups, wherein a joinable column shares at least a set of column values with a column in the received database table; and
join, upon a user selection, a joinable database table having a joinable column with the received database table.
15 . A system for searching joinable tables in digital backups of database applications for performing join operations comprising:
one or more processors configured to:
receive a database table to perform a join operation on, wherein the received database table is included in the digital backups;
search for joinable columns across at least a set of tables included in the digital backups, wherein a joinable column shares at least a set of column values with a column in the received database table; and
join, upon a user selection, a joinable database table having a joinable column with the received database table.
16 . The system of claim 15 , wherein the one or more processors are further configured to:
join each table having a joinable column with the received database table.
17 . The system of claim 15 , wherein the joinable database table and the received database table are of different database applications.
18 . The system of claim 17 , wherein a database application is at least any one of: a NoSQL database application and a SQL database application.
19 . The system of claim 15 , wherein digital backups of each the joinable database table and the received database table include any one of:
normalized columns values, a category of each column, and searchable values.
20 . The system of claim 19 , wherein the one or more processors are further configured to:
determine the category of each column by prompting a large language model (LLM) with at least a set of values of a column; and
save original column names in the metadata of the digital backups.
21 . The system of claim 20 , wherein the one or more processors are further configured to:
normalize column values of database tables stored in the digital backups, wherein the normalization includes applying a normalization function selected based on a category of a respective column;
save the normalized column values in digital backups; and
save a reverse normalization function in metadata of the digital backups.
22 . The system of claim 19 , wherein the one or more processors are further configured to:
generate searchable values by applying a MinHash function on normalized values in each column of each database table in the digital backups; and
calculate a set of ‘n’ minimal hash values on all rows of each database table, wherein the parameter n′ is preconfigured.
23 . The system of claim 22 , wherein the one or more processors, when searching for joinable columns, are configured to:
search for columns having a number of k′ identical searchable values; wherein k′ is a number smaller than the number of n′.
24 . The system of claim 22 , wherein the one or more processors, when searching for joinable columns, are configured to:
identify candidate joinable columns based on a hash map; and
determine if each candidate joinable column includes a set of k′ identical searchable values; wherein k′ is a number smaller than the number of n′.
25 . The system of claim 15 , wherein the one or more processors are further configured to:
store in the digital backups the joined table.
26 . The system of claim 15 , wherein the database applications are hosted on one or more virtual machines.
27 . The system of claim 15 , wherein the digital backups include a plurality of backup files, a backup file includes a plurality of rows stored in a file having a column-oriented format.