Techniques for reducing backup read access in differential backup of databases
A method and system for reducing deduplication in database backup are provided. The method includes selecting a first primary key of a database; extracting a plurality of row groups from the database based on the selected first primary key; storing the plurality of row groups arranged according to the first primary key in a file having a column-oriented data storage format as a database backup at a first time; reading at least a portion of the database backup at a second time to detect a plurality of changes; determining a position of each row corresponding to a change of the plurality of changes; selecting a second primary key of the database as a primary key; and generating a database backup of the database at the second time based on the selected second primary key.
1 . A method for reducing deduplication in database backup, comprising:
selecting a first primary key of a database;
extracting a plurality of row groups from the database based on the selected first primary key;
storing the plurality of row groups arranged according to the first primary key in one or more files having a column-oriented data storage format as a database backup at a first time;
reading at least a portion of the database backup at a second time to detect a plurality of changes wherein detecting the plurality of changes includes reading, at the second time, a plurality of rows from the database by querying the database based on utilization of a range of values of the first primary key, and comparing a hash value of the plurality of rows to a corresponding value stored in the database backup;
determining a position of each row corresponding to a change of the plurality of changes;
selecting a second primary key of the database as a primary key, when utilizing the second primary key generates a smaller number of row groups than utilizing the first primary key, wherein the smaller number of row groups includes a change in at least a row; and
generating a database backup of the database at the second time based on the selected second primary key, wherein prior to generating the database backup at the second time, the one or more files of the database backup at the first time are rearranged based on the selected second primary key.
2 . The method of claim 1 , further comprising:
selecting an index key as the first primary key.
3 . The method of claim 1 , further comprising:
selecting a first key and a second key of the database as the first primary key; and
selecting the first key and a third key of the database as the second primary key.
4 . The method of claim 1 , further comprising:
selecting a first portion of rows of the database;
generating a first plurality of row groups based on extracting the first portion of rows based on the first primary key;
generating a second plurality of row groups based on extracting the first portion of rows based on the second primary key; and
selecting the second primary key in response to determining that a number of row groups of the second plurality of row groups which include a change is smaller than the number of row groups of the first plurality of row groups which include a change.
5 . The method of claim 1 , further comprising:
periodically initiating a check to determine if querying the database utilizing the second primary key results in smaller differential backups than querying the database utilizing the first primary key.
6 . The method of claim 1 , further comprising:
extracting the plurality of row groups by querying the database based on a range of the first primary key.
7 . The method of claim 6 , further comprising:
extracting the plurality of row groups at a second time by querying the database based on a range of the second primary key.
8 . The method of claim 1 , wherein selecting the second primary key comprises selecting a unique key that ensures all values in a column or combination of columns are unique across records.
9 . The method of claim 1 , further comprising:
generating a differential backup at a third time based on the second primary key.
10 . A non-transitory computer-readable medium storing a set of instructions for reducing deduplication in database backup, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the device to:
select a first primary key of a database;
extract a plurality of row groups from the database based on the selected first primary key;
store the plurality of row groups arranged according to the first primary key in one or more files having a column-oriented data storage format as a database backup at a first time;
read at least a portion of the database backup at a second time to detect a plurality of changes, wherein detecting the plurality of changes includes reading, at the second time, a plurality of rows from the database by querying the database based on utilization of a range of values of the first primary key, and comparing a hash value of the plurality of rows to a corresponding value stored in the database backup;
determine a position of each row corresponding to a change of the plurality of changes;
select a second primary key of the database as a primary key, when utilizing the second primary key generates a smaller number of row groups than utilizing the first primary key, wherein the smaller number of row groups includes a change in at least a row; and
generate a database backup of the database at the second time based on the selected second primary key, wherein prior to generating the database backup at the second time, the one or more files of the database backup at the first time are rearranged based on the selected second primary key.
11 . A system for reducing deduplication in database backup comprising:
one or more processors configured to:
select a first primary key of a database;
extract a plurality of row groups from the database based on the selected first primary key;
store the plurality of row groups arranged according to the first primary key in a file one or more files having a column-oriented data storage format as a database backup at a first time;
read at least a portion of the database backup at a second time to detect a plurality of changes, wherein detecting the plurality of changes includes reading, at the second time, a plurality of rows from the database by querying the database based on utilization of a range of values of the first primary key, and comparing a hash value of the plurality of rows to a corresponding value stored in the database backup;
determine a position of each row corresponding to a change of the plurality of changes;
select a second primary key of the database as a primary key, when utilizing the second primary key generates a smaller number of row groups than utilizing the first primary key, wherein the smaller number of row groups includes a change in at least a row; and
generate a database backup of the database at the second time based on the selected second primary key, wherein prior to generating the database backup at the second time, the one or more files of the database backup at the first time are rearranged based on the selected second primary key.
12 . The system of claim 11 , wherein the one or more processors are further configured to:
select an index key as the first primary key.
13 . The system of claim 11 , wherein the one or more processors are further configured to:
select a first key and a second key of the database as the first primary key; and
select the first key and a third key of the database as the second primary key.
14 . The system of claim 11 , wherein the one or more processors are further configured to:
select a first portion of rows of the database;
generate a first plurality of row groups based on extracting the first portion of rows based on the first primary key;
generate a second plurality of row groups based on extracting the first portion of rows based on the second primary key; and
select the second primary key in response to determining that a number of row groups of the second plurality of row groups which include a change is smaller than the number of row groups of the first plurality of row groups which include a change.
15 . The system of claim 11 , wherein the one or more processors are further configured to:
periodically initiate a check to determine if querying the database utilizing the second primary key results in smaller differential backups than querying the database utilizing the first primary key.
16 . The system of claim 11 , wherein the one or more processors are further configured to:
extract the plurality of row groups by querying the database based on a range of the first primary key.
17 . The system of claim 16 , wherein the one or more processors are further configured to:
extract the plurality of row groups at a second time by querying the database based on a range of the second primary key.
18 . The system of claim 11 , wherein the one or more processors are further configured to:
select the second primary key as a unique key that ensures all values in a column or combination of columns are unique across records.
19 . The system of claim 11 , wherein the one or more processors are further configured to:
generate a differential backup at a third time based on the second primary key.