Infinite versioning by automatic coalescing
Embodiments disclosed herein provide systems, methods, and computer readable media for infinite versioning by automatic coalescing. In a particular embodiment, a method provides determining an age range for a plurality of data versions stored in a secondary data repository and identifying first data versions of the plurality of data versions that are within the age range. The method further provides determining a compaction ratio for the first data versions and compacting the first data versions based on the compaction ratio.
1. A method of versioning data by automatic coalescing, the method comprising:
determining one or more age ranges for a plurality of data versions stored in a secondary data repository, each data version in the plurality of data versions storing a set of changes made to a set of data items since a previous data version;
for a first age range of the one or more age ranges:
determining first data versions of the plurality of data versions that are within the first age range;
determining a first compaction ratio defining a number of data versions to be compacted into one single data version based on the first age range;
compacting the first data versions to one or more first compacted data versions, each first compacted data version including a change among the set of changes included in the number of data versions of the first data versions; and
storing the one or more first compacted data versions.
2. The method of claim 1 , wherein compacting the first data versions to one or more first compacted data versions comprises:
generating a list of logical block address ranges for the one or more first compacted data versions, each logical block address mapping to a physical location in the secondary data repository.
3. The method of claim 1 , wherein compacting the first data versions based on the first compaction ratio comprises:
grouping the first data versions into one or more sequential data version groups each including the number of data versions of the first data versions; and
for each sequential data version group: removing overwritten changes of the number of data versions.
4. The method of claim 1 , further comprising determining a second compaction ratio for a second age range of the one or more age ranges, wherein the second age range is older than the first age range and the second compaction ratio is greater than the first compaction ratio.
5. The method of claim 1 , wherein the data items are stored in a primary data repository separate from the secondary data repository.
6. The method of claim 1 , further comprising determining second data versions that are within a second age range, wherein the second age range is older than the first age range and the second data versions are more than the first data versions.
7. A non-transitory computer readable storage medium having instructions stored thereon for versioning data by automatic coalescing, the instructions, when executed by a data compaction system, directing the data compaction system to perform operations, the operations including at least:
determining one or more age ranges for a plurality of data versions stored in a secondary data repository, each data version in the plurality of data versions storing a set of changes made to a set of data items since a previous data version;
for a first age range of the one or more age ranges:
determining first data versions of the plurality of data versions that are within the first age range;
determining a first compaction ratio defining a number of data versions to be compacted into one single data version based on the first age range;
compacting the first data versions to one or more first compacted data versions, each first compacted data version including a change among the set of changes included in the number of data versions of the first data versions; and
storing the one or more first compacted data versions.
8. The medium of claim 7 , wherein compacting the first data versions to one or more first compacted data versions comprises:
generating a list of logical block address ranges for the one or more first compacted data versions, each logical block address mapping to a physical location in the secondary data repository.
9. The medium of claim 7 , wherein compacting the first data versions based on the first compaction ratio comprises:
grouping the first data versions into one or more sequential data version groups each including the number of data versions of the first data versions; and
for each sequential data version group: removing overwritten changes of the number of data versions.
10. The medium of claim 7 , wherein the operations further comprise determining a second compaction ratio for a second age range of the one or more age ranges, wherein the second age range is older than the first age range and the second compaction ratio is greater than the first compaction ratio.
11. The medium of claim 7 , wherein the data items are stored in a primary data repository separate from the secondary data repository.
12. The medium of claim 7 , wherein the operations further comprise determining second data versions that are within a second age range, wherein the second age range is older than the first age range and the second data versions are more than the first data versions.
13. A data compaction system for versioning data by automatic coalescing, the data compaction system comprising:
a computer processor; and
one or more non-transitory computer readable storage media including instructions which, when executed by the computer processor, cause the data compaction system to perform operations, the operations comprising, at least:
determining one or more age ranges for a plurality of data versions stored in a secondary data repository, each data version in the plurality of data versions storing a set of changes made to a set of data items since a previous data version;
for a first age range of the one or more age ranges:
determining first data versions of the plurality of data versions that are within the first age range;
determining a first compaction ratio defining a number of data versions to be compacted into one single data version based on the first age range;
compacting the first data versions to one or more first compacted data versions, each first compacted data version including a change among the set of changes included in the number of data versions of the first data versions; and
storing the one or more first compacted data versions.
14. The data compaction system of claim 13 , wherein compacting the first data versions to one or more first compacted data versions comprises:
generating a list of logical block address ranges for the one or more first compacted data versions, each logical block address mapping to a physical location in the secondary data repository.
15. The data compaction system of claim 13 , wherein compacting the first data versions based on the first compaction ratio comprises:
grouping the first data versions into one or more sequential data version groups each including the number of data versions of the first data versions; and
for each sequential data version group: removing overwritten changes of the number of data versions.
16. The data compaction system of claim 13 , wherein the operations further comprise determining a second compaction ratio for a second age range of the one or more age ranges, wherein the second age range is older than the first age range and the second compaction ratio is greater than the first compaction ratio.
17. The data compaction system of claim 13 , wherein the data items are stored in a primary data repository separate from the secondary data repository.
18. The data compaction system of claim 13 , wherein the operations further comprise determining second data versions that are within a second age range, wherein the second age range is older than the first age range and the second data versions are more than the first data versions.