Eliminating duplicate data in storage systems with boot consolidation
View Patent ↗Technologies for eliminating duplicate data provisions within a storage system supporting boot consolidation can efficiently identify duplicate data provisions within a data storage system and eliminate duplication by remapping duplicate provisions to point to the same physical storage space. Signatures of provisions within a storage system may be calculated and compared. Matching, or collisions, within the list of provision signatures can indicate candidate provisions for de-duplication. De-duplication territories may be provided as an indirect mapping mechanism in support of the remapping of duplicated provisions. Access statistics associated with provisions within a storage system may be collected. Access statistics can support the scheduling of de-duplication processes. Data de-duplication can support substantial storage space consolidation and significantly improve caching efficiency within a data storage system.
1. A method for removing duplicate provisions within a data storage system, the method comprising:
providing a plurality of storage territories, each of the storage territories being subdivided into one or more provisions within a physical storage of the data storage system;
providing a de-duplication territory;
identifying a plurality of duplicate provisions among the one or more provisions within the physical storage of the data storage system;
allocating a new provision within the de-duplication territory for each of the plurality of duplicate provisions for consolidating the plurality of duplicate provisions;
remapping each of the plurality of duplicate provisions to one of the newly allocated provisions within the de-duplication territory; and
consolidating the newly allocated provisions associated with each of the plurality of duplicate provisions within the de-duplication territory to point to a single physical storage provision within the physical storage, wherein the plurality of duplicate provisions are indirectly mapped to the single physical storage provision within the physical storage through the de-duplication territory.
2. The method of claim 1 , further comprising writing to consolidated provisions as a writable snapshot branched off of a read-only snapshot.
3. The method of claim 1 , wherein identifying duplicate provisions occurs in response to detecting a software upgrade.
4. The method of claim 1 , wherein identifying duplicate provisions comprises determining provision signatures and identifying provisions with matching signatures.
5. The method of claim 4 , further comprising verifying provisions with matching signatures by performing a full comparison of the provisions.
6. The method of claim 1 , further comprising scheduling provisions for duplicate identification and consolidation based on a least-accessed-first ordering.
7. The method of claim 1 , further comprising scheduling provisions for duplicate identification and consolidation during periods of reduced storage access.
8. A non-transitory computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
provide a plurality of storage territories, each of the storage territories being subdivided into one or more provisions within a physical storage;
provide a de-duplication territory;
identify a plurality of duplicate provisions among the one or more provisions within the physical storage;
allocate a new provision within the de-duplication territory for each of the plurality of duplicate provisions for consolidating the plurality of duplicate provisions;
remap each of the plurality of duplicate provisions to one of the newly allocated provisions within the de-duplication territory; and
consolidate the newly allocated provisions associated with each of the plurality of duplicate provisions within the de-duplication territory to point to a single physical storage provision within the physical storage, wherein the plurality of duplicate provisions are indirectly mapped to the single physical storage provision within the physical storage through the de-duplication territory.
9. The non-transitory computer storage medium of claim 8 , having further computer-executable instructions stored thereon which, when executed by the computer system, cause the computer system to write to consolidated provisions as a writable snapshot branched off of a read-only snapshot.
10. The non-transitory computer storage medium of claim 8 , wherein identifying duplicate provisions occurs in response to detecting a software upgrade.
11. The non-transitory computer storage medium of claim 8 , wherein identifying duplicate provisions comprises determining provision signatures and identifying provisions with matching signatures.
12. The non-transitory computer storage medium of claim 11 , having further computer-executable instructions stored thereon which, when executed by the computer system, cause the computer system to verify provisions with matching signatures by performing a full comparison of the provisions.
13. The non-transitory computer storage medium of claim 8 , having further computer-executable instructions stored thereon which, when executed by the computer system, cause the computer system to schedule provisions for duplicate identification and consolidation based on a least-accessed-first ordering.
14. The non-transitory computer storage medium of claim 8 , having further computer-executable instructions stored thereon which, when executed by the computer system, cause the computer system to schedule provisions for duplicate identification and consolidation during periods of reduced storage access.
15. A data storage system comprising:
a storage server;
a physical storage associated with the storage server;
a processing unit associated with the storage server;
and one or more modules for execution on the processing unit, operable to:
provide a plurality of storage territories, each of the storage territories being subdivided into one or more provisions within the physical storage;
provide a de-duplication territory;
identify a plurality of duplicate provisions among the one or more provisions within the physical storage;
allocate a new provision within the de-duplication territory for each of the plurality of duplicate provisions for consolidating the plurality of duplicate provisions;
remap each of the plurality of duplicate provisions to one of the newly allocated provisions within the de-duplication territory; and
consolidate the newly allocated provisions associated with each of the plurality of duplicate provisions within the de-duplication territory to point to a single physical storage provision within the physical storage, wherein the plurality of duplicate provisions are indirectly mapped to the single physical storage provision within the physical storage through the de-duplication territory.
16. The data storage system of claim 15 , wherein the one or more modules for execution on the processing unit are further operable to write to consolidated provisions as a writable snapshot branched off of a read-only snapshot.
17. The data storage system of claim 15 , wherein identifying duplicate provisions occurs in response to detecting a software upgrade.
18. The data storage system of claim 15 , wherein identifying duplicate provisions comprises determining provision signatures and identifying provisions with matching signatures.
19. The data storage system of claim 18 , wherein the one or more modules for execution on the processing unit are further operable to verify provisions with matching signatures by performing a full comparison of the provisions.
20. The data storage system of claim 15 , wherein the one or more modules for execution on the processing unit are further operable to schedule provisions for duplicate identification and consolidation during periods of reduced storage access.