Determining content-dependent deltas between data sectors
In one implementation, a method includes identifying a first content-dependent feature associated with a data sector. The method further includes determining a baseline data sector associated with the data sector. The method further includes determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector. The method further includes providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.
1. A storage system comprising:
a plurality of embedded storage devices; and
a plurality of storage system controllers, operatively coupled to the plurality embedded storage devices via a bus, the plurality of storage system controllers to:
identify a first content-dependent feature associated with a data sector;
determine a baseline data sector associated with the data sector;
assign a deduplicated data object associated with the data sector to be the baseline data sector;
determine a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector; and
provide the content-dependent delta and an indicator to the baseline data sector for storage on the plurality of storage devices.
2. The storage system of claim 1 , wherein the baseline data sector is a data sector with a highest number of deduplicates.
3. The storage system of claim 1 , wherein the data sector is a fixed size.
4. The storage system of claim 1 , wherein to determine the baseline data sector, the plurality of storage system controllers is to:
identify a snapshot data sector associated with the data sector; and
assign the snapshot data sector to be the baseline data sector.
5. The storage system of claim 1 , the plurality of storage system controllers further to:
identify a plurality of data files stored on the plurality of embedded storage devices;
determine a content type of each of the plurality of data files; and
identify a first subset of the plurality of data files, wherein each data file of the first subset corresponds to a first content type.
6. The storage system of claim 5 , the plurality of storage system controllers further to:
identify the first content-dependent feature of a first data file of the first subset of data files and a corresponding second content-dependent feature of the baseline data sector, wherein the first content-dependent feature and the second content-dependent feature are dependent upon a first content type; and
deduplicate the first data file in view of the content-dependent delta being less than or equal to a delta threshold.
7. The storage system of claim 5 , the plurality of storage system controllers further to:
receive a request to locate a second data file from the plurality of embedded storage devices;
determine, based on metadata associated with the second file, that the second data file corresponds to a second content type;
search a second subset of data files, wherein each data file of the second subset corresponds to the second content type;
locate the second data file in the second subset; and
provide the located second data file.
8. A storage system comprising:
a plurality of storage devices; and
a storage system controller, operatively coupled to the plurality of storage devices via a bus, the storage system controller to:
identify a first content-dependent feature associated with a data sector;
determine a baseline data sector associated with the data sector;
assign a snapshot data sector associated with the data sector to be the baseline data sector;
determine a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector; and
provide the content-dependent delta and an indicator to the baseline data sector for storage on the plurality of storage devices.
9. The storage system of claim 8 , wherein to determine the baseline data sector, the storage system controller is to:
identify a deduplicated data object associated with the data sector; and
assign the deduplicated data object to be the baseline data sector.
10. The storage system of claim 8 , wherein the data sector is a fixed size.
11. The storage system of claim 8 , wherein the data sector is a variable size.
12. The storage system of claim 8 , the storage system controller further to:
identify a plurality of data files stored on the plurality of storage devices;
determine a content type of each of the plurality of data files; and
identify a first subset of the plurality of data files, wherein each data file of the first subset corresponds to a first content type.
13. The storage system of claim 12 , the storage system controller further to:
identify the first content-dependent feature of a first data file of the first subset of data files and a corresponding second content-dependent feature of the baseline data sector, wherein the first content-dependent feature and the second content-dependent feature are dependent upon a first content type; and
deduplicate the first data file in view of the content-dependent delta being less than or equal to a delta threshold.
14. The storage system of claim 12 , the storage system controller further to:
receive a request to locate a second data file from the plurality of storage devices;
determine, based on metadata associated with the second file, that the second data file corresponds to a second content type;
search a second subset of data files, wherein each data file of the second subset corresponds to the second content type;
locate the second data file in the second subset; and
provide the located second data file.
15. A method comprising:
identifying a first content-dependent feature associated with a data sector;
determining a baseline data sector associated with the data sector;
assign a deduplicated data object associated with the data sector to be the baseline data sector;
determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector; and
providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.
16. The method of claim 15 , wherein the baseline data sector is a data sector with a highest number of deduplicates.
17. The method of claim 15 , wherein determining the baseline data sector comprises:
identifying a snapshot data sector associated with the data sector; and
assigning the snapshot data sector to be the baseline data sector.
18. The method of claim 15 , the method further comprising:
identifying a plurality of data files stored on the plurality of storage devices;
determining a content type of each of the plurality of data files; and
identifying a first subset of the plurality of data files, wherein each data file of the first subset corresponds to a first content type.
19. The method of claim 18 , the method further comprising:
identifying the first content-dependent feature of a first data file of the first subset of data files and a corresponding second content-dependent feature of the baseline data sector, wherein the first content-dependent feature and the second content-dependent feature are dependent upon a first content type; and
deduplicating the first data file in view of the content-dependent delta being less than or equal to a delta threshold.
20. The method of claim 18 , the method further comprising:
receiving a request to locate a second data file from the plurality of storage devices;
determining, based on metadata associated with the second file, that the second data file corresponds to a second content type;
searching a second subset of data files, wherein each data file of the second subset corresponds to the second content type;
locating the second data file in the second subset; and
providing the located second data file.