Controlling configurable variable data reduction
View Patent ↗Example apparatus, methods, and computers control configurable, variable data reduction. One example method includes identifying data reduction controlling attributes in an object to be data reduced by a configurable variable data reducer. The attributes provide information upon which decisions concerning whether and/or how to data reduce the object can be based. The example method also includes controlling a configurable variable data reducer to selectively data reduce the object based, at least in part, on the data reduction controlling attributes. The control exercised can determine whether, where, when, and/or how data reduction will proceed.
1. An apparatus, comprising:
a processor;
a memory; and
an interface to connect the processor, memory, and a set of logics, the set of logics comprising:
two or more boundary logics configured to determine a set of chunk boundaries for an object to be data reduced by a data reducer, wherein the chunk boundaries are two or more of run based, delimiter based, rules based, or data dependent; and
a control logic to control the data reducer to chunk the object based, at least in part, on the set of chunk boundaries,
where the two or more boundary logics are constrained to place boundaries in locations that satisfy one or more of, a minimum chunk size, and a maximum chunk size.
2. The apparatus of claim 1 , where the two or more boundary logics determine the set of chunk boundaries based, at least in part, on an attribute of the object.
3. The apparatus of claim 2 , the attribute comprising one or more of, an object size, an object type, an object format, an object organization, an object entropy, an object context, and a data reduction control field associated with the object.
4. The apparatus of claim 1 , the two or more boundary logics comprising two or more of:
a run boundary logic configured to identify a set of run based chunk boundaries for the object and to add the set of run based chunk boundaries to the set of chunk boundaries;
a delimiter boundary logic configured to identify a set of delimiter based chunk boundaries for the object and to add the set of delimiter based chunk boundaries to the set of chunk boundaries;
a rules based boundary logic configured to identify a set of rules based chunk boundaries for the object and to add the set of rules based chunk boundaries to the set of chunk boundaries; and
a data dependent boundary logic configured to identify a set of data dependent chunk boundaries for the object and to add the set of data dependent chunk boundaries to the set of chunk boundaries.
5. The apparatus of claim 4 , where the run boundary logic is configured to identify a run based chunk boundary in response to identifying a set of repeating characters, where the set of repeating characters includes at least a threshold number of repeating characters.
6. The apparatus of claim 5 , where the run boundary logic is configured to create a run based chunk boundary at the end of the set of repeating characters.
7. The apparatus of claim 5 , where the run boundary logic is configured to perform one or more of, run length encoding of members of the set of repeating characters, and controlling the data reducer to run length encode members of the set of repeating characters.
8. The apparatus of claim 4 , where the delimiter boundary logic is configured to identify a delimiter based chunk boundary in response to identifying a character the object.
9. The apparatus of claim 8 , where the delimiter boundary logic is configured to create a delimiter based chunk boundary after the delimiter character.
10. The apparatus of claim 4 , where the rages based boundary logic is configured to identify a rules based chunk boundary in response to examining the contents of the object and determining that a rule associated with the object has been satisfied.
11. The apparatus of claim 1 , where the set of chunk boundaries contains one or more of, chunk boundaries identified by only one boundary logic and chunk boundaries identified by two or more boundary logics.
12. The apparatus of claim 1 , where the control logic is configured to control the data reducer to perform data reduction on the object as one or more of, a backup process, an archive process, a near-line process, and a primary process.
13. The apparatus of claim 1 , where the control logic is configured to control the data reducer to perform the data reduction as one or more of, a monolithic process, a distributed process, a local process, a remote process, an in-line process, a post-processing process, a single pass, and multiple passes.
14. The apparatus of claim 1 , where the control logic is configured to control the data reducer to perform one or more of, a fixed block length data deduplication, a variable block length data deduplication, a roiling hash based deduplication, and a single file duplicate data reduction.
15. The apparatus of claim 1 , comprising a deduplication eligibility logic configured to determine whether to control the data reducer to data reduce the object.
16. A computer, comprising:
a first component configured to produce a set of predicted data reduction results for an object to be data reduced, where a predicted data reduction result is a function of a data reduction technique to be applied based on an attribute of the object, wherein the attribute is used to determine at least a set of chunk boundaries for the object in locations that satisfy one or more of, a minimum chunk size, and a maximum chunk size, wherein the chunk boundaries are two or more of run based, delimiter based, rules based, or data dependent; and
a second component configured to control data reduction of the object as a function of the set of predicted data reduction results.