Prioritizing data deduplication
View Patent ↗An example method includes controlling a data de-duplication apparatus to arrange a de-duplication schedule based on the presence or absence of a replication indicator in an item to be de-duplicated. The method also includes selectively controlling the de-duplication schedule based on a replication priority. In one embodiment, the method includes, upon determining that a chunk of data is associated with a replication indicator, controlling the data de-duplication apparatus to schedule the chunk for de-duplication ahead of chunks not associated with a replication indicator. In one embodiment, the method also includes, upon determining that the chunk is associated with a replication priority, controlling the data de-duplication apparatus to schedule the chunk for de-duplication ahead of chunks of data not associated with a replication priority. The schedule location is based, at least in part, on the replication priority. The method also includes controlling de-duplication order based on the schedule.
1. A method for controlling a data de-duplication apparatus, comprising:
upon determining that a chunk of data to be de-duplicated by the data de-duplication apparatus is associated with a replication indicator located in the chunk of data, controlling the data de-duplication apparatus to schedule the chunk of data for de-duplication in a de-duplication schedule location ahead of chunks of data not associated with a replication indicator, where the schedule location is based, at least in part, on the replication indicator; and
upon determining that the chunk of data to be de-duplicated is not associated with a replication indicator located in the chunk of data, controlling the data de-duplication apparatus to schedule the chunk of data for de-duplication in a de-duplication schedule location behind chunks of data having associated replication indicators,
where the chunk of data is a portion of a piece of ingested data,
where the replication indicator controls whether the chunk of data is to be replicated, and
where the replication indicator controls the order in which chunks are deduplicated so that chunks that are going to be replicated are deduplicated before chunks that are not going to be replicated.
2. The method of claim 1 , comprising:
upon determining that the chunk of data to be de-duplicated by the data de-duplication apparatus is associated with a replication priority, controlling the data de-duplication apparatus to schedule the chunk of data for de-duplication in a de-duplication schedule location ahead of chunks of data not associated with a replication priority, where the schedule location is based, at least in part, on the replication priority, where the replication priority describes a relative order in which an item is to be replicated;
and
upon determining that the chunk of data to be de-duplicated is not associated with a replication priority, controlling the data de-duplication apparatus to schedule the chunk of data for de-duplication in a de-duplication schedule location behind chunks of data having associated replication priorities,
where the replication priority is located in the chunk of data or is located in metadata associated with the chunk of data.
3. The method of claim 2 , comprising:
controlling the data de-duplication apparatus to selectively associate replication indicators with items to be de-duplicated that are also to be replicated, where associating replication indicators with items to be de-duplicated includes adding a replication indicator to an item.
4. The method of claim 2 , comprising:
controlling the data de-duplication apparatus to selectively associate replication priorities with items to be replicated.
5. The method of claim 4 , where associating the replication priorities with items to be de-duplicated includes one or more of, adding a replication priority to an item, and adding a replication priority to metadata associated with an item.
6. The method of claim 5 , the item being one of, a virtual tape cartridge, and a file.
7. The method of claim 2 , comprising:
controlling the data de-duplication apparatus to ingest the item to be de-duplicated, where ingesting the item comprises sub-dividing the item into one or more chunks of data to be de-duplicated, and where status associated with a replication indicator associated with the item is maintained by the one or more chunks of data.
8. The method of claim 7 , where status associated with a replication priority associated with the item is maintained by the one or more chunks of data.
9. The method of claim 1 , comprising:
controlling the data de-duplication apparatus to produce a de-duplicated chunk of data from a chunk of data to be de-duplicated, where the de-duplicated chunk of data is selected in order according to the de-duplication schedule.
10. The method of claim 9 , comprising:
upon determining that a de-duplicated chunk of data is to be replicated, providing the de-duplicated chunk of data to a data replication apparatus.
11. The method of claim 10 , comprising:
upon determining that the de-duplicated chunk of data to be replicated by the data replication apparatus is associated with a replication priority, controlling the data replication apparatus to schedule the chunk of de-duplicated data for replication in a replication schedule location determined by the replication priority, where the replication priority describes a relative order in which an item is to be replicated.
12. The method of claim 11 , comprising:
controlling the data replication apparatus to produce a replicated chunk of data from a de-duplicated chunk of data, where the de-duplicated chunk of data is selected in order according to the replication schedule.
13. An apparatus for prioritizing chunks that are going to be replicated ahead of chunks that are not going to be replicated, comprising:
a processor;
a memory; and
an interface connecting the processor, the memory, and a set of logics, the set of logics comprising:
a de-duplication scheduling logic that manages a de-duplication schedule of chunks of data for de-duplication, where the de-duplication schedule is based, at least in part, on a de-duplication priority associated with a chunk; and
a de-duplication logic that de-duplicates chunks of data in an order determined by the de-duplication schedule, where the de-duplication scheduling logic manages the de-duplication schedule based, at least in part, on a replication indicator located in the chunk and a replication priority associated with the chunk, and
a replication logic that replicates a de-duplicated chunk of data produced by the de-duplication logic,
where the chunks of data are portions of a piece of ingested data, the portions being less than an entire file,
where the replication indicator controls whether an individual chunk of data is to be replicated,
where the replication indicator controls whether an individual chunk is to be replicated,
where the replication indicator is located in the chunk of data, and
where the replication indicator controls the order in which chunks are deduplicated so that chunks that are going to be replicated are deduplicated before chunks that are not going to be replicated.
14. The apparatus of claim 13 , the set of logics comprising:
a replication scheduling logic configured to manage a replication schedule of de-duplicated chunks of data to be replicated, where the replication schedule is based, at least in part, on a replication priority associated with a de-duplicated chunk, where the replication priority is located in the de-duplicated chunk of data or is located in metadata associated with the de-duplicated chunk of data, and where the replication priority describes a relative order in which an item is to be replicated.
15. The apparatus of claim 13 , the set of logics comprising:
a de-duplication priority logic configured to establish a de-duplication priority for an item, the item being one of, a virtual tape cartridge, and a file.
16. The apparatus of claim 15 , where the de-duplication priority is established based on whether the item is to be replicated.
17. The apparatus of claim 13 , the set of logics comprising:
a replication priority logic configured to establish a replication priority for an item, the item being one of, a virtual tape cartridge, and a file, where the replication priority describes a relative order in which an item is to be replicated.