Granted Patent B2

US 8,799,244 · App. 13/462,861 · Granted Aug 5, 2014

Abstract

Claims (52)

1. An apparatus, comprising:

a processor;

a memory; and

an interface to connect the processor, the memory, and a set of logics, the set of logics comprising:

a parse scope logic configured to determine a set of de-duplication rules shared between the apparatus and a second, different apparatus, where the apparatus and the second apparatus are both part of a de-duplication environment in which an object is to be de-duplicated, where the parse scope logic is configured to determine the set of de-duplication rules by transmitting de-duplication rule data to the second apparatus and by receiving return de-duplication rule data from the second apparatus; and

a de-duplication logic configured to de-duplicate the object as controlled by the set of shared de-duplication rules,

where the de-duplication rule data includes first information about a context associated with an object to be de-duplicated, second information identifying a previously negotiated shared parse scope associated with an object, third information identifying how an object is to be parsed, fourth information identifying an action to be performed after an object is parsed, fifth information identifying one or more of, a delta hash function, and a delta storage schema for an object, or sixth information identifying balancing rules for balancing performance and reduction, and

where the return de-duplication rule data includes an acceptance notification that identifies that the second apparatus is aware of a dialect identified by the de-duplication rule data and agrees to use the dialect, an update notification that identifies that the second apparatus was not aware of the dialect, that the second apparatus agrees to use the dialect, and that the second apparatus has been updated as a function of receiving the de-duplication rule data, a counteroffer notification comprising an alternative dialect data, where the alternative dialect data comprises one or more of, a rule, a hash of a rule, a rule identifier, a plurality of rules, a hash of the plurality of rules, and a plurality of rule identifiers, where the alternative dialect data is associated with an alternate dialect in use by the second apparatus, a request notification that identifies a hash of an item in the dialect for which a corresponding actual item is requested by the second apparatus, or an acknowledgement notification that identifies that the second apparatus will participate in the shared de-duplication action as controlled by a persisted parse scope previously negotiated for the object.

2. The apparatus of claim 1 , where de-duplication rule data describes one or more of, how a layered parser is to be configured, how an object is to be sub-divided, how an object is to be hashed, how an object is to be indexed, how an object is to be compressed, how an object is to be encrypted, and an acknowledgement of agreement on a de-duplication rule.

3. The apparatus of claim 1 , where the parse scope logic is configured to nest the de-duplication rule by data reducing the de-duplication rule data.

4. The apparatus of claim 1 , comprising a persistence logic configured to persist the set of de-duplication rules in one or more of, the apparatus, the second apparatus, and the object.

5. The apparatus of claim 4 , where the persistence logic is configured to selectively control the second apparatus to accept a set of de-duplication rules associated with the apparatus, where the persistence logic is configured to selectively control the apparatus to accept a set of de-duplication rules associated with the second apparatus, and where the persistence logic is configured to selectively control both the apparatus and the second apparatus to accept a set of de-duplication rules previously negotiated for the object.

6. A non-transitory computer readable medium storing computer executable instructions that when executed by a computer control the computer to perform a method, the method comprising:

upon determining that a first de-duplication logic and a second de-duplication logic are to participate in a shared de-duplication action for an object,

controlling the first de-duplication logic to establish a shared parse scope with the second de-duplication logic, by:

controlling the first de-duplication logic to provide dialect data to the second de-duplication logic, and

receiving feedback data from the second de-duplication logic, the feedback data being produced in response to the dialect data,

the dialect data comprising one or more of, a rule, a hash of a rule, a rule identifier, a plurality of rules, a hash of the plurality of rules, and a plurality of rule identifiers, where the rule comprises one or more of: first information about a context associated with an object to be de-duplicated, second information identifying a previously negotiated shared parse scope associated with an object, third information identifying how an object is to be parsed; fourth information identifying an action to be performed after an object is parsed; fifth information identifying one or more of, a delta hash function, and a delta storage schema for an object; and sixth information identifying balancing rules for balancing performance and reduction;

where the feedback data comprising one or more of: an acceptance notification that identifies that the second de-duplication logic is aware of the dialect identified by the dialect data and agrees to use the dialect identified by the dialect data, an update notification that identifies that the second de-duplication logic was not aware of the dialect identified by the dialect data, that the second de-duplication logic agrees to use the dialect identified by the dialect data, and that the second de-duplication logic has been updated as a function of receiving the dialect data, a counteroffer notification comprising an alternative dialect data, where the alternative dialect data comprises one or more of, a rule, a hash of a rule, a rule identifier, a plurality of rules, a hash of the plurality of rules, and a plurality of rule identifiers, where the alternative dialect data is associated with an alternate dialect in use by the second de-duplication logic, a request notification that identifies a hash of an item in the dialect for which a corresponding actual item is requested by the second de-duplication logic, and an acknowledgement notification that identifies that the second de-duplication logic will participate in the shared de-duplication action as controlled by a persisted parse scope previously negotiated for the object

and

persisting the shared parse scope.

7. The computer readable medium of claim 6 , the method comprising:

controlling the first de-duplication logic to perform a portion of the shared de-duplication action on the object as controlled by the shared parse scope.

8. The computer readable medium of claim 6 , the method comprising:

upon receiving an acknowledgement notification, establishing the persisted parse scope as the shared parse scope;

upon receiving an acceptance notification, establishing the dialect as the shared parse scope;

upon receiving an update notification, establishing the dialect as the shared parse scope; and

upon receiving a request notification, providing the corresponding actual item requested by the second de-duplication logic.

9. The computer readable medium of claim 6 , the method comprising:

upon receiving a counteroffer notification:

providing a dominance notification that identifies that the first de-duplication logic and the second de-duplication logic are to use the dialect identified by the dialect data;

providing an acceptance notification that identifies that the first de-duplication logic is aware of the alternate dialect identified by the alternate dialect data and agrees to use the alternate dialect identified by the alternate dialect data;

providing an update notification that identifies that the first de-duplication logic was not aware of the alternate dialect identified by the alternate dialect data, that the first de-duplication logic agrees to use the alternate dialect identified by the alternate dialect data, and that the first de-duplication logic has been updated as a function of receiving the alternate dialect data;

providing a request notification that identifies a hash of a rule associated with the alternate dialect for which a corresponding actual rule is requested by the first de-duplication logic; and

providing an acknowledgement notification that identifies that the first de-duplication logic will participate in the shared de-duplication action as controlled by a persisted parse scope previously negotiated for the object.

10. The computer readable medium of claim 6 , where the shared de-duplication action comprises data reducing the object from a first non-reduced state to a second reduced state.

11. The computer readable medium of claim 10 , the method comprising persisting the shared parse scope in one or more of, the object in the first non-reduced state, the object in the second reduced state, the first de-duplication logic, and the second de-duplication logic.

12. The computer readable medium of claim 11 , where persisting the shared parse scope comprises storing one or more of, a rule, a hash of a rule, a plurality of rules, a hash of the plurality of rules, a parse parameter, and context data concerning parsing performed according to the shared parse scope, and where establishing the shared parse scope with the second de-duplication logic can be based, at least in part, on a hash of a persisted shared parse scope.

13. The computer readable medium of claim 6 , where the shared parse scope comprises one or more de-duplication rules.

14. The computer readable medium of claim 6 , where the third information comprises one or more of, partitioning parameters comprising fixed, variable, or delimiter based partitioning parameters, hash function identifiers, and parse layering order parameters, where the partitioning parameters can be configured at one or more of, a file scope level, an object scope level, and a sub-object scope level.

15. The computer readable medium of claim 6 , where the fourth information comprises one or more of, information concerning compressing a reduced object, information concerning encrypting a reduced object, information concerning verifying the existence of an object, and information concerning transmitting an object.

16. The computer readable medium of claim 6 , where the shared de-duplication action is a security related action that includes determining a hash without transmitting the object or adding the object to a persistent data de-duplication data store.

17. The computer readable medium of claim 6 , the method comprising controlling the first de-duplication logic and the second de-duplication logic to accept a dialect associated with a replication event,

where the replication event comprises distribution of an updated parse scope comprising one or more of, a rule, a hash of a rule, a rule identifier, a plurality of rules, a hash of the plurality of rules, and a plurality of rule identifiers; and

where the replication event occurs independent of an intended communication between de-duplication logics.

18. A computer, comprising:

a first component configured to negotiate a set of rules by which two de duplication logics agree to be bound with respect to performing a de-duplication action on an object;

a second component configured to perform the de-duplication action as controlled by the set of rules; and

a third component configured to persist the set of rules to control future de-duplication actions associated with the object,

where the set of rules include rules describing how a layered parser is to be configured, how an object is to be sub-divided, how an object is to be hashed, how an object is to be indexed, how an object is to be compressed, how an object is to be encrypted, and an acknowledgement of agreement on a de-duplication rule,

where the set of rules includes first information about a context associated with an object to be de-duplicated, second information identifying a previously negotiated shared parse scope associated with an object, third information identifying how an object is to be parsed, fourth information identifying an action to be performed after an object is parsed, fifth information identifying one or more of, a delta hash function, and a delta storage schema for an object, and sixth information identifying balancing rules for balancing performance and reduction, and

where the first component receives, in response to providing the set of rules, return de-duplication rule data that includes an acceptance notification that identifies that a second apparatus to which the set of rules was provided is aware of a dialect identified by the de-duplication rule data and agrees to use the dialect, an update notification that identifies that the second apparatus was not aware of the dialect, that the second apparatus agrees to use the dialect, and that the second apparatus has been updated as a function of receiving the de-duplication rule data, a counteroffer notification comprising an alternative dialect data, where the alternative dialect data comprises one or more of, a rule, a hash of a rule, a rule identifier, a plurality of rules, a hash of the plurality of rules, and a plurality of rule identifiers where the alternative dialect data is associated with an alternate dialect in use by the second apparatus, a request notification that identifies a hash of an item in the dialect for which a corresponding actual item is requested by the second apparatus, or an acknowledgement notification that identifies that the second apparatus will participate in the shared de-duplication action as controlled by a persisted parse.

Assignments (11)