Method and system for sharing pre-calculated fingerprints and data chunks amongst storage systems on a cloud local area network

1. A method for implementing data chunk transfer by a protection storage system (PSS), the method comprising:

performing an optimization protocol by maintaining a fingerprint hit probability (FHP), for each PSS of a set of PSSs to obtain a set of FHPs, wherein the FHP is a ratio of a number of query responses indicating a fingerprint match to a number query of responses indicating no fingerprint match;

selecting, based on the FHPs, a first subset of PSSs of the set of PSSs;

receiving, by a first protection storage system (PSS), a first backup request from a client, wherein the first backup request comprises a first fingerprint, wherein the first fingerprint is a digital signature that uniquely identifies a first data chunk, wherein the client and the first PSS are connected via a wide area network (WAN);

generating a first fingerprint query comprising the first fingerprint;

transmitting the first fingerprint query to the first subset of PSSs, wherein the first PSS and the first subset of PSSs are connected via a local area network (LAN);

obtaining, in response to the first fingerprint query and from a second PSS of the first subset of PSSs, the first data chunk; and

updating, in response to obtaining the first data chunk from the second PSS, a fingerprint hit probability (FHP) associated with the second PSS,

wherein the performing of the optimization protocol further comprises:

receiving a set of backup requests from the client, wherein a cardinality of the set of backup requests meets a predefined request count criterion;

maintaining, based on the set of backup requests, the FHP for each PSS of the set of PSSs to obtain the set of FHPs; and

assessing each FHP of the set of FHPs to select the first subset of PSSs.

2. The method of claim 1 , wherein assessing each FHP of the set of FHPs to select the first subset of PSSs, comprises:

determining that the each FHP meets a predefined hit probability criterion;

identifying, based on the determining, each PSS associated with the each FHP as a fingerprint query recipient; and

including, based on the identifying, the each PSS into the first subset of PSSs.

3. The method of claim 1 , further comprising:

prior to generating the first fingerprint query:

determining that the first fingerprint is not stored in a local fingerprint database,

wherein, in response to the determining, the first fingerprint query is generated.

4. The method of claim 1 , further comprising:

after obtaining the first data chunk:

generating, in a local data chunk database, a first database entry comprising a new storage ID and the first data chunk;

generating, in a local fingerprint database, a second database entry comprising the first fingerprint and the new storage ID; and

issuing, to the client, a backup response indicating that a data chunk corresponding to the first fingerprint is already stored.

5. The method of claim 1 , further comprising:

after obtaining the first data chunk:

receiving, from the client, a second backup request comprising a second fingerprint;

generating a second fingerprint query comprising the second fingerprint;

transmitting the second fingerprint query to a second subset of PSSs of the set of PSSs;

receiving a set of query responses from the second subset of PSSs;

determining, based on the set of query responses, that none of the PSS of the second subset of PSSs contains the second fingerprint; and

obtaining, in response to the determining, a second data chunk associated with the second fingerprint from the client.

6. The method of claim 1 , further comprising:

after obtaining the first data chunk:

receiving, from the client, a second backup request comprising a second fingerprint;

determining that the second fingerprint is stored in a local fingerprint database; and

issuing, in response to the determining and to the client, a backup response indicating that a data chunk corresponding to the second fingerprint is already stored.

7. A system, comprising:

a set of protection storage systems (PSSs); and

a client operatively connected to the set of PSSs by way of a wide area network (WAN),

wherein a first PSS of the set of PSSs comprises a non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by the first PSS, enables the first PSS to perform a method comprising:

performing an optimization protocol by maintaining a fingerprint hit probability (FHP), for each PSS of the set of PSSs to obtain a set of FHPs, wherein the FHP is a ratio of a number of query responses indicating a fingerprint match to a number query of responses indicating no fingerprint match;

selecting, based on the FHPs, a first subset of PSSs of the set of PSSs;

receiving, by the first PSS, a first backup request from the client, wherein the first backup request comprises a first fingerprint, wherein the first fingerprint is a digital signature that uniquely identifies a first data chunk;

generating a first fingerprint query comprising the first fingerprint;

transmitting the first fingerprint query to the first subset of PSSs of the set of PSSs, wherein the first PSS and the first subset of PSSs are connected via a local area network (LAN);

obtaining, in response to the first fingerprint query and from a second PSS of the first subset of PSSs, the first data chunk; and

updating, in response to obtaining the first data chunk from the second PSS, a fingerprint hit probability (FHP) associated with the second PSS,

wherein the performing of the optimization protocol further comprises:

receiving a set of backup requests from the client, wherein a cardinality of the set of backup requests meets a predefined request count criterion;

maintaining, based on the set of backup requests, the FHP for each PSS of the set of PSSs to obtain the set of FHPs; and

assessing each FHP of the set of FHPs to select the first subset of PSSs.

8. The system of claim 7 , wherein the system is deployed as a data deduplication system.

9. A non-transitory computer readable medium (CRM) comprising computer readable program code, which when executed by a computer processor, enables the computer processor to perform a method for implementing data chunk transfer by a Protection Storage System (PSS), the method comprising:

performing an optimization protocol by maintaining a fingerprint hit probability (FHP), for each PSS of a set of PSSs to obtain a set of FHPs, wherein the FHP is a ratio of a number of query responses indicating a fingerprint match to a number query of responses indicating no fingerprint match;

selecting, based on the FHPs, a first subset of PSSs of the set of PSSs;

receiving, by a first protection storage system (PSS), a first backup request from a client, wherein the first backup request comprises a first fingerprint, wherein the first fingerprint is a digital signature that uniquely identifies a first data chunk, wherein the client and the first PSS are connected via a wide area network (WAN);

generating a first fingerprint query comprising the first fingerprint;

transmitting the first fingerprint query to the first subset of PSSs, wherein the first PSS and the first subset of PSSs are connected via a local area network (LAN);

obtaining, in response to the first fingerprint query and from a second PSS of the first subset of PSSs, the first data chunk; and

updating, in response to obtaining the first data chunk from the second PSS, a fingerprint hit probability (FHP) associated with the second PSS,

wherein the performing of the optimization protocol further comprises:

receiving a set of backup requests from the client, wherein a cardinality of the set of backup requests meets a predefined request count criterion;

maintaining, based on the set of backup requests, the FHP for each PSS of the set of PSSs to obtain the set of FHPs; and

assessing each FHP of the set of FHPs to select the first subset of PSSs.

10. The non-transitory CRM of claim 9 , wherein the method further comprises:

determining that the each FHP meets a predefined hit probability criterion;

identifying, based on the determining, each PSS associated with the each FHP as a fingerprint query recipient; and

including, based on the identifying, the each PSS into the first subset of PSSs.

11. The non-transitory CRM of claim 9 , wherein the method further comprises:

prior to generating the first fingerprint query:

determining that the first fingerprint is not stored in a local fingerprint database,

wherein, in response to the determining, the first fingerprint query is generated.

12. The non-transitory CRM of claim 9 , wherein the method further comprises:

after obtaining the first data chunk:

generating, in a local data chunk database, a first database entry comprising a new storage ID and the first data chunk;

generating, in a local fingerprint database, a second database entry comprising the first fingerprint and the new storage ID; and

issuing, to the client, a backup response indicating that a data chunk corresponding to the first fingerprint is already stored.

13. The non-transitory CRM of claim 9 , wherein the method further comprises:

after obtaining the first data chunk:

receiving, from the client, a second backup request comprising a second fingerprint;

generating a second fingerprint query comprising the second fingerprint;

transmitting the second fingerprint query to a second subset of PSSs of the set of PSSs;

receiving a set of query responses from the second subset of PSSs;

determining, based on the set of query responses, that none of the PSS of the second subset of PSSs contains the second fingerprint; and

obtaining, in response to the determining, a second data chunk associated with the second fingerprint from the client.

14. The non-transitory CRM of claim 9 , wherein the method further comprises:

after obtaining the first data chunk:

receiving, from the client, a second backup request comprising a second fingerprint;

determining that the second fingerprint is stored in a local fingerprint database; and

issuing, in response to the determining and to the client, a backup response indicating that a data chunk corresponding to the second fingerprint is already stored.