System and method for efficient episodic replication of data to cloud storage
View Patent ↗In one example, a method managing data includes transmitting backup format information to a storage platform, breaking data targeted for backup into data pieces, computing a unique fingerprint for each of the data pieces, and transmitting metadata concerning the data pieces to the storage platform, where the metadata includes the fingerprints. The method further includes receiving from the storage platform a list of any new fingerprints that do not correspond to any of the data pieces, and grouping data pieces together to form a jumbo data piece, where the data pieces that form the jumbo data piece are data pieces to which the new fingerprints correspond.
1. A method, comprising
transmitting, from a thin gateway connector at an enterprise premises, backup format information to a storage platform;
breaking data targeted for backup into data chunks, and the backup format information concerns a backup dataset that includes the data;
computing a fingerprint for each of the data chunks;
transmitting, from the thin gateway connector to a cloud storage gateway of the storage platform, chunk grouping metadata concerning the data chunks to the storage platform, wherein the chunk grouping metadata includes the fingerprints and also lists an order of the data chunks;
in response to transmission of the chunk grouping metadata to the cloud storage gateway, receiving, at the thin gateway connector from the storage platform, a list of any new fingerprints not included in the chunk grouping metadata and that do not correspond to any of the data chunks;
in response to receipt of the list from the storage platform, grouping data chunks together to form a single jumbo data chunk, wherein the data chunks that form the jumbo data chunk are only data chunks to which a new fingerprint respectively corresponds; and
transmitting, by the thin gateway connector, the single jumbo data chunk to the cloud storage gateway.
2. The method as recited in claim 1 , wherein the method is performed with only a single SSL socket connection between the thin gateway connector and the cloud storage gateway.
3. The method as recited in claim 1 , wherein the chunk grouping metadata identifies the data chunks to the cloud storage gateway.
4. The method as recited in claim 1 , wherein no data or metadata is persistently stored at the thin gateway connector.
5. The method as recited in claim 1 , wherein receipt of the list of new fingerprints received by the thin gateway connector from the cloud storage gateway constitutes receipt of a request from the cloud storage gateway that the thin gateway connector only upload respective data chunks that correspond to the new fingerprints.
6. The method as recited in claim 1 , wherein the thin gateway connector forms the single jumbo data chunk.
7. A non-transitory storage medium having stored therein computer-executable instructions which are executable by one or more hardware processors to perform operations comprising:
transmitting, from a thin gateway connector at an enterprise premises, backup format information to a storage platform;
breaking data targeted for backup into data chunks, and the backup format information concerns a backup dataset that includes the data;
computing a fingerprint for each of the data chunks;
transmitting, from the thin gateway connector to a cloud storage gateway of the storage platform, chunk grouping metadata concerning the data chunks to the storage platform, wherein the chunk grouping metadata includes the fingerprints and also lists an order of the data chunks;
in response to transmission of the chunk grouping metadata to the cloud storage gateway, receiving, at the thin gateway connector from the storage platform, a list of any new fingerprints not included in the chunk grouping metadata and that do not correspond to any of the data chunks;
in response to receipt of the list from the storage platform, grouping data chunks together to form a single jumbo data chunk, wherein the data chunks that form the jumbo data chunk are only data chunks to which a new fingerprint respectively corresponds; and
transmitting, by the thin gateway connector, the single jumbo data chunk to the cloud storage gateway.
8. The non-transitory storage medium as recited in claim 7 , wherein the operations are performed with only a single SSL socket connection between the thin gateway connector and the cloud storage gateway.
9. The non-transitory storage medium as recited in claim 7 , wherein the chunk grouping metadata identifies the data chunks to the cloud storage gateway.
10. The non-transitory storage medium as recited in claim 7 , wherein receipt of the list of new fingerprints received by the thin gateway connector from the cloud storage gateway constitutes receipt of a request from the cloud storage gateway that the thin gateway connector only upload respective data chunks that correspond to the new fingerprints.
11. The non-transitory storage medium as recited in claim 7 , wherein the single jumbo data chunk preserves information enabling the identification of boundaries for the data chunks.
12. The non-transitory storage medium as recited in claim 7 , wherein no data or metadata is persistently stored at the thin gateway connector.
13. The non-transitory storage medium as recited in claim 7 , wherein the thin gateway connector forms the single jumbo data chunk.
14. A system, comprising:
one or more hardware processors; and
a non-transitory storage medium having stored therein computer-executable instructions which are executable by the one or more hardware processors to perform operations comprising:
transmitting, from a thin gateway connector at an enterprise premises, backup format information to a storage platform;
breaking data targeted for backup into data chunks, and the backup format information concerns a backup dataset that includes the data;
computing a fingerprint for each of the data chunks;
transmitting, from the thin gateway connector to a cloud storage gateway of the storage platform, chunk grouping metadata concerning the data chunks to the storage platform, wherein the chunk grouping metadata includes the finger prints and also lists an order of the data chunks;
in response to transmission of the chunk grouping metadata to the cloud storage gateway, receiving, at the thin gateway connector from the storage platform, a list of any new fingerprints not included in the chunk grouping metadata and that do not correspond to any of the data chunks;
in response to receipt of the list from the storage platform, grouping data chunks together to form a single jumbo data chunk, wherein the data chunks that form the jumbo data chunk are only data chunks to which a new fingerprint respectively corresponds; and
transmitting, by the thin gateway connector, the single jumbo data chunk to the cloud storage gateway.
15. The system as recited in claim 14 , wherein the operations are performed with only a single SSL socket connection between the thin gateway connector and the cloud storage gateway.
16. The system as recited in claim 14 , wherein the chunk grouping metadata identifies the data chunks to the cloud storage gateway.
17. The system as recited in claim 14 , wherein receipt of the list of new fingerprints received by the thin gateway connector from the cloud storage gateway constitutes receipt of a request from the cloud storage gateway that the thin gateway connector only upload respective data chunks that correspond to the new fingerprints.
18. The system as recited in claim 14 , wherein the single jumbo data chunk preserves information enabling the identification of boundaries for the data chunks.
19. The system as recited in claim 14 , wherein no data or metadata is persistently stored at the thin gateway connector.
20. The system as recited in claim 14 , wherein the thin gateway connector forms the single jumbo data chunk.