Cloud data backup storage manager
View Patent ↗Techniques for cloud data backup are disclosed. Example methods may store backup data from a client computer on portions of data storage media associated with peer computers. In some example embodiments, a file may be encoded into segments using an error-tolerant encoding scheme, and the segments may be stored on peer computers. In some example embodiments, individual segments may be stored on more than one peer machine and/or an individual peer machine may not receive all of the segments associated with the file.
1. A method of managing data back up at a node comprising:
identifying at least one file to be backed up from the node, wherein the node includes a local storage partition securely accessible by the node and a cloud storage partition that is accessible by a plurality of peer computers;
encoding the file into K file segments, wherein K is an integer greater than one;
requesting at least R * K storage blocks on a cloud storage partition comprised of the plurality of peer computers, wherein R is a redundancy factor; and
transmitting segments from the node to the plurality of peer computers such that each of the K file segments is stored on at least R peer computers.
2. The method of claim 1 , wherein the node is associated with a unique client identifier, the file is associated with a unique file identifier, and segments are associated with unique segment identifiers on an individual basis.
3. The method of claim 1 , wherein at least one of the segments does not comprise the file in its entirety.
4. The method of claim 3 , wherein less than all of the K segments associated with the file are transmitted to at least one of the peer computers.
5. The method of claim 1 , further comprising:
querying the peer computers to identify segments associated with the file stored on any of the peer computers;
receiving at least one of the segments associated with the file; and
reconstructing a copy of the file from the received at least one segments.
6. The method of claim 1 , further comprising:
querying the peer computers to identify segments associated with the file stored on any of the peer computers; and
if a number of any one of the K segments stored on the peer computers is less than R, transmitting the one of the K segments to one or more of the peer computers such that each of the K segments is stored on R peer computers.
7. The method of claim 1 , further comprising:
encrypting at least one of the segments prior to transmission to the peer computers.
8. The method of claim 1 , wherein the file comprises a plurality of data words including a plurality of bits; and wherein an individual segment includes B bits of an individual data word, wherein B is an integer greater than or equal to one and less than K.
9. The method of claim 1 , wherein encoding the file into the K file segments includes
encoding the file using an error detection and correction code, and
dividing the file into the K file segments such that the file is reconstructed from any K-1 of the file segments.
10. The method of claim 1 , wherein encoding the file into K file segments includes
encoding the file using an error detection and correction code, and
dividing the file into the K file segments such that, for an integer N, the file is reconstructed from any K-N of the file segments.
11. The method of claim 10 , further comprising:
querying the peer computers to identify segments associated with the file stored on any of the peer computers;
receiving at least one of the segments associated with the file; and
if S segments of the K segments are received, wherein S is an integer greater than or equal to one and less than K,
reconstructing at least one missing segment from the received segments, and
transmitting the reconstructed missing segment to at least one of the peer computers such that each of the K segments is stored on R peer computers.
12. The method of claim 1 , wherein individual segments comprise an error-detecting code.
13. The method of claim 12 , further comprising:
querying the peer computers to identify segments associated with the file stored on any of the peer computers;
receiving at least one segment associated with the file;
checking the received at least one segment for error-containing segments using said error-detecting code; and
if any error-containing segments are identified,
reconstructing at least one of the error-containing segments from at least one error-free segment, and
replacing the at least one error-containing segment with an error-free copy on at least one of the peer computers.
14. An apparatus for managing data backup at a node comprising:
a client computer associated with a unique client identifier, wherein the client computer includes a local storage partition securely accessible by the client computer and a cloud storage partition that is accessible by a plurality of peer computers, the client computer being configured to:
identify a file for backup from the client computer, the file being associated with a unique file identifier;
separate the file into K file segments, wherein K is an integer greater than one, wherein individual segments are associated with unique segment identifiers, and wherein no single segment comprises the complete file;
transmit the segments to a cloud storage partition comprised of the plurality of peer computers such that no single peer computer receives all K segments associated with the file and such that individual segments are received by at least R peer computers, wherein R is a redundancy factor;
receive from the peer computers at least some of the segments associated with the file; and
reconstruct the file, based upon at least the unique file identifier and the unique segment identifier, from the received segments.
15. The apparatus of claim 14 , wherein the client computer is configured to encrypt at least one of the segments.
16. The apparatus of claim 14 , wherein the client computer is configured to separate the file into K file segments by encoding individual bytes of the file into 12-bit Hamming-type code.
17. The apparatus of claim 14 , wherein the client is configured to request at least R*K storage blocks on the peer computers.
18. The apparatus of claim 17 , wherein the client is configured to request the at least R*K storage blocks on the peer computers from a cloud storage manager; and wherein the client is configured to receive the identities of the peer computers from the cloud storage manager.
19. The apparatus of claim 14 , wherein at least one segment comprises an error-detecting code.
20. A storage medium comprising machine-readable instructions stored thereon, which, when executed by one or more processing units, operatively enable a computing platform to:
identify a file stored locally on a client computer node for backup, wherein the node includes a local storage partition securely accessible by the node and a cloud storage partition that is accessible by a plurality of peer computers;
create a plurality of file segments from the file, wherein individual segments comprise less than the complete file, and wherein individual segments comprise an error-detecting code; and
transmit individual segments from the node to a cloud storage partition comprised of the plurality of peer computers such that individual segments are stored on at least R peer computers, wherein R is a redundancy factor, and wherein S of the segments are transmitted to individual peer computers, wherein S is an integer greater than or equal to one and less than K.
21. The storage medium of claim 20 , wherein the storage medium further comprises machine-readable instructions enabling the computing platform to:
query the peer computers to ascertain which, if any, of the segments is held by each of the peer computers; and
for any one of the segments held by one or more R peer computers, transmit the segment to additional peer computers such that the segment is held by R peer computers.
22. The storage medium of claim 20 , wherein the storage medium further comprises machine-readable instructions enabling the computing platform to:
request storage blocks on the peer computers;
wherein the plurality of file segments comprises K file segments, wherein K is an integer greater than one; and
wherein at least R*K storage blocks are requested.
23. The storage medium of claim 20 , wherein the storage medium further comprises machine-readable instructions enabling the computing platform to:
request the storage block on the peer computers from a cloud storage manager; and
receive, from the cloud manager, identities of the peer computers associated with the storage blocks.