Multi cloud transactional storage for availability and security
One example method includes exposing a block storage which is distributed across a group of multiple sites, receiving a primary write request that identifies data to be stored, separating data identified in the primary write request into multiple data pieces, encoding the data pieces by creating multiple new blocks of data based on the multiple data pieces, where the data pieces are encoded in such a way that when a sufficient number, but fewer than all, of the multiple new blocks of data are retrieved, the data identified in the write request is recoverable by decoding, and writing the new blocks of data to different respective sites of the group, where writing of the new blocks of data is performed in conjunction with a plurality of secondary write requests, each of which corresponds to one of the new blocks of data.
1. A system, comprising:
one or more hardware processors; and
a computer readable storage medium having stored therein instructions which are executable by the one or more hardware processors to perform operations comprising:
exposing a block storage which is distributed across a group of multiple sites;
receiving a primary write request that identifies data to be stored;
separating data identified in the primary write request into ‘n’ data pieces, where ‘n’≥2;
encoding the ‘n’ data pieces by creating multiple new blocks of data, and each of the new blocks of data comprises a different respective combination of the ‘n’ data pieces, where the data pieces are encoded in such a way that when a sufficient number, but fewer than all, of the multiple new blocks of data are retrieved, the data identified in the write request is recoverable with a key that was used to encode the ‘n’ data pieces; and
writing the new blocks of data to different respective sites of the group, where writing of the new blocks of data is performed in conjunction with a plurality of secondary write requests, each of which corresponds to one of the new blocks of data.
2. The system as recited in claim 1 , wherein encoding the data pieces comprises using a polynomial to determine respective values of the data pieces.
3. The system as recited in claim 1 , wherein the group of multiple sites comprises a virtual active/active block storage configuration.
4. The system as recited in claim 1 , wherein a single site that stores a new block of data has no information concerning the data identified in the write request.
5. The system as recited in claim 1 , wherein decoding commences after a sufficient number, but fewer than all, of the new blocks of data are received at a decoding entity.
6. The system as recited in claim 1 , wherein decoding the data identified in the write request comprises using a polynomial and the received new blocks of data to derive the data pieces.
7. The system as recited in claim 1 , wherein the operations further comprise:
receiving a primary read command concerning the stored data;
translating the primary read command into multiple different secondary read commands; and
transmitting each of the secondary read commands to a respective site of the group of sites, and each of the secondary read commands refers to one or more of the new blocks of data created by the encoding process.
8. The system as recited in claim 7 , wherein the operations further comprise:
collectively receiving, from fewer than all of the sites, a sufficient number of the new blocks of data to enable decoding of the data identified in the primary read command, where the sufficient number of the new blocks of data is fewer than all of the new blocks of data; and
using the received new blocks of data, and a polynomial, to decode the data identified in the write request.
9. The system as recited in claim 1 , wherein the operations further comprise blocking a read access command issued by one of the sites, while maintaining access to the new blocks of data from the other sites.
10. The system as recited in claim 1 , wherein a write operation associated with the write request is considered complete when enough of the ‘n’ corresponding data pieces have been written so as to enable, when read out, decoding of the ‘x’ pieces of data.
11. The system as recited in claim 1 , wherein a single site that stores a new block of data has no information concerning the data that was encoded.
12. The system as recited in claim 1 , wherein each of the new blocks of data has a different respective value generated by the encoding.
13. The system as recited in claim 12 , wherein the value of one of the new blocks of data provides no information as to the value of another of the new blocks of data.
14. The system as recited in claim 1 , wherein each of the new blocks of data has a different respective value generated by the encoding, and when the sufficient number of the new data blocks is retrieved, the respective values of remaining data blocks that have not been retrieved are derivable from the respective values of the retrieved data blocks.
15. The system as recited in claim 14 , wherein the ‘n’ data pieces are decodable from the data block values with the key.
16. The system as recited in claim 1 , wherein the ‘n’ data pieces are encoded using a Reed-Solomon code.