Systems and methods for secure file management via an aggregation of cloud storage services
The present disclosure describes systems and methods for aggregation and management of cloud storage among a plurality of providers via file fragmenting to provide increased reliability and security. In one implementation, fragments or blocks may be distributed among a plurality of cloud storage providers, such that no provider retains a complete copy of a file. Accordingly, even if an individual service is compromised, a malicious actor cannot access the data. In another implementation, fragments may be duplicated and distributed to multiple providers, such that loss of communications to any one provider does not result in inability to access the data. This implementation may be combined with error correction techniques to allow recovery, even with loss of multiple providers. File synchronization may also be faster in these implementations by dividing reading and writing operations among multiple providers.
1. A method for file management, comprising:
receiving a request associated with a file comprised of a plurality of fragments, the request associated with a client device and comprising metadata of the file, and wherein the file comprises a bitstream having an ordered set of bits, and the plurality of fragments comprise a rotated bitstream relative to the bitstream of the file;
distributing the plurality of fragments of the file across a plurality of cloud storage providers by:
determining a first cloud storage provider of the plurality of cloud storage providers;
selecting a first fragment of the plurality of fragments for storing on the first cloud storage provider of the plurality of cloud storage providers;
determining a first account at the first cloud storage provider for storage of the file;
storing the first fragment of a plurality of fragments of the file at the first cloud storage provider in association with the first account;
determining a second cloud storage provider of the plurality of cloud storage providers;
selecting a second fragment of the plurality of fragments for storing on a second cloud storage provider of the plurality of cloud storage providers;
determining a second account at the second cloud storage provider for storage of the file;
storing the second fragment of the plurality of fragments of the file at the second cloud storage provider in association with the second account; and
storing a first association between the first fragment and the first cloud storage provider and a second association between the second fragment and the second cloud storage provider.
2. The method of claim 1 , wherein the distribution of the file across the plurality of cloud storage providers, including the selection of the first fragment is based on a file distribution policy associated with the file, the client device, or a user of the client device.
3. The method of claim 2 , wherein the file distribution policy is identified based on the metadata of the file.
4. The method of claim 3 , wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on an amount of free space associated with the first account at the first cloud storage provider or the second account at the second cloud storage provider.
5. The method of claim 3 , wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on storage availability or utilization associated with one or more of multiple storage providers.
6. The method of claim 1 , wherein the first association comprises an identification of the first account and the second association comprises an identification of the second account.
7. The method of claim 1 , wherein the metadata of the file comprises a name or location of the file.
8. A system for secure file management in an aggregation of cloud storage services, comprising:
a processor;
a non-transitory computer readable medium, comprising instructions for:
receiving a request associated with a file comprised of a plurality of fragments, the request associated with a client device and comprising metadata of the file, and wherein the file comprises a bitstream having an ordered set of bits, and the plurality of fragments comprise a rotated bitstream relative to the bitstream of the file;
distributing the plurality of fragments of the file across a plurality of cloud storage providers by:
determining a first cloud storage provider of the plurality of cloud storage providers;
selecting a first fragment of the plurality of fragments for storing on the first cloud storage provider of the plurality of cloud storage providers;
determining a first account at the first cloud storage provider for storage of the file;
storing the first fragment of a plurality of fragments of the file at the first cloud storage provider in association with the first account;
determining a second cloud storage provider of the plurality of cloud storage providers;
selecting a second fragment of the plurality of fragments for storing on a second cloud storage provider of the plurality of cloud storage providers;
determining a second account at the second cloud storage provider for storage of the file;
storing the second fragment of the plurality of fragments of the file at the second cloud storage provider in association with the second account; and
storing a first association between the first fragment and the first cloud storage provider and a second association between the second fragment and the second cloud storage provider.
9. The system of claim 8 , wherein the distribution of the file across the plurality of cloud storage providers, including the selection of the first fragment is based on a file distribution policy associated with the file, the client device, or a user of the client device.
10. The system of claim 9 , wherein the file distribution policy is identified based on the metadata of the file.
11. The system of claim 10 , wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on an amount of free space associated with the first account at the first cloud storage provider or the second account at the second cloud storage provider.
12. The system of claim 10 , wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on storage availability or utilization associated with one or more of multiple storage providers.
13. The system of claim 8 , wherein the first association comprises an identification of the first account and the second association comprises an identification of the second account.
14. The system of claim 8 , wherein the metadata of the file comprises a name or location of the file.
15. A non-transitory computer readable medium, comprising instructions for:
receiving a request associated with a file comprised of a plurality of fragments, the request associated with a client device and comprising metadata of the file, and wherein the file comprises a bitstream having an ordered set of bits, and the plurality of fragments comprise a rotated bitstream relative to the bitstream of the file;
distributing the plurality of fragments of the file across a plurality of cloud storage providers by:
determining a first cloud storage provider of the plurality of cloud storage providers;
selecting a first fragment of the plurality of fragments for storing on the first cloud storage provider of the plurality of cloud storage providers;
determining a first account at the first cloud storage provider for storage of the file;
storing the first fragment of a plurality of fragments of the file at the first cloud storage provider in association with the first account;
determining a second cloud storage provider of the plurality of cloud storage providers;
selecting a second fragment of the plurality of fragments for storing on a second cloud storage provider of the plurality of cloud storage providers;
determining a second account at the second cloud storage provider for storage of the file;
storing the second fragment of the plurality of fragments of the file at the second cloud storage provider in association with the second account; and
storing a first association between the first fragment and the first cloud storage provider and a second association between the second fragment and the second cloud storage provider.
16. The non-transitory computer readable medium of claim 1 , wherein the distribution of the file across the plurality of cloud storage providers, including the selection of the first fragment is based on a file distribution policy associated with the file, the client device, or a user of the client device.
17. The non-transitory computer readable medium of claim 16 , wherein the file distribution policy is identified based on the metadata of the file.
18. The non-transitory computer readable medium of claim 17 , Wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on an amount of free space associated with the first account at the first cloud storage provider or the second account at the second cloud storage provider.
19. The non-transitory computer readable medium of claim 17 , wherein the distribution of the file across the plurality of cloud storage providers based on the file distribution policy is based on storage availability or utilization associated with one or more of multiple storage providers.
20. The non-transitory computer readable medium of claim 15 , wherein the first association comprises an identification of the first account and the second association comprises an identification of the second account.
21. The non-transitory computer readable medium of claim 15 , wherein the metadata of the file comprises a name or location of the file.