Method and system for distributed data storage with enhanced security, resilience, and control
A method and system for encrypting and reconstructing data files, including related metadata, is disclosed. The method involves separately encrypting data and metadata as chaining processes and integrating a plurality of encryption/encoding techniques together with strategic storage distribution techniques and parsing techniques which results in the integrated benefits of the collection of techniques. As disclosed, the content data is separated from its metadata, encryption keys may be embedded in the metadata, and in a content data encryption chaining process, the method chunks, encrypts, shards, and stores content data and separately shards and stores metadata, and stored in a flexible, distributed, and efficient manner, at least in part to assure improved resiliency In addition, the processes are preferably implemented locally, including at the site of the content data or a proxy server.
1. A method for a processor to encrypt at least one computer file based on a combination of computational and theoretical cryptography, said computer file accessible at least on a local device, said computer file including a content data portion and an associated metadata portion, comprising the steps of:
generating a plurality of randomly generated encryption keys;
selecting at least one computer file for encryption;
parsing said content data portion of said computer file into a chain of content chunks, each said chunk assigned a chunk ID;
using computational cryptography, said computational cryptography includes use of one or more encryption algorithms and erasure coding, using said at least one encryption key per chunk, encrypting each of said content chunks;
using computational cryptography encoding and parsing each of said content chunks into a plurality of content shards;
using theoretical cryptography, said theoretical cryptography includes secret sharing methods and storing a secret in multiple shards, n, without use of an encryption key, encrypting said chunk IDs;
augmenting said metadata portion with said encrypted chunk IDs thereby forming an augmented metadata portion;
parsing said plurality of randomly generated encryption keys into a plurality of key shards;
using theoretical cryptography, encrypting said plurality of key shards;
adding said encrypted plurality of key shards into said augmented metadata portion;
parsing a subset of said metadata portion into a plurality of metadata shards;
encrypting said metadata shards;
delivering said plurality of content shards to at least a first storage location; and
delivering said plurality of metadata shards to at least a second storage location;
wherein said at least a first storage location differs from said at least a second storage location, said method is configured to protect stored data from brute force attacks, and said method is configured such that decryption requires knowledge of t out of n content shards, t out of n key shards, and t out of n metadata shards, where t and n are integers.
2. The method of claim 1 , where the encrypted key shards and said chunk IDs are separately stored.
3. The method of claim 1 , where the steps of parsing said at least one key into a plurality of key shards and encrypting said plurality of key shards is at least partially performed using Shamir's Secret Sharing Scheme (SSSS).
4. The method of claim 1 , where the computational cryptography portion of the method includes use of Reed-Solomon encoding.
5. The method of claim 1 , where the step of encrypting each of said content chunks includes use of AES-256.
6. The method of claim 1 , where said content data portion is fully encrypted before encoding.
7. The method of claim 1 , where at least one file attribute in said metadata portion is not encrypted.
8. The method of claim 1 , where at least some of said metadata portion is stored in a vault on said local device.
9. The method of claim 1 , where a number, n, of each of metadata storage, key storage, and data storage is configurable and each is greater than 2.
10. The method of claim 1 , where t is a number of required shards to reconstruct and n is a number of shards stored, parameters t and n of metadata shards, key shards, and data shards are each independently configurable, individually selectable by a user, and where t is an integer greater than 1 and n is n integer greater than t.