System and method for storing data blocks in a volume of data
A method for storing data blocks within a volume of data is disclosed. The method involves creating a frame header to identify encoding and a size of a data block, storing the frame header and the data block in the volume of data, and creating an identifying sequence to identify if a data block is encoded or unencoded and storing the data block with the identifying sequence and with a frame header if the data block is encoded and storing the data block without the identifying sequence and without a frame header if the data block is unencoded.
1. A method for operating a storage system to store data blocks of a file within a volume of data, the method comprising:
for data blocks of a file that are to be encrypted;
creating frame headers to identify an encryption type and a size of the data blocks of the file;
generating identifying sequences to identify that the data blocks of the file are encrypted; and
storing encrypted data blocks of the file in the volume of data with the frame headers and with the identifying sequences;
the method further comprising:
encrypting data blocks of the file that are currently stored in the volume but that were not previously encrypted within the volume and storing the encrypted blocks in the volume, while the storage system continues to store data blocks to the volume and to retrieve data blocks from the volume; and
decrypting data blocks of the file that are currently stored in the volume as encrypted data blocks and storing the decrypted blocks in the volume, while the storage system continues to store data blocks to the volume and to retrieve data blocks from the volume;
wherein the storage system treats the volume of data as a single atomic instance without maintaining an index of encoding of the data blocks stored within the volume.
2. The method as recited in claim 1 further comprising applying different encoding types to different data blocks of the file.
3. The method as recited in claim 1 wherein the data blocks are the smallest logical unit of storage.
4. The method as recited in claim 1 wherein the data blocks are 4 k bytes of data.
5. The method as recited in claim 1 wherein the data blocks are the unit of data that is persisted to a storage system in which the volume of data is stored.
6. The method as recited in claim 1 wherein data is persisted in units of whole data blocks and wherein a data block is the smallest logic unit of storage.
7. The method as recited in claim 1 further comprising for data blocks of the file that are not to be encoded;
generating an identifying sequence; and
storing unencoded data blocks of the file in the volume of data without the identifying sequence and without a frame header unless the identifying sequence matches existing data within the data blocks.
8. The method as recited in claim 1 further comprising, where an encryption key has been changed from a first encryption key to a second encryption key, maintaining some of the blocks of the file in the volume encrypted with the first key, encrypting some blocks of the file in the volume with the second encryption key such that the volume stores a mix of some blocks of data that are encrypted with the first encryption key and some blocks of data that are encrypted with the second encryption key.
9. A non-transitory computer-readable storage medium containing program instructions for operating a storage system to store data blocks of a file within a volume of data in a persistent data storage system, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform steps comprising:
for data blocks of a file that are to be encrypted;
creating frame headers to identify an encryption type and a size of the data blocks of the file;
generating identifying sequences to identify that the data blocks of the file are encrypted; and
storing encrypted data blocks in the volume of data with the frame headers and the identifying sequences; and
further comprising:
encrypting data blocks of the file that are currently stored in the volume but that were not previously encrypted within the volume and storing the encrypted blocks in the volume, while the storage system continues to store data blocks to the volume and to retrieve data blocks from the volume; and
decrypting data blocks of the file that are currently stored in the volume as encrypted data blocks and storing the decrypted blocks in the volume, while the storage system continues to store data blocks to the volume and to retrieve data blocks from the volume;
wherein the storage system treats the volume of data as a single atomic instance without maintaining an index of encoding of the data blocks stored within the volume.
10. The non-transitory computer-readable storage medium of claim 9 containing program instructions for storing data blocks of a file within a volume of data in a persistent data storage system, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform steps further comprising applying different encoding types to different data blocks of the file.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the data blocks are the smallest logical unit of storage.
12. The non-transitory computer-readable storage medium of claim 9 , wherein the data blocks are 4 k bytes of data.
13. The non-transitory computer-readable storage medium of claim 9 , wherein the data blocks are the unit of data that is persisted to a storage system in which the volume of data is stored.
14. The non-transitory computer-readable storage medium of claim 9 , wherein data is persisted in units of whole data blocks and wherein a data block is the smallest logic unit of storage.
15. The non-transitory computer-readable storage medium of claim 9 containing program instructions for storing data blocks of a file within a volume of data in a persistent data storage system, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform steps further comprising for data blocks of the file that are not to be encoded;
generating an identifying sequence; and
storing unencoded data blocks of the file in the volume of data without the identifying sequence and without a frame header unless the identifying sequence matches existing data within the data blocks.
16. The non-transitory computer-readable storage medium of claim 9 , where an encryption key has been changed from a first encryption key to a second encryption key, maintaining some of the blocks of the file in the volume encrypted with the first key, encrypting some blocks of the file in the volume with the second encryption key such that the volume stores a mix of some blocks of data that are encrypted with the first encryption key and some blocks of data that are encrypted with the second encryption key.
17. A non-transitory computer-readable storage medium containing program instructions for storing data blocks of a file within a volume of data in a persistent data storage system, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to perform steps comprising:
for data blocks of a file that are to be encoded;
creating frame headers to identify an encoding type and a size of the data blocks of the file;
generating identifying sequences to identify that the data blocks of the file are encoded; and
storing encoded data blocks in the volume of data with the frame headers and the identifying sequences,
for data blocks of the file that are not to be encoded;
generating an identifying sequence; and
storing unencoded data blocks of the file in the volume of data without the identifying sequence and without a frame header unless the identifying sequence matches existing data within the data blocks.