Methods and systems for securing data in a distributed storage system
A method for securing data, the method including obtaining, from a metadata node and by file system (FS) client executing on a client application node, a data layout and an encryption key, encrypting, by the client application node, the data stored on the client application node using the encryption key to obtain encrypted data, generating, by a memory hypervisor module executing on the client application node, at least one input/output (I/O) request, wherein the at least one I/O request specifies a location in a storage pool, wherein the location is determined using the data layout, and issuing, by the memory hypervisor module, the at least one I/O request to the storage pool, wherein processing the at least one I/O request results in at least a portion of the encrypted data being stored at the location.
1. A method for securing data, the method comprising:
obtaining, from a metadata node and by file system (FS) client executing on a client application node, a data layout and an encryption key,
wherein the data layout comprises a file layout and a file system layout;
encrypting, by the FS client, the data stored on the client application node using the encryption key to obtain encrypted data,
wherein the data is specified using a [file, file offset] of a virtual address space for an application also executing on the client application node;
generating, by a memory hypervisor module excluding the FS client and also executing on the client application node, at least one input/output (I/O) request,
wherein the at least one I/O request specifies a location in a storage pool,
wherein the location is determined using the data layout, determination of the location comprising:
mapping the file offset to a logical block of the file layout for the file,
mapping the logical block to a file system block (FSB) of the file system layout for a file system of the metadata node,
mapping the FSB to a portion of a slice of a scale out volume (SOV), and
mapping the portion of the slice of the SOV to the location; and
issuing, by the memory hypervisor module, the at least one I/O request directly to a storage node of the storage pool,
wherein processing the at least one I/O request results in at least a portion of the encrypted data being stored at the location.
2. The method of claim 1 , further comprising:
issuing, by the client application node, a request to commit metadata associated with the data to the metadata node,
wherein the metadata comprises the data layout and an encryption key identifier associated with the encryption key.
3. The method of claim 1 , wherein the location in the storage pool is a location in a memory device.
4. The method of claim 3 , wherein the memory device is located on the storage node.
5. The method of claim 4 , wherein the storage pool comprises the memory device, a second memory device on the metadata node, and a third memory device on the client application node.
6. The method of claim 3 , wherein the memory device is a persistent memory (PMEM) device.
7. The method of claim 1 , wherein the at least one I/O request being issued is not processed by the metadata node.
8. A non-transitory computer readable medium comprising instructions which, when executed by a processor, enables the processor to perform a method for securing data, the method comprising:
obtaining, from a metadata node and by file system (FS) client executing on a client application node, a data layout and an encryption key,
wherein the data layout comprises a file layout and a file system layout;
encrypting, by the FS client, the data stored on the client application node using the encryption key to obtain encrypted data,
wherein the data is specified using a [file, file offset] of a virtual address space for an application also executing on the client application node;
generating, by a memory hypervisor module excluding the FS client and also executing on the client application node, at least one input/output (I/O) request,
wherein the I/O request specifies a location in a storage pool,
wherein the location is determined using the data layout, determination of the location comprising:
mapping the file offset to a logical block of the file layout for the file,
mapping the logical block to a file system block (FSB) of the file system layout for a file system of the metadata node,
mapping the FSB to a portion of a slice of a scale out volume (SOV), and
mapping the portion of the slice of the SOV to the location; and
issuing, by the memory hypervisor module, the at least one I/O request directly to a storage node of the storage pool,
wherein processing the at least one I/O request results in at least a portion of the encrypted data being stored at the location.
9. The non-transitory computer readable medium of claim 8 , the method further comprising:
issuing, by the client application node, a request to commit metadata associated with the data to the metadata node,
wherein the metadata comprises the data layout and an encryption key identifier associated with the encryption key.
10. The non-transitory computer readable medium of claim 8 , wherein the location in the storage pool is a location in a memory device.
11. The non-transitory computer readable medium of claim 10 , wherein the memory device is located on the storage node.
12. The non-transitory computer readable medium of claim 11 , wherein the storage pool comprises the memory device, a second memory device on the metadata node, and a third memory device on the client application node.
13. The non-transitory computer readable medium of claim 10 , wherein the memory device is a persistent memory (PMEM) device.
14. The non-transitory computer readable medium of claim 8 , wherein the at least one I/O request being issued is not processed by the metadata node.
15. A system, comprising:
a client application node comprising a processor whereon a file system (FS) client and a memory hypervisor module, excluding the FS client, execute,
wherein the FS client is configured to:
obtain a data layout and an encryption key from a metadata node,
wherein the data layout comprises a file layout and a file system layout; and
encrypt the data stored on the client application node using the encryption key to obtain encrypted data,
wherein the data is specified using a [file, file offset] of a virtual address space for an application also executing on the client application node, and
wherein the memory hypervisor module is configured to:
generate at least one input/output (I/O) request,
wherein the I/O request specifies a location in a storage pool,
wherein the location is determined using the data layout, determination of the location comprising:
mapping the file offset to a logical block of the file layout for the file,
mapping the logical block to a file system block (FSB) of the file system layout for a file system of the metadata node,
mapping the FSB to a portion of a slice of a scale out volume (SOV), and
mapping the portion of the slice of the SOV to the location,
wherein the storage pool comprises a storage node comprising memory device, and
wherein the location is in the memory device; and
issue the at least one I/O request directly to the storage node;
the metadata node comprising a second processor and configured to:
provide the data layout to the FS client;
the storage node comprising a third processor and the memory device, and configured to:
process the at least one I/O request,
wherein processing the at least one I/O request results in at least a portion of the encrypted data being stored at the location.
16. The system of claim 15 , wherein the storage pool comprises the memory device, a second memory device on the metadata node, and a third memory device on the client application node.
17. The system of claim 15 , wherein the at least one I/O request being issued is not processed by the metadata node.
18. The system of claim 15 , wherein the memory device is a persistent memory (PMEM) device.