End-to-end datapath protection
A technique of protecting data in a data storage system includes, in response to receiving a write request that specifies data to be written in the data storage system, calculating an error detection code (EDC) of the specified data. The technique further includes persisting the specified data and the EDC in persistent cache and, in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request.
1. A method of protecting data in a data storage system, the method comprising:
in response to receiving a write request that specifies data to be written in the data storage system, calculating an error detection code (EDC) of the specified data;
persisting the specified data and the EDC in persistent cache; and
in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request,
wherein the write request is received from a host application, and wherein the method further comprises acknowledging fulfillment of the write request to the host application only after persisting the specified data and the EDC.
2. The method of claim 1 , wherein receiving the write request includes:
ingesting the specified data into a set of kernel buffers in kernel space of the data storage system; and
transferring the specified data from the set of kernel buffers to a set of buffer caches in userspace of the data storage system.
3. The method of claim 2 , wherein calculating the EDC of the specified data is performed while transferring the specified data from the set of kernel buffers to the set of buffer caches.
4. The method of claim 3 , wherein calculating the EDC of the specified data includes:
fetching, by a CPU (central processing unit) of the data storage system, the specified data from the set of kernel buffers into a local cache of the CPU; and
calculating the EDC from the specified data in the local cache of the CPU.
5. The method of claim 2 , further comprising, while transferring the specified data from the set of kernel buffers to the set of buffer caches, checking the specified data for any of a set of predetermined patterns.
6. The method of claim 2 , wherein returning the specified data in satisfaction of the read request includes:
transferring the specified data to a second set of kernel buffers in the kernel memory of the data storage system; and
providing the specified data from the second set of kernel buffers to the host application,
wherein validating the specified data is performed while transferring the specified data to the second set of kernel buffers.
7. The method of claim 6 , wherein validating the specified data includes:
calculating a checksum of the specified data; and
confirming that the calculated checksum is consistent with the EDC.
8. The method of claim 1 , further comprising performing a flushing operation to flush the specified data to backend storage, wherein the method further comprises validating the specified data while performing the flushing operation.
9. The method of claim 8 , wherein the method further comprises calculating a deduplication fingerprint of the specified data while performing the flushing operation.
10. The method of claim 8 wherein while performing the flushing operation, the method further comprises loading the specified data into local cache of a CPU of the data storage system, the CPU then performing the acts of:
calculating a new checksum of the specified data from the specified data in the local cache;
confirming that the new checksum is consistent with the EDC, and
calculating a deduplication fingerprint of the specified data in the local cache.
11. The method of claim 1 , further comprising:
opening a transaction that specifies a set of changes in metadata of the data storage system;
forming a set of metadata pages of the transaction;
calculating a set of EDCs for the set of metadata pages; and
committing the transaction by the set of metadata pages and the set of EDCs to the persistent cache.
12. The method of claim 11 , further comprising:
performing a flushing operation to flush the new metadata of the transaction to backend storage; and
validating the new metadata while performing the flushing operation.
13. The method of claim 12 ,
wherein performing the flushing operation includes writing the new metadata of the transaction in pages, the pages having associated generation numbers, and
wherein writing the new metadata includes redundantly storing the pages in at least two disks of a mirrored array.
14. The method of claim 13 , further comprising resolving differences between two instances of a page of metadata as read from two disks of the mirrored array by replacing an instance having a smaller generation number with an instance having a larger generation number.
15. The method of claim 12 , further comprising processing a request to read an element of metadata from backend storage and, prior to returning the element of metadata in satisfaction of the request to read, validating the element of metadata.
16. The method of claim 12 , further comprising processing a request to read an element of metadata from the persistent cache and, prior to returning the element of metadata in satisfaction of the request to read, validating the element of metadata.
17. A computerized apparatus, comprising control circuitry that includes a set of processing units coupled to memory, the control circuitry constructed and arranged to:
in response to receipt of a write request that specifies data to be written, calculate an error detection code (EDC) of the specified data;
persist the specified data and the EDC in persistent cache; and
in response to receipt of a read request for reading the specified data, validate the specified data using the EDC before returning the specified data in satisfaction of the read request,
wherein the control circuitry constructed and arranged to receive the write request is further constructed and arranged to ingest the specified data into a set of kernel buffers in kernel space of the data storage system and transfer the specified data from the set of kernel buffers to a set of buffer caches in userspace of the data storage system.
18. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a computerized apparatus, cause the computerized apparatus to perform a method of protecting data, the method comprising:
in response to receiving a write request that specifies data to be written in the computerized apparatus, calculating an error detection code (EDC) of the specified data;
persisting the specified data and the EDC in persistent cache; and
in response to receiving a read request for reading the specified data, validating the specified data using the EDC before returning the specified data in satisfaction of the read request,
wherein receiving the write request includes ingesting the specified data into a set of kernel buffers in kernel space of the data storage system and transferring the specified data from the set of kernel buffers to a set of buffer caches in userspace of the data storage system.
19. The computer program product of claim 18 , wherein calculating the EDC of the specified data is performed while transferring the specified data from the set of kernel buffers to the set of buffer caches.
20. The computer program product of claim 19 , wherein the method further comprises, while transferring the specified data from the set of kernel buffers to the set of buffer caches, checking the specified data for any of a set of predetermined patterns.