Proactive hardening of data storage system
Disclosed are systems and methods for proactively recovering files stored in flash storage devices. The method may be performed at a flash file system. The method may include receiving a write command targeting a first file in a flash memory. The method may also include generating a reference hash corresponding to the first file, and storing the reference hash in the flash memory. The method may also include receiving a read command targeting the first file. In response to receiving the read command, the method may also include: providing a request for a logical block address corresponding to the first file to the flash manager, and receiving a response for the read command. The method may also include, in accordance with a determination that one or more hashes do not map to the first file, performing a file recovery operation for a second file based on the one or more hashes.
1. A storage system, comprising:
a flash file system configured to:
receive a write command targeting a first file in a flash memory;
in response to receiving the write command:
generate a reference hash corresponding to the first file; and
provide the reference hash to a flash manager causing the reference hash to be stored in one or more physical blocks in the flash memory;
receive a read command targeting the first file in the flash memory; and
in response to receiving the read command:
generate a first logical block address corresponding to the first file;
provide a request for the first logical block address to the flash manager;
receive a response for the read command from the flash manager, wherein the response includes one or more hashes; and
in accordance with a determination that the one or more hashes do not map to the first file corresponding to the reference hash, perform a file recovery operation for a second file corresponding to the one or more hashes.
2. The storage system of claim 1 , further comprising:
the flash manager configured to:
receive the reference hash from the flash file system;
provide the reference hash to the flash memory;
receive the request for the first logical block address in the flash memory;
generate a physical address and an initial end-to-end tag corresponding to the first logical block address;
provide the physical address to the flash memory;
receive data corresponding to the physical address from the flash memory, wherein the data includes a new end-to-end tag and a new hash;
in accordance with a determination that the new end-to-end tag does not match the initial end-to-end tag, perform a data recovery operation to obtain data corresponding to the first logical block address, wherein a mismatch between the initial end-to-end tag and the new end-to-end tag indicates that the data is retrieved from a second logical block address different from the first logical block address; and
provide the data corresponding to the first logical block address and the new hash to the flash file system, wherein the one or more hashes comprise the new hash.
3. The storage system of claim 2 , further comprising:
the flash memory configured to:
receive the reference hash from the flash manager;
store the reference hash in one or more physical blocks;
receive the physical address from the flash manager;
retrieve the data corresponding to the physical address, wherein the data includes the new end-to-end tag and the new hash; and
provide the data to the flash manager.
4. The storage system of claim 1 , wherein the one or more hashes do not map to the first file corresponding to the reference hash because of latching in the flash memory that causes the request for the first logical block address to be misdirected to a second logical block address corresponding to the second file.
5. The storage system of claim 1 , wherein the second file is accessed by a host at a lower frequency than the first file.
6. The storage system of claim 2 , wherein the flash manager is configured to provide the new hash to the flash file system in accordance with the determination that the new end-to-end tag does not match the initial end-to-end tag.
7. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
rebalance data portions associated with a second logical block address corresponding to the second file before receiving a read command for data corresponding to the second logical block address.
8. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
perform a corrective action on a second logical block address corresponding to the second file, without receiving a read command for data corresponding to the second logical block address.
9. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
provide information to a host causing the host to take corrective action on a second logical block address corresponding to the second file.
10. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
cause moving data from the flash memory to another flash memory.
11. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
in accordance with a determination that a second logical block address corresponding to the second file falls on a snapshot, trigger a new snapshot.
12. The storage system of claim 1 , wherein the flash file system is configured to:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
provide, to a host, information corresponding to a second logical block address corresponding to the second file, on completion of the read command.
13. A method for proactive hardening of flash storage devices, the method comprising:
at a flash file system:
receiving a write command targeting a first file in a flash memory;
in response to receiving the write command:
generating a reference hash corresponding to the first file; and
providing the reference hash to a flash manager causing the reference hash to be stored in one or more physical blocks in the flash memory;
receiving a read command targeting the first file in the flash memory; and
in response to receiving the read command:
generating a first logical block address corresponding to the first file;
providing a request for the first logical block address to the flash manager;
receiving a response for the read command from the flash manager, wherein the response includes one or more hashes; and
in accordance with a determination that the one or more hashes do not map to the first file corresponding to the reference hash, performing a file recovery operation for a second file corresponding to the one or more hashes.
14. The method of claim 13 , further comprising:
at the flash manager:
receiving the reference hash from the flash file system;
providing the reference hash to the flash memory;
receiving the request for the first logical block address in the flash memory;
generating a physical address and an initial end-to-end tag corresponding to the first logical block address;
providing the physical address to the flash memory;
receiving data corresponding to the physical address from the flash memory, wherein the data includes a new end-to-end tag and a new hash;
in accordance with a determination that the new end-to-end tag does not match the initial end-to-end tag, performing a data recovery operation to obtain data corresponding to the first logical block address, wherein a mismatch between the initial end-to-end tag and the new end-to-end tag indicates that the data is retrieved from a second logical block address different from the first logical block address; and
providing the data corresponding to the first logical block address and the new hash to the flash file system, wherein the one or more hashes comprise the new hash.
15. The method of claim 14 , further comprising:
at the flash memory:
receiving the reference hash from the flash manager;
storing the reference hash in one or more physical blocks;
receiving the physical address from the flash manager;
retrieving the data corresponding to the physical address, wherein the data includes the new end-to-end tag and the new hash; and
providing the data to the flash manager.
16. The method of claim 13 , wherein the one or more hashes do not map to the first file corresponding to the reference hash because of latching in the flash memory that causes the request for the first logical block address to be misdirected to a second logical block address corresponding to the second file.
17. The method of claim 13 , wherein the second file is accessed by a host at a lower frequency than the first file.
18. The method of claim 14 , wherein the flash manager provides the new hash to the flash file system in accordance with the determination that the new end-to-end tag does not match the initial end-to-end tag.
19. The method of claim 13 , further comprising:
at the flash file system:
in accordance with the determination that the one or more hashes do not map to the first file corresponding to the reference hash:
rebalancing data portions associated with a second logical block address corresponding to the second file before receiving a read command for data corresponding to the second logical block address.
20. A system, comprising:
means for receiving a write command targeting a first file in a flash memory;
means for, in response to receiving the write command:
generating a reference hash corresponding to the first file; and
providing the reference hash to a flash manager causing the reference hash to be stored in one or more physical blocks in the flash memory;
means for receiving a read command targeting the first file in the flash memory; and
means for, in response to receiving the read command:
generating a first logical block address corresponding to the first file;
providing a request for the first logical block address to the flash manager;
receiving a response for the read command from the flash manager, wherein the response includes one or more hashes; and
in accordance with a determination that the one or more hashes do not map to the first file corresponding to the reference hash, performing a file recovery operation for a second file corresponding to the one or more hashes.