Managing storage device errors during processing of inflight input/output requests
Techniques are provided for managing storage device errors during processing of inflight input/output (I/O) operations. A storage control system (e.g., a software-defined storage system) receives an I/O write request which includes data to be written to at least one storage device of a plurality of storage devices managed by the storage control system, and commences a write operation to write the data to the at least one storage device. In response to a storage device I/O error resulting from a failure of the write operation associated with the at least one storage device, the storage control system accesses a logical storage device in a non-volatile system memory device, and writes the data to the logical storage device in the non-volatile system memory device to complete the failed write operation.
1. A method, comprising:
managing, by a storage control system, a plurality of storage devices of a storage node;
instantiating, by the storage control system, one or more logical storage devices, wherein each logical storage device is associated with a respective storage device of the plurality of storage devices, and wherein the one or more logical storage devices are maintained by the storage control system in a non-volatile memory device of system memory of the storage node;
receiving, by the storage control system, an input/output (I/O) write request comprising data to be written to at least one storage device of the plurality of storage devices managed by the storage control system;
commencing, by the storage control system, a write operation to write the data to the at least one storage device; and
in response to a storage device I/O error resulting from a failure of the write operation associated with the at least one storage device, the storage control system:
accessing a logical storage device in the non-volatile memory device of the system memory, which is associated with the at least one storage device for which the storage device I/O error was returned; and
writing the data to the accessed logical storage device in the non-volatile memory device to complete the failed write operation.
2. The method of claim 1 , wherein accessing the logical storage device in the non-volatile memory device of the system memory comprises:
accessing a designated region of memory in the non-volatile memory device which is associated with the logical storage device; and
updating an address mapping table for the logical storage device, wherein the address mapping table is configured to map a logical address to a physical address in the designated region of memory in the non-volatile memory device which is associated with the logical storage device.
3. The method of claim 2 , wherein updating the address mapping table for the logical storage device comprises updating the address mapping table to map a destination address of the at least one storage device, as provided with the I/O write request, to a physical address in the designated region of memory in the non-volatile memory device where the data is stored in the logical storage device.
4. The method of claim 1 , wherein the one or more logical storage devices are maintained in respective designated regions of the non-volatile memory device, and wherein each logical storage device comprises a dedicated address mapping table which is stored in the non-volatile memory device in association with the logical storage device.
5. The method of claim 1 , further comprising:
receiving, by the storage control system, a second I/O write request; and
in response to determining, by the storage control system, that the second I/O write request is directed to the at least one storage device associated with the storage device I/O error, automatically writing data associated with the second I/O request to the associated logical storage device in the non-volatile memory device.
6. The method of claim 1 , further comprising sending, by the storage control system, a notification of the storage device I/O error to a management node to enable the management node to reconfigure a storage environment to prevent at least one client node from issuing a new I/O request destined to the at least one storage device associated with the storage device I/O error.
7. The method of claim 1 , further comprising performing, by the storage control system, a recovery process to recover from the storage device I/O error, wherein the recovery process comprises recovering data stored in the logical storage device in the non-volatile memory device.
8. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to implement a method comprising:
managing, by a storage control system, a plurality of storage devices of a storage node;
instantiating, by the storage control system, one or more logical storage devices, wherein each logical storage device is associated with a respective storage device of the plurality of storage devices, and wherein the one or more logical storage devices are maintained by the storage control system in a non-volatile memory device of system memory of the storage node;
receiving, by the storage control system, an input/output (I/O) write request comprising data to be written to at least one storage device of the plurality of storage devices managed by the storage control system;
commencing, by the storage control system, a write operation to write the data to the at least one storage device; and
in response to a storage device I/O error resulting from a failure of the write operation associated with the at least one storage device, the storage control system:
accessing a logical storage device in the non-volatile memory device of the system memory, which is associated with the at least one storage device for which the storage device I/O error was returned; and
writing the data to the accessed logical storage device in the non-volatile memory device to complete the failed write operation.
9. The article of manufacture of claim 8 , wherein the program code for accessing the logical storage device in the non-volatile system memory device of the system memory comprises program code that is executable by the one or more process to implement a process which comprises:
accessing a designated region of memory in the non-volatile memory device which is associated with the logical storage device; and
updating an address mapping table for the logical storage device, wherein the address mapping table is configured to map a logical address to a physical address in the designated region of memory in the non-volatile system memory device which is associated with the logical storage device.
10. The article of manufacture of claim 9 , wherein the program code for updating the address mapping table for the logical storage device comprises program code that is executable by the one or more process to implement a process which comprises updating the address mapping table to map a destination address of the at least one storage device, as provided with the I/O write request, to a physical address in the designated region of memory in the non-volatile system memory device where the data is stored in the logical storage device.
11. The article of manufacture of claim 8 , wherein the one or more logical storage devices are maintained in respective designated regions of the non-volatile memory device, and wherein each logical storage device comprises a dedicated address mapping table which is stored in the non-volatile system memory device in association with the logical storage device.
12. The article of manufacture of claim 8 , further comprising program code that is executable by the one or more process to implement a process which comprises:
receiving, by the storage control system, a second I/O write request; and
in response to determining, by the storage control system, that the second I/O write request is directed to the at least one storage device associated with the storage device I/O error, automatically writing data associated with the second I/O request to the associated logical storage device in the non-volatile memory device.
13. The article of manufacture of claim 8 , further comprising program code that is executable by the one or more process to implement a process which comprises sending, by the storage control system, a notification of the storage device I/O error to a management node to enable the management node to reconfigure a storage environment to prevent at least one client node from issuing a new I/O request destined to the at least one storage device associated with the storage device I/O error.
14. The article of manufacture of claim 8 , further comprising program code that is executable by the one or more process to implement a process which comprises performing, by the storage control system, a recovery process to recover from the storage device I/O error, wherein the recovery process comprises recovering data stored in the logical storage device in the non-volatile memory device.
15. An apparatus, comprising:
at least one processor; and
a system memory configured to store program code, wherein the program code is executable by the at least one processor to implement a storage control system which is configured to:
manage a plurality of storage devices of a storage node;
instantiate one or more logical storage devices, wherein each logical storage device is associated with a respective storage device of the plurality of storage devices, and wherein the one or more logical storage devices are maintained by the storage control system in a non-volatile memory device of system memory of the storage node;
receive an input/output (I/O) write request comprising data to be written to at least one storage device of the plurality of storage devices managed by the storage control system;
commence a write operation to write the data to the at least one storage device; and
in response to a storage device I/O error resulting from a failure of the write operation associated with the at least one storage device, the storage control system is configured to:
access a logical storage device in the non-volatile memory device of the system memory, which is associated with the at least one storage device for which the storage device I/O error was returned; and
write the data to the accessed logical storage device in the non-volatile memory device to complete the failed write operation.
16. The apparatus of claim 15 , wherein in accessing the logical storage device in the non-volatile memory device of the system memory, the storage control system is configured to:
access a designated region of memory in the non-volatile system memory device which is associated with the logical storage device; and
update an address mapping table for the logical storage device, wherein the address mapping table is configured to map a logical address to a physical address in the designated region of memory in the non-volatile memory device which is associated with the logical storage device.
17. The apparatus of claim 15 , wherein in updating the address mapping table for the logical storage device, the storage control system is configured to update the address mapping table to map a destination address of the at least one storage device, as provided with the I/O write request, to a physical address in the designated region of memory in the non-volatile system memory device where the data is stored in the logical storage device.
18. The apparatus of claim 15 , wherein the one or more logical storage devices are maintained in respective designated regions of the non-volatile system memory device, and wherein each logical storage device comprises a dedicated address mapping table which is stored in the non-volatile memory device in association with the logical storage device.
19. The apparatus of claim 15 , wherein the storage control system is further configured to send a notification of the storage device I/O error to a management node to enable the management node to reconfigure a storage environment to prevent at least one client node from issuing a new I/O request destined to the at least one storage device associated with the storage device I/O error.
20. The apparatus of claim 15 , wherein the storage control system is further configured to perform a recovery process to recover from the storage device I/O error, wherein the recovery process comprises recovering data stored in the logical storage device in the non-volatile memory device.