Method and system for user-space storage I/O stack with user-space flash translation layer

1. A computer-implemented method for facilitating a user-space storage I/O stack, the method comprising:

generating, by a file system in the user-space, a logical block address associated with an I/O request which indicates data to be read or written;

receiving, by a flash translation layer module in the user-space from the file system, the logical block address, wherein the flash translation layer module operates by communicating with the file system and a block device driver;

mapping, by the flash translation layer module in the user-space, a physical block address corresponding to the logical block address;

notifying the block device driver of the physical block address;

operating, by the block device driver, the mapped physical block address to access or place the data in a non-volatile memory by bypassing a kernel space;

estimating a latency associated with executing the I/O request; and

in response to determining that the estimated latency is greater than or equal to a predetermined threshold, and that the I/O request is a read request, reading the requested data from a location other than the physical block address.

2. The method of claim 1 , wherein in response to determining that the estimated latency is greater than or equal to the predetermined threshold, and that the I/O request is a write request, the method further comprises:

generating, by the flash translation layer module in the user-space, a new physical block address.

3. The method of claim 1 , wherein in response to determining that the estimated latency is less than the predetermined threshold, the method further comprises:

in response to determining that the I/O request is a read request:

reading the requested data from the physical block address; and

sending the read data to a requesting host; and

in response to determining that the I/O request is a write request:

writing the requested data to the physical block address.

4. The method of claim 1 , wherein determining the estimated latency comprises:

in response to determining that physical media associated with the physical block address is currently undergoing a program operation which is not suspendable, including in the estimated latency a latency associated with completing the program operation;

in response to determining that the physical media associated with the physical block address is currently undergoing an erase operation which is not suspendable, including in the estimated latency a latency associated with completing the erase operation; and

in response to determining that raw data in a queue is currently undergoing a retry operation, including in the estimated latency a latency associated with completing the retry operation.

5. The method of claim 1 , wherein determining whether the estimated latency is greater than or equal to the predetermined threshold comprises:

determining, by an arbitrator module, whether the estimated latency exceeds or is equal to a currently available quota; and

in response to determining that the estimated latency does not exceed the currently available quota:

measuring an actual latency associated with completing the I/O request;

determining actual QoS statistics based on the actual latency; and

updating the currently available quota by comparing the actual QoS statistics with the predetermined threshold.

6. The method of claim 5 ,

wherein determining that the estimated latency is greater than or equal to the predetermined threshold is in response to determining that the estimated latency exceeds or is equal to the currently available quota; and

wherein in response to determining that the estimated latency does not exceed the currently available quota, the method further comprises:

generating calibrations for subsequent use in determining the estimated latency by comparing the actual latency with the estimated latency,

wherein the calibrations are subsequently used by the latency estimator module to determine the estimated latency for another I/O request.

7. The method of claim 1 , wherein the block device driver is associated with the user-space.

8. The method of claim 1 , wherein mapping, by the flash translation layer module in the user-space, the physical block address corresponding to the logical block address comprises:

optimizing, by the flash translation layer module, the mapping based on a host's visibility into a current status and a historical usage of the physical block address.

9. The method of claim 1 , further comprising:

optimizing, by the flash translation layer module, a scheduling of background operations performed by a system comprising the file system, the flash translation layer module, the block device driver, and the non-volatile memory.

10. The method of claim 9 , wherein optimizing the scheduling of the background operations further comprises:

in response to determining that the I/O request is a write request, determining that a first drive or a first channel is busy and assigning the write request to a second drive or a second channel with a second load which is lower than a first load of the first drive or the first channel; and

in response to determining that the I/O request is a read request, analyzing an estimated latency associated with executing the I/O request based on a status or condition of physical media associated with the non-volatile memory.

11. A computer system for facilitating a user-space storage I/O stack, the system comprising:

a processor; and

a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method, the method comprising:

generating, by a file system in the user-space, a logical block address associated with an I/O request which indicates data to be read or written;

receiving, by a flash translation layer module in the user-space from the file system, the logical block address, wherein the flash translation layer module operates by communicating with the file system and a block device driver;

mapping, by the flash translation layer module in the user-space, a physical block address corresponding to the logical block address;

notifying the block device driver of the physical block address;

operating, by the block device driver, the mapped physical block address to access or place the data in a non-volatile memory by bypassing a kernel space;

estimating a latency associated with executing the I/O request; and

in response to determining that the estimated latency is greater than or equal to a predetermined threshold, and that the I/O request is a read request, reading the requested data from a location other than the physical block address.

12. The computer system of claim 11 , wherein in response to determining that the estimated latency is greater than or equal to the predetermined threshold, and that the I/O request is a write request, the method further comprises:

generating, by the flash translation layer module in the user-space, a new physical block address.

13. The computer system of claim 11 , wherein in response to determining that the estimated latency is less than the predetermined threshold, the method further comprises:

in response to determining that the I/O request is a read request:

reading the requested data from the physical block address; and

sending the read data to a requesting host; and

in response to determining that the I/O request is a write request:

writing the requested data to the physical block address.

14. The computer system of claim 11 , wherein determining the estimated latency comprises:

in response to determining that physical media associated with the physical block address is currently undergoing a program operation which is not suspendable, including in the estimated latency a latency associated with completing the program operation;

in response to determining that the physical media associated with the physical block address is currently undergoing an erase operation which is not suspendable, including in the estimated latency a latency associated with completing the erase operation; and

in response to determining that raw data in a queue is currently undergoing a retry operation, including in the estimated latency a latency associated with completing the retry operation.

15. The computer system of claim 11 , wherein determining whether the estimated latency is greater than or equal to the predetermined threshold comprises:

determining, by an arbitrator module, whether the estimated latency exceeds or is equal to a currently available quota; and

in response to determining that the estimated latency does not exceed the currently available quota:

measuring an actual latency associated with completing the I/O request;

determining actual QoS statistics based on the actual latency; and

updating the currently available quota by comparing the actual QoS statistics with the predetermined threshold.

16. The computer system of claim 15 ,

wherein determining that the estimated latency is greater than or equal to the predetermined threshold is in response to determining that the estimated latency exceeds or is equal to the currently available quota; and

wherein in response to determining that the estimated latency does not exceed the currently available quota, the method further comprises:

generating calibrations for subsequent use in determining the estimated latency by comparing the actual latency with the estimated latency,

wherein the calibrations are subsequently used by the latency estimator module to determine the estimated latency for another I/O request.

17. The computer system of claim 11 , wherein the block device driver is associated with the user-space.

18. The computer system of claim 11 , wherein mapping, by the flash translation layer module in the user-space, the physical block address corresponding to the logical block address comprises:

optimizing, by the flash translation layer module, the mapping based on a host's visibility into a current status and a historical usage of the physical block address.

19. The computer system of claim 11 , wherein the method further comprises:

optimizing, by the flash translation layer module, a scheduling of background operations performed by a system comprising the file system, the flash translation layer module, the block device driver, and the non-volatile memory.

20. The computer system of claim 19 , wherein optimizing the scheduling of the background operations further comprises:

in response to determining that the I/O request is a write request, determining that a first drive or a first channel is busy and assigning the write request to a second drive or a second channel with a second load which is lower than a first load of the first drive or the first channel; and

in response to determining that the I/O request is a read request, analyzing an estimated latency associated with executing the I/O request based on a status or condition of physical media associated with the non-volatile memory.