Coordinated garbage collection of flash devices in a distributed storage system
A distributed storage system can include a storage node ( 125, 130, 135 ). The storage node ( 125, 130, 135 ) can include a Solid State Drive (SSD) or other storage device that employs garbage collection ( 140, 145, 150, 155, 160, 165, 225, 230 ), a device garbage collection monitor ( 205 ), a garbage collection coordinator ( 210 ), an Input/Output (I/O) redirector ( 215 ), and an I/O resynchronizer ( 220 ). The device garbage collection monitor ( 205 ) can determine whether any storage devices ( 140, 145, 150, 155, 160, 165, 225, 230 ) need to perform garbage collection. The garbage collection coordinator ( 210 ) can schedule when the storage device ( 140, 145, 150, 155, 160, 165, 225, 230 ) can perform garbage collection. The I/O redirector ( 215 ) can redirect read requests ( 905 ) and write requests ( 1005 ) away from the storage device ( 140, 145, 150, 155, 160, 165, 225, 230 ) when it is performing garbage collection. The I/O resynchronizer ( 220 ) can ensure that data on the storage device ( 140, 145, 150, 155, 160, 165, 225, 230 ) is up-to-date after garbage collection finishes.
1. A distributed storage system node, comprising:
at least one storage device;
a device garbage collection monitor to determine a count of free erase blocks on each of the storage devices, the device garbage collection monitor including:
a receiver to receive the count of free erase blocks from the at least one storage device; and
a comparator to compare the count of the free erase blocks on the at least one storage device with a free erase block threshold, wherein the device garbage collection monitor is operative to select a selected storage device of the at least one storage device when the count of the free erase blocks on the selected storage device is below the free erase block threshold;
a garbage collection coordinator to schedule the garbage collection event on the selected storage device of the at least one storage device;
an Input/Output (I/O) redirector to process a read request using a second storage device in a second distributed storage system node storing a replicated copy of the requested data and to redirect a write request; and
an I/O resynchronizer to update the storage device to incorporate any write requests.
2. A distributed storage system node according to claim 1 , further comprising a logging device to store the write request redirected by the I/O redirector.
3. A distributed storage system node according to claim 1 , wherein:
the I/O redirector is operative to redirect a write request to a third storage device in a third distributed storage system node; and
the I/O resynchronizer is operative to replay any write requests from the third storage device.
4. A distributed storage system node according to claim 3 , wherein the second distributed storage system node is the third distributed storage system node.
5. A distributed storage system node according to claim 1 , wherein the device garbage collection monitor further includes a time estimator to estimate a time required for the garbage collection event.
6. A distributed storage system node according to claim 5 , wherein the time estimator is operative to estimate the time required for the garbage collection event using at least one of an erase cycle time for the selected storage device, a time taken for a prior garbage collection event, and an available capacity of the storage device.
7. A distributed storage system node according to claim 5 , wherein the garbage collection coordinator is operative to inform at least one monitor of an identifier for the selected storage device and an estimated time required for the garbage collection event.
8. A method, comprising:
selecting a storage device in a distributed storage system node to begin a garbage collection event, including:
determining a free erase block count for the selected storage device;
comparing the free erase block count with a free erase block threshold; and
if the free erase block count is below the free erase block threshold, selecting the storage device to begin the garbage collection event;
determining an estimated time required for the selected storage device to perform the garbage collection event;
determining a scheduled start time for the garbage collection event;
instructing the selected storage device, at the scheduled start time, to begin the garbage collection event; and
redirecting a write request to a second storage device in a second distributed storage system node.
9. A method according to claim 8 , wherein determining an estimated time required for the selected storage device to perform the garbage collection event includes determining the estimated time required for the selected storage device to perform the garbage collection event using at least one of an erase cycle time for the selected storage device, a time taken for a prior garbage collection event, and an available capacity of the storage device.
10. A method according to claim 8 , wherein determining a scheduled start time for the garbage collection event includes receiving the scheduled start time from a monitor.
11. A method according to claim 10 , wherein determining a scheduled start time for the garbage collection event further includes receiving a duration for the garbage collection event from the monitor.
12. A method according to claim 8 , further comprising redirecting a read request away from the selected storage device.
13. A method according to claim 12 , wherein
redirecting a read request away from the selected storage device includes redirecting the read request away from the selected storage device to a third storage device in a third distributed storage system node.
14. A method according to claim 8 , further comprising redirecting a write request to a logging device.
15. A method, comprising:
selecting a storage device in a distributed storage system node to begin a garbage collection event, including:
determining a free erase block count for the selected storage device;
comparing the free erase block count with a free erase block threshold; and
if the free erase block count is below the free erase block threshold, selecting the storage device to begin the garbage collection event;
determining an estimated time required for the selected storage device to perform the garbage collection event;
determining a scheduled start time for the garbage collection event;
instructing the selected storage device, at the scheduled start time, to begin the garbage collection event; and
redirecting a read request away from the selected storage device to a second storage device in a second distributed storage system node.
16. A method according to claim 15 , wherein determining an estimated time required for the selected storage device to perform the garbage collection event includes determining the estimated time required for the selected storage device to perform the garbage collection event using at least one of an erase cycle time for the selected storage device, a time taken for a prior garbage collection event, and an available capacity of the storage device.
17. A method according to claim 15 , wherein determining a scheduled start time for the garbage collection event includes receiving the scheduled start time from a monitor.
18. A method according to claim 17 , wherein determining a scheduled start time for the garbage collection event further includes receiving a duration for the garbage collection event from the monitor.
19. A method according to claim 15 , further comprising redirecting a write request to a logging device.