Temporarily limiting access to a storage device
Temporarily limiting access to a storage device, including: determining that a storage device of a plurality of storage devices in a storage system is operating outside of a defined performance range; determining that the storage device operating outside of the defined performance range may be caused by a rehabilitative action performed on the storage device; and modifying a storage operation issuance policy for one or more storage devices of the plurality of storage devices until a determination that the storage device is operating within the defined performance range.
1. A method comprising:
determining that a storage device of a plurality of storage devices in a storage system is operating outside of a defined performance range;
determining that the storage device operating outside of the defined performance range may be caused by a rehabilitative action performed on the storage device; and
modifying a storage operation issuance policy for one or more storage devices of the plurality of storage devices until a determination that the storage device is operating within the defined performance range, wherein the modified storage operation issuance policy delays a writing, to the storage device, of a shard during a RAID stripe write operation.
2. The method of claim 1 , wherein the modified storage operation issuance policy prevents issuance of write operations to the storage device and allows issuance of read operations to the storage device.
3. The method of claim 1 , wherein determining that the storage device operating outside of the defined performance range is caused by the rehabilitative action performed on the storage device comprises:
determining that a latency of one or more write operations issued to the storage device exceeds a first threshold; and
determining that a latency of one or more read operations issued to the storage device falls below a second threshold.
4. The method of claim 1 , further comprising:
determining that the storage device is operating within the defined performance range; and
updating the storage operation issuance policy in response to the storage device operating within the defined performance range.
5. The method of claim 4 , wherein determining that the storage device is operating within the defined performance range comprises:
issuing one or more test write operations to the storage device; and
determining that the one or more test write operations are completed within a threshold duration.
6. The method of claim 5 , wherein the one or more test write operations are directed to one or more logical addresses that have been garbage collected but not reused or unmapped.
7. The method of claim 5 , wherein the one or more test operations are issued at a predefined interval.
8. An apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
determining that a storage device of a plurality of storage devices in a storage system is operating outside of a defined performance range;
determining that the storage device operating outside of the defined performance range may be caused by a rehabilitative action performed on the storage device; and
modifying a storage operation issuance policy for one or more storage devices of the plurality of storage devices until a determination that the storage device is operating within the defined performance range, wherein the modified storage operation issuance policy delays a writing, to the storage device, of a shard during a RAID stripe write operation.
9. The apparatus of claim 8 , wherein the modified storage operation issuance policy prevents issuance of write operations to the storage device and allows issuance of read operations to the storage device.
10. The apparatus of claim 8 , wherein determining that the storage device operating outside of the defined performance range is caused by the rehabilitative action performed on the storage device comprises:
determining that a latency of one or more write operations issued to the storage device exceeds a first threshold; and
determining that a latency of one or more read operations issued to the storage device falls below a second threshold.
11. The apparatus of claim 8 , wherein the modified storage operation issuance policy excludes the storage device from selection for forming RAID stripes.
12. The apparatus of claim 8 , wherein the steps further comprise:
determining that the storage device is operating within the defined performance range; and
updating the storage operation issuance policy in response to the storage device operating within the defined performance range.
13. The apparatus of claim 12 , wherein determining that the storage device is operating within the defined performance range comprises:
issuing one or more test write operations to the storage device; and
determining that the one or more test write operations are completed within a threshold duration.
14. The apparatus of claim 13 , wherein the one or more test write operations are directed to one or more logical addresses that have been garbage collected but not reused or unmapped.
15. The apparatus of claim 12 , wherein the one or more test operations are issued at a predefined interval.
16. The apparatus of claim 8 , wherein initiating, in response to the storage device operating outside of the defined performance range, a drive evacuation for the storage device.
17. The apparatus of claim 16 , wherein the steps further comprise ending, in response to the determination that the storage device is operating within the defined performance range, the drive evacuation.
18. A method comprising:
determining that a storage device of a plurality of storage devices in a storage system is operating outside of a defined performance range;
determining that the storage device operating outside of the defined performance range may be caused by a rehabilitative action performed on the storage device; and
modifying a storage operation issuance policy for one or more storage devices of the plurality of storage devices until a determination that the storage device is operating within the defined performance range, wherein the modified storage operation issuance policy excludes the storage device from selection for forming RAID stripes.
19. A method comprising:
determining that a storage device of a plurality of storage devices in a storage system is operating outside of a defined performance range;
determining that the storage device operating outside of the defined performance range may be caused by a rehabilitative action performed on the storage device;
initiating, in response to the storage device operating outside of the defined performance range, a drive evacuation for the storage device; and
modifying a storage operation issuance policy for one or more storage devices of the plurality of storage devices until a determination that the storage device is operating within the defined performance range; and
ending, in response to a determination that the storage device is operating within the defined performance range, the drive evacuation.