Multi-path layer configured for detection and mitigation of slow drain issues in a storage area network
A processing device is configured to control delivery of input-output (IO) operations from a host device to a storage system over selected ones of a plurality of paths through a network, and to monitor response times for particular ones of the IO operations sent from the host device to the storage system. The processing device is further configured, responsive to at least a subset of the monitored response times being above a threshold over a time period, to generate an alert, to send the alert to an intermediary device coupled to the host device and the storage system, to receive from the intermediary device a throttling command generated in the intermediary device based at least in part on the alert, and responsive to receipt of the throttling command, to at least temporarily reduce a rate at which additional ones of the IO operations are sent from the host device to the storage system.
1. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
wherein the at least one processing device is configured:
to control delivery of input-output operations from a host device to a storage system over selected ones of a plurality of paths through a network;
to monitor response times for particular ones of the input-output operations sent from the host device to the storage system;
responsive to at least a subset of the monitored response times being above a threshold over a time period, to generate an alert;
to send the alert to an intermediary device coupled to the host device and the storage system;
to receive from the intermediary device a throttling command generated in the intermediary device based at least in part on the alert; and
responsive to receipt of the throttling command, to at least temporarily reduce a rate at which additional ones of the input-output operations are sent from the host device to the storage system.
2. The apparatus of claim 1 wherein the at least one processing device comprises at least a portion of the host device.
3. The apparatus of claim 1 wherein the paths are associated with respective initiator-target pairs, the initiators being implemented on the host device and the targets being implemented on the storage system, the initiators of the initiator-target pairs comprising respective host bus adaptors of the host device and the targets of the initiator-target pairs comprising respective storage array ports of the storage system.
4. The apparatus of claim 1 wherein the throttling command is generated in the intermediary device based at least in part on identification in the intermediary device of one or more of the plurality of paths that each exhibits at least a threshold amount of mismatch between a negotiated rate of its initiator and a negotiated rate of its target.
5. The apparatus of claim 4 wherein the mismatch between the negotiated rate of the initiator and the negotiated rate of the target comprises the negotiated rate of the initiator being less than the negotiated rate of the target.
6. The apparatus of claim 1 wherein said at least one processing device is further configured to send negotiated rate information of respective initiators of the host device to the intermediary device for utilization by the intermediary device in generating the throttling command.
7. The apparatus of claim 1 wherein said at least one processing device comprises a multi-path input-output driver of the host device, with the multi-path input-output driver of the host device being configured to control the delivery of the input-output operations from the host device to the storage system over the selected ones of the plurality of paths through the network.
8. The apparatus of claim 7 wherein the multi-path input-output driver is further configured to perform at least a portion of the monitoring of response times, the generating of the alert, the sending of the alert, the receiving of the throttling command, and the at least temporarily reducing of the rate.
9. The apparatus of claim 7 wherein the intermediary device comprises a multi-path input-output management station that is coupled to the multi-path input-output driver of the host device and to multi-path input-output drivers of respective ones of one or more additional host devices coupled to the storage system.
10. The apparatus of claim 1 wherein the intermediary device comprises at least one server configured to obtain negotiated rate information for respective initiators from the host device and one or more additional host devices coupled to the storage system, and to obtain negotiated rate information for respective targets from the storage system, for utilization by the intermediary device in generating the throttling command.
11. The apparatus of claim 1 wherein the intermediary device obtains negotiated rate information for respective initiators from the host device via an operating system (OS) native command line interface (CLI).
12. The apparatus of claim 1 wherein the intermediary device obtains negotiated rate information for respective targets from the storage system via a representational state transfer (REST) call to the storage system.
13. The apparatus of claim 1 wherein the throttling command is generated in the intermediary device responsive to a determination that the host device exhibits a greater degree of mismatch between one or more of its initiator negotiated rates and one or more corresponding target negotiated rates of the storage system than at least one other host device coupled to the storage system.
14. The apparatus of claim 1 wherein said at least one processing device is further configured:
to receive from the intermediary device an additional command generated in the intermediary device; and
responsive to receipt of the additional command, to at least partially reverse the reduction in the rate at which additional ones of the input-output operations are sent from the host device to the storage system.
15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code, when executed by at least one processing device comprising a processor coupled to a memory, causes the at least one processing device:
to control delivery of input-output operations from a host device to a storage system over selected ones of a plurality of paths through a network;
to monitor response times for particular ones of the input-output operations sent from the host device to the storage system;
responsive to at least a subset of the monitored response times being above a threshold over a time period, to generate an alert;
to send the alert to an intermediary device coupled to the host device and the storage system;
to receive from the intermediary device a throttling command generated in the intermediary device based at least in part on the alert; and
responsive to receipt of the throttling command, to at least temporarily reduce a rate at which additional ones of the input-output operations are sent from the host device to the storage system.
16. The computer program product of claim 15 wherein the intermediary device comprises at least one server configured to obtain negotiated rate information for respective initiators from the host device and one or more additional host devices coupled to the storage system, and to obtain negotiated rate information for respective targets from the storage system, for utilization by the intermediary device in generating the throttling command.
17. The computer program product of claim 15 wherein the throttling command is generated in the intermediary device responsive to a determination that the host device exhibits a greater degree of mismatch between one or more of its initiator negotiated rates and one or more corresponding target negotiated rates of the storage system than at least one other host device coupled to the storage system.
18. A method comprising:
controlling delivery of input-output operations from a host device to a storage system over selected ones of a plurality of paths through a network;
monitoring response times for particular ones of the input-output operations sent from the host device to the storage system;
responsive to at least a subset of the monitored response times being above a threshold over a time period, generating an alert;
sending the alert to an intermediary device coupled to the host device and the storage system;
receiving from the intermediary device a throttling command generated in the intermediary device based at least in part on the alert; and
responsive to receipt of the throttling command, at least temporarily reducing a rate at which additional ones of the input-output operations are sent from the host device to the storage system.
19. The method of claim 18 wherein the intermediary device comprises at least one server configured to obtain negotiated rate information for respective initiators from the host device and one or more additional host devices coupled to the storage system, and to obtain negotiated rate information for respective targets from the storage system, for utilization by the intermediary device in generating the throttling command.
20. The method of claim 18 wherein the throttling command is generated in the intermediary device responsive to a determination that the host device exhibits a greater degree of mismatch between one or more of its initiator negotiated rates and one or more corresponding target negotiated rates of the storage system than at least one other host device coupled to the storage system.