Reactive deadlock management in storage area networks
Systems and methods in accordance with various embodiments can detect and alleviate potential or actual deadlock of a storage switch or storage area network when attempting to write data to a mirrored virtual target. In accordance with one embodiment, a timer is started when a storage switch routes a write command to the physical targets corresponding to a virtual target of the write command. If each physical target does not return a transfer ready resource within a predetermined timeout period, the switch determines that a potential or actual deadlock has occurred. An abort command is sent to each of the physical devices. The abort command can clear the command from the targets and also free any allocated transfer ready resources. After receiving an acceptance response from each physical target, the state of the write command at the switch can be cleared. The write command can then be re-issued to the physical devices. In one embodiment, a queue depth for the virtual target can be lowered after failing to receive transfer ready resources from each target. By decreasing the queue depth, the maximum number of commands the switch will process for a virtual target is decreased to decrease the probability of future deadlock. In one embodiment, the queue depth is increased after no deadlock is detected for a period of time.
1. A method of writing data to a virtual target, comprising:
receiving a write command for a virtual target having at least first and second corresponding mirrored physical targets;
providing said write command to said first physical target and said second physical target;
determining whether a first transfer ready signal is returned by said first physical target and a second transfer ready signal is returned by said second physical target within a timeout period; and
providing an abort command to said first physical target and said second physical target only if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.
2. The method of claim 1 , further comprising:
lowering a queue depth for said virtual target if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.
3. The method of claim 2 , wherein:
said queue depth is a maximum number of commands that will be processed for said virtual target.
4. The method of claim 3 , wherein:
said maximum number of commands is a maximum number of commands that will be processed for said virtual target by a storage switch.
5. The method of claim 3 , wherein:
said maximum number of commands is a maximum number of commands that will be processed for said virtual target at a port of a storage switch.
6. The method of claim 2 , wherein:
lowering said queue depth includes lowering said queue depth to a value of one.
7. The method of claim 2 , further comprising:
determining whether a deadlock situation occurs within a period of time after lowering said queue depth; and
increasing said queue depth if said deadlock situation does not occur within said period of time.
8. The method of claim 7 , wherein:
increasing said queue depth if said deadlock situation does not occur includes increasing said queue depth by a value of one.
9. The method of claim 7 , wherein:
said period of time is 15 minutes.
10. The method of claim 2 , further comprising:
again providing another write command to said first physical target and said second physical target if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.
11. The method of claim 1 , wherein:
said timeout period is five seconds.
12. The method of claim 1 , wherein said write command is a first write command, said method further comprising:
receiving a second write command; and
determining that said second write command exceeds a queue depth for said virtual target.
13. The method of claim 12 , wherein said second write command is received from an initiator, said method further comprising:
providing a signal to said initiator to indicate that said second write command will not be processed.
14. The method of claim 12 , further comprising:
maintaining said second write command in a process loop; and
determining whether said second write command exceeds said queue depth for said virtual target after a period of time.
15. The method of claim 12 , further comprising:
queuing said second write command; and
dequeuing and providing said second write command to said first physical target and said second physical target when said second write command does not exceed said queue depth.
16. The method of claim 1 , wherein:
said virtual target is a mirrored virtual target; and
data corresponding to said write command is to be synchronously routed to said first and second physical targets.
17. The method of claim 16 , wherein:
said mirrored virtual target is a mirrored virtual logical unit.
18. The method of claim 1 , wherein:
said first and second physical targets are first and second physical devices.
19. The method of claim 1 , wherein:
said first and second physical targets are first and second physical logical units.
20. A storage switch, comprising:
at least one port adapted to receive a write command, said write command is for a virtual target having at least first and second corresponding mirrored physical targets, said at least one port is further adapted to provide said write command to said first and second physical targets;
at least one processing unit in communication with said at least one port, said at least one processing unit provides an abort command to said first physical target and said second physical target only if, within a timeout period after said write command is provided to said first physical target and said second physical target, a first transfer ready signal is not received from said first physical target and a second transfer ready signal is not received from said second physical target.
21. The storage switch of claim 20 , wherein:
said at least one processing unit lowers a queue depth for said virtual target if, within said timeout period, said first transfer ready signal is not received from said first physical target and said second transfer ready signal is not received from said second physical target.
22. The storage switch of claim 21 , wherein:
said queue depth is a maximum number of commands that will be processed for said virtual target.
23. The storage switch of claim 22 , wherein:
said maximum number of commands is a maximum number of commands that will be processed for said virtual target by said storage switch.
24. The storage switch of claim 22 , wherein:
said maximum number of commands is a maximum number of commands received at said at least one port that will be processed for said virtual target.
25. The storage switch of claim 21 , wherein:
said at least one processing unit lowers said queue depth to a value of one.
26. The storage switch of claim 21 , wherein:
said at least one processing unit increases said queue depth, after lowering said queue depth, if a deadlock situation is not detected by said storage switch within a period of time after lowering said queue depth.
27. The storage switch of claim 26 , wherein:
said at least one processing unit increases said queue depth by a value of one.
28. The storage switch of claim 26 , wherein:
said period of time is 15 minutes.
29. The storage switch of claim 20 , wherein:
said at least one port includes an ingress port and at least one egress port, said storage switch receives said write command at said ingress port from an initiator, said write command is received at said at least one egress port from said ingress port and is provided to said first physical target and said second physical target via said at least one egress port.
30. The storage switch of claim 29 , wherein said at least one processing unit includes:
a first processing unit in communication with said ingress port, said first processing unit provides said write command to said at least one egress port; and
at least one second processing unit in communication with said at least one egress port, said at least one second processing unit receives said write command from said first processing unit and provides said write command to said first physical target and said second physical target via said at least one egress port.
31. The storage switch of claim 30 , wherein:
said first processing unit provides said abort command to said at least one second processing unit if a first transfer ready signal is not received from said first physical target and a second transfer ready signal is not received from said second physical target within said timeout period; and
said at least one second processing unit provides said abort command to said first physical target and said second physical target.
32. The storage switch of claim 30 , wherein:
said at least one egress port includes a first and second egress port, said first egress port is coupled with said first physical target, said second egress port is coupled with said second physical target; and
said at least one second processing unit includes a second processing unit and a third processing unit, said second processing unit is in communication with said first egress port and said third processing unit is in communication with said second egress port.
33. The storage switch of claim 30 , wherein:
said virtual target has said first corresponding physical target, said second corresponding physical target, and a third corresponding physical target;
said at least one egress port includes a first egress port, a second egress port, and a third egress port, said first egress port is coupled with said first physical target, said second egress port is coupled with said second physical target, said third egress port is coupled with said third physical target; and
said at least one second processing unit includes a second processing unit, a third processing unit, and a fourth processing unit, said second processing unit is in communication with said first egress port, said third processing unit is in communication with said second egress port, and said fourth processing unit is in communication with said third egress port.
34. The storage switch of claim 20 , wherein:
said at least one port again provides another write command to said first physical target and said second physical target if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.
35. The storage switch of claim 20 , wherein:
said virtual target is a mirrored virtual target; and
data associated with said write command is to be synchronously routed to said first physical target and said second physical target.
36. The storage switch of claim 35 , wherein:
said mirrored virtual target is a mirrored virtual logical unit.
37. The storage switch of claim 20 , wherein:
said first and second physical targets are first and second physical logical units.
38. The storage switch of claim 20 , wherein:
said first and second physical targets are first and second physical storage devices.
39. The storage switch of claim 20 , wherein:
said timeout period is five seconds.
40. One or more processor readable storage devices for storing processor readable code, said processor readable code for programming one or more processors to perform a method comprising the steps of:
receiving a write command for a virtual target having at least a first and a second corresponding mirrored physical targets;
providing said write command to said first physical target and said second physical target;
determining whether a first transfer ready signal is returned by said first physical target and a second transfer ready signal is returned by said second physical target within a timeout period; and
providing an abort command to said first physical target and said second physical target only if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.
41. A storage switch comprising:
means for receiving a write command for a virtual target having at least first and second corresponding mirrored physical targets;
means for providing said write command to said first physical target and said second physical target;
means for determining whether a first transfer ready signal is returned by said first physical target and a second transfer ready signal is returned by said second physical target within a timeout period; and
means for providing an abort command to said first physical target and said second physical target only if said first transfer ready signal and said second transfer ready signal are not returned within said timeout period.