IP Library › Granted Patent US 7,421,509
Granted Patent B2
US 7,421,509 · App. 10/051,339 · Granted Sep 2, 2008

Enforcing quality of service in a storage network

Assignee: EMC Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,421,509
App. No.
10/051,339
Filed
Jan 18, 2002
Granted
Sep 2, 2008
Kind
B2
Examiner
LIN, WEN TAI
Art Unit
2154
USPC
709/235
Abstract

A system in accordance with an embodiment of the invention provides Quality of Service (QoS) for Storage Access. Such QoS is partially enabled in one embodiment by the automatic pooling of storage devices and provisioning virtual targets from those pools. QoS is enforced in one embodiment by keeping the bandwidth for each connection within a specified range, and particularly, by controlling the number of allowed concurrent requests from an initiator. Load balancing is also provided in one embodiment, improving response times for requests, further easing the ability to provide QoS.

Claims (77)

1. A method for use in a storage network, the storage network including at least one initiator, at least one storage device, and a storage switch having a linecard connected to the at least one initiator and the at least one storage device for communication with the at least one initiator and the at least one storage device, the method comprising:

providing, by the linecard of the storage switch, quality of service to the at least one initiator for accessing the at least one storage device in the storage network, wherein providing quality of service includes guaranteeing a minimum bandwidth to the at least one initiator to access the storage device by estimating an actual bandwidth utilized by the initiator, where the actual bandwidth is estimated by a number of requests per second times an average size of requests from the at least one initiator, and wherein a request includes the packets sent back and forth between the at least one initiator and the at least one storage device necessary to complete the request.

2. The method of claim 1 , wherein the step of providing quality of service includes controlling the number of packets from the at least one initiator to the at least one storage device during a period of time.

3. The method of claim 1 , wherein the step of providing quality of service includes controlling the number of requests from the at least one initiator to the at least one storage device.

4. The method of claim 1 , wherein the step of providing quality of service includes adjusting a number of concurrent requests allowed to be sent by the at least one initiator.

5. The method of claim 1 , wherein the step of guaranteeing a minimum bandwidth to the at least one initiator includes adjusting the number of requests allowed the at least one initiator to keep the bandwidth utilized by the at least one initiator within a specified range.

6. The method of claim 5 , further including:

guaranteeing up to a maximum bandwidth to the at least one initiator to access the at least one storage device; and

wherein adjusting the number of requests includes reducing the number of concurrent requests allowed by the at least one initiator when the actual bandwidth exceeds the maximum bandwidth.

7. A method for use in a storage network, the storage network including at least one initiator, at least one storage device, and at least one storage switch having a linecard, wherein the at least one initiator and the at least one storage device are both in communication with the linecard of the storage switch, the method comprising:

guaranteeing, by the linecard of the storage switch, a minimum bandwidth to the at least one initiator to access the at least one storage device in the storage network; and

estimating, by the linecard of the storage switch, an actual bandwidth utilized by the at least one initiator, where the actual bandwidth is estimated by a number of requests per second times an average size of requests from the at least one initiator, and wherein a request includes the packets sent back and forth between the at least one initiator and the at least one storage device necessary to complete the request.

8. The method of claim 7 , further comprising:

adjusting a number of concurrent requests allowed to be sent by the at least one initiator.

9. The method of claim 8 , wherein the step of adjusting includes: reducing the number of concurrent requests allowed to be sent by the at least one initiator.

10. The method of claim 8 , wherein the step of adjusting includes:

increasing the number of concurrent requests allowed to be sent by the at least one initiator.

11. The method of claim 7 , further including guaranteeing, by the storage switch up to a maximum bandwidth to the at least one initiator to access the storage device.

12. The method of claim 11 , further including:

reducing the number of concurrent requests allowed by the at least one initiator when the actual bandwidth exceeds its maximum bandwidth.

13. The method of claim 7 , wherein estimating the actual bandwidth includes determining if a buffer includes a number of packets beyond a specified threshold.

14. A method for use in a storage network, the storage network including a plurality of initiators, a plurality of targets, and a storage switch having a linecard for communicating with the at least one initiator and the at least one storage device, the method comprising:

guaranteeing, by the linecard of the storage switch, a respective minimum bandwidth for each of a plurality of connections, wherein each respective connection is a connection from a respective initiator to a respective target via the storage switch in the storage network;

monitoring, by the linecard of the storage switch, an actual bandwidth utilized by each initiator, where the actual bandwidth is estimated by a number of requests per second from the initiator times an average size of the requests from the initiator, where a request includes the packets sent back and forth between an initiator and a target necessary to complete the request; and

determining if the actual bandwidth used by one initiator is excessive, and, if excessive, adjusting, by the linecard of the storage switch, a number of allowed concurrent requests for at least one initiator.

15. The method of claim 14 , wherein monitoring the actual bandwidth includes determining if a buffer includes a number of packets beyond a specified threshold.

16. The method of claim 14 , wherein adjusting a number of allowed concurrent requests includes reducing the number of allowed concurrent requests to the one initiator that is using excessive bandwidth.

17. The method of claim 16 , wherein adjusting a number of allowed concurrent requests includes increasing the number of allowed concurrent requests to another initiator.

18. The method of claim 14 , wherein the targets are virtual targets.

19. The method of claim 14 , further including guaranteeing, by the storage switch, up to a respective maximum bandwidth for each of the plurality of connections, wherein determining if the actual bandwidth used by one initiator is excessive includes determining if the one initiator has exceeded its maximum bandwidth.

20. A method for use in a storage network, the storage network including at least one initiator, at least one storage device, and a storage switch having a linecard, wherein the at least one initiator and the at least one storage device are both in communication with the linecard of the storage switch, the method comprising:

providing a connection from the at least one initiator to the at least one storage device via the linecard of the storage switch in the storage network; and

adjusting, by the linecard of the storage switch, the number of requests allowed the at least one initiator to keep the bandwidth utilized by the at least one initiator within a specified range, wherein the bandwidth is estimated by a number of requests per second from the at least one initiator times an average size of the requests from the at least one initiator, and wherein a request includes the packets sent back and forth between the at least one initiator and the at least one storage device necessary to complete the request.

21. The method of claim 20 , wherein the number of requests allowed the at least one initiator is the number of concurrent requests allowed the at least one initiator.

22. A switch for use in a storage network, the switch having a linecard comprising:

a port to be coupled to an external device, wherein the external device includes at least one initiator and a storage device; and

a bandwidth controller, the bandwidth controller including a processor, a traffic manager, and a buffer, for controlling bandwidth through the port by controlling a number of requests per second times an average size of the requests so as to keep the bandwidth utilized by the at least one initiator within a specified range, where a request includes the packets sent back and forth between the at least one initiator and the storage device necessary to complete the request.

23. The switch of claim 22 , wherein the processor is a storage processor.

24. The switch of claim 22 , wherein the port and the bandwidth controller are on one of a plurality of linecards in the switch, wherein each linecard includes a respective port and a respective bandwidth controller.

25. A switch having a linecard, the linecard including:

a storage processor, including a request controller that estimates bandwidth by a number of requests per second times an average size of the requests, and wherein the request controller is designed to adjust the number of requests allowed to an initiator to keep the bandwidth utilized by the initiator within a specified range, where a request includes the packets sent back and forth between an initiator and a target necessary to complete the request;

a traffic manager in communication with the storage processor;

a buffer in communication with the traffic manager;

wherein if a specified threshold in the buffer is reached, the traffic manager is designed to activate the request controller to control the bandwidth.

26. A storage switch for use in a storage network, the storage switch having a linecard comprising:

a first port to be coupled to at least one initiator;

a second port to be coupled to at least one storage device; and

means for providing quality of service for a connection from the at least one initiator to the at least one storage device in the storage network, comprising:

means for guaranteeing a minimum bandwidth to at least one initiator to access a storage device; and

means for estimating an actual bandwidth utilized by the at least one initiator, where the actual bandwidth is estimated by the number of requests per second times the average size of the requests from the at least one initiator, and where a request includes the packets sent back and forth between the at least one initiator and the storage device necessary to complete the request.

27. The switch of claim 26 , wherein means for providing quality of service further comprises:

means for adjusting the number of concurrent requests allowed to be sent by the at least one initiator to keep the bandwidth utilized by the at least one initiator within a specified range having as a lower limit the minimum bandwidth.

28. The switch of claim 26 , wherein means for providing quality of service includes:

a processor;

a traffic manager; and

a buffer.

29. The switch of claim 28 , wherein the processor is a storage processor.

30. A storage network, including:

an initiator;

a storage device;

a switch having a linecard in communication with the initiator and the storage device;

wherein the linecard of the switch includes a traffic manager in communication with a buffer;

wherein when the buffer includes a number of packets from the initiator that exceeds a specified threshold, then the switch is designed to notify the initiator to reduce a number of concurrent requests, and where a request includes the packets sent back and forth between the an initiator and a target necessary to complete the request.

31. A machine readable media which has instructions stored thereon, which when executed on a linecard of a storage switch in a storage network including an initiator and a storage device in communication with the linecard of the storage switch causes the linecard of the storage switch to perform the following steps:

guaranteeing a minimum bandwidth to the initiator to access the storage device in the storage network; and

estimating an actual bandwidth utilized by the initiator, where the actual bandwidth is estimated by a number of requests per second times an average size of requests from the initiator, where a request includes the packets sent back and forth between the initiator and the storage device necessary to complete the request.

32. The machine readable media of claim 31 , further including instructions for performing the step of:

adjusting a number of concurrent requests allowed to be sent by the initiator.

33. The machine readable media of claim 32 , wherein the step of adjusting includes:

reducing the number of concurrent requests allowed to be sent by the initiator.

34. The machine readable media of claim 32 , wherein the step of adjusting includes:

increasing the number of concurrent requests allowed to be sent by the initiator.

35. The machine readable media of claim 31 , further including instructions for performing the step of:

guaranteeing, by the linecard of the storage switch, up to a maximum bandwidth to the initiator to access the storage device.

36. The machine readable media of claim 35 , further including instructions for performing the step of:

reducing the number of concurrent requests allowed by the initiator when it exceeds its maximum bandwidth.

37. The machine readable media of claim 31 , wherein estimating the actual bandwidth includes determining if a buffer includes a number of packets beyond a specified threshold.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 29, 2005
From: MARANTI NETWORKS, INC.
To: EMC CORPORATION
Reel/Frame 016327/0943 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2002
From: LOLAYEKAR, SANTOSH C.; CHENG, YU-PING; TAN, ENYEW
To: MARANTI NETWORKS, INC.
Reel/Frame 012516/0101 →
Continuity (2)
Provisional Application 6032570400 · Sep 28, 2001
Related Publication 20030079019A1 · Apr 24, 2003