IP Library Granted Patent US 10,282,107
Granted Patent B1
US 10,282,107 · App. 14/985,769 · Granted May 7, 2019

Controlling I/O response time to meet service levels

Inventors: Owen Martin (Hopedale, MA); Hui Wang (Upton, MA); Jaeyoo Jung (Shrewsbury, MA); James L. Davidson (Tyngsboro, MA); Jeffrey Wilson (Franklin, MA)
Assignee: EMC IP Holding Company LLC
G06F3/0611G06F3/0647G06F3/0673
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,282,107
App. No.
14/985,769
Filed
Dec 31, 2015
Granted
May 7, 2019
Kind
B1
Examiner
HO, AARON D
Art Unit
2139
USPC
711/165
Abstract

Described are techniques for controlling I/O response time. It is determined whether a first observed I/O response time for a storage group exceeds I/O performance specified by a service level objective. If so, first processing may be performed to decrease I/O performance for the storage group. The first processing may include: for each I/O directed to the storage group that is received within a first subsequent time period, determining a first response time denoting an amount of time taken to service each I/O prior to returning a response for each I/O; determining whether the first response time for each I/O is less than a response time delay threshold; and if the first response time for each I/O is less than the response time delay threshold, applying a response time delay to each I/O prior to returning a response for each I/O.

Claims (59)

1. A method of controlling I/O response time comprising:

determining, for a time period, whether a first observed I/O response time for a storage group of one or more logical devices exceeds I/O performance specified by a service level objective, wherein the service level objective includes a minimum response time and a maximum response time; and

responsive to the first observed I/O response time exceeding I/O performance of the service level objective for the time period, performing first processing to decrease I/O performance for the storage group, said first processing comprising:

for each I/O directed to the storage group that is received by a data storage system within a first subsequent time period, determining a first response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O, wherein the first response time is a total amount of elapsed time measured from a starting point in time when said each I/O is received by the data storage system to an end point in time when the data storage system is ready to return a response for said each I/O;

determining whether the first response time for said each I/O is less than a response time delay threshold, wherein the response time delay threshold is an I/O response time that is selected in accordance with the minimum response time of the service level objective and that is less than the minimum response time of the service level objective; and

if the first response time for said each I/O is less than the response time delay threshold that is an I/O response time less than the minimum response time of the service level objective, applying a response time delay to said each I/O prior to returning a response for said each I/O, wherein subsequent to the end point in time when the data storage system is ready to return a response for said each I/O, the data storage system waits an additional amount of time denoted by the response time delay prior to returning the response for said each I/O; and

wherein the first processing includes additional processing that applies a plurality of response time delays, in a number of successive time periods subsequent to the first subsequent time period, to I/Os that are directed to the storage group and have associated response times less than the response time delay threshold, wherein said additional processing is performed until an observed I/O response time measured for the storage group for a time period is more than a delay removal threshold, wherein the delay removal threshold is more than the minimum response time of the service level objective and is less than the maximum response time of the service level objective.

2. The method of claim 1 , wherein said each I/O is delayed by a front end component by an amount of time denoted by the response time delay.

3. The method of claim 2 , wherein the front end component communicates with a host that sends said each I/O request to the data storage system providing physical storage for storing data of the storage group.

4. The method of claim 1 , wherein said additional processing further comprises:

receiving a second observed I/O response time measured for the storage group for the first subsequent time period;

determining whether the second observed I/O response time exceeds the delay removal threshold; and

responsive to determining that the second observed I/O response time does not exceed the delay removal threshold, performing second processing comprising:

for each I/O directed to the storage group that is received within a second subsequent time period, determining a second response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O;

determining whether the second response time for said each I/O is less than the response time delay threshold; and

if the second response time for said each I/O is less than the response time delay threshold, applying a second response time delay to said each I/O prior to returning a response for said each I/O, said second response time delay being larger than a first response time delay equal to the response time delay.

5. The method of claim 4 , wherein the first response time delay is an initial value and the second response time delay is determined in accordance with the initial value, the response time delay threshold and a number of consecutive time periods for which an observed I/O response time for the storage group does not exceed the delay removal threshold.

6. The method of claim 5 , wherein a plurality of observed I/O response times are determined for a plurality of subsequent time periods for the storage group and each of the plurality of observed I/O response times does not exceed the delay removal threshold, and the method further includes performing other processing for a first of the plurality of subsequent time periods comprising:

determining a current response time delay in accordance with an initial value used for a response time delay for the storage group, the response time delay threshold and a number of time periods for which an observed I/O response time for the storage group has not exceeded the delay removal threshold; and

applying the current response time delay to each I/O that is directed to the storage group, received in the first of the plurality of subsequent time periods, and has a measured response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O and wherein the measured response time is less than the response time delay threshold for the storage group.

7. The method of claim 6 , further comprising:

determining that a next observed I/O response time for the storage group exceeds the delay removal threshold and performing other processing to apply another time delay that is less than a maximum response time delay applied previously in any of the plurality of subsequent time periods for the storage group.

8. The method of claim 1 , wherein the observed I/O response time exceeds performance of the service level objective when the observed I/O response time is less than the minimum response time.

9. The method of claim 1 , wherein the first observed I/O response time is an average I/O response time for I/Os directed to the storage group.

10. The method of claim 1 , wherein each of the logical devices of the storage group is a virtually provisioned logical device.

11. The method of claim 1 , wherein the method of controlling I/O response time is performed by a data storage optimizer that performs data movement optimizations, and the method further comprises:

determining whether there is a demotion that can be performed for the storage group to move data of the storage group from a first storage tier to a second storage tier that has a lower performance ranking that the first storage tier; and

if it is determined that there is no demotion that can be performed for the storage group, performing said first processing to decrease I/O performance for the storage group.

12. The method of claim 11 , further comprising:

if it is determined that there is a demotion that can be performed for the storage group, performing said demotion to decrease I/O performance for the storage group.

13. The method of claim 1 , wherein the method further comprises:

determining for a first time period whether a corresponding observed I/O response time for the storage group exceeds performance of the service level objective; and

responsive to determining the corresponding observed I/O response time for the storage group exceeds performance of the service level objective, selecting, in accordance with one or more criteria, an action performed to reduce I/O performance of the storage group, wherein the one or more criteria includes a first value denoting a percentage of I/Os directed to the storage group that are read cache hits, a second value denoting a percentage of I/Os directed to the storage group that are writes, and a number of data demotions previously performed with respect to data from the storage group to reduce performance of the storage group.

14. The method of claim 13 , wherein the action is any of a data demotion that demotes data from the storage group, and processing to apply a response time delay to selected I/Os directed to the storage group where the selected I/Os would have a response time less than the response time delay threshold without applying a response time delay.

15. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of controlling I/O response time comprising:

determining, for a time period, whether a first observed I/O response time for a storage group of one or more logical devices exceeds I/O performance specified by a service level objective, wherein the service level objective includes a minimum response time and a maximum response time; and

responsive to the first observed I/O response time exceeding I/O performance of the service level objective for the time period, performing first processing to decrease I/O performance for the storage group, said first processing comprising:

for each I/O directed to the storage group that is received by a data storage system within a first subsequent time period, determining a first response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O, wherein the first response time is a total amount of elapsed time measured from a starting point in time when said each I/O is received by the data storage system to an end point in time when the data storage system is ready to return a response for said each I/O;

determining whether the first response time for said each I/O is less than a response time delay threshold, wherein the response time delay threshold is an I/O response time that is selected in accordance with the minimum response time of the service level objective and that is less than the minimum response time of the service level objective; and

if the first response time for said each I/O is less than the response time delay threshold that is an I/O response time less than the minimum response time of the service level objective, applying a response time delay to said each I/O prior to returning a response for said each I/O, wherein subsequent to the end point in time when the data storage system is ready to return a response for said each I/O, the data storage system waits an additional amount of time denoted by the response time delay prior to returning the response for said each I/O; and

wherein the first processing includes additional processing that applies a plurality of response time delays, in a number of successive time periods subsequent to the first subsequent time period, to I/Os that are directed to the storage group and have associated response times less than the response time delay threshold, wherein said additional processing is performed until an observed I/O response time measured for the storage group for a time period is more than a delay removal threshold, wherein the delay removal threshold is more than the minimum response time of the service level objective and is less than the maximum response time of the service level objective.

16. The non-transitory computer readable medium of claim 15 , wherein said each I/O is delayed by a front end component by an amount of time denoted by the response time delay.

17. The non-transitory computer readable medium of claim 16 , wherein the front end component communicates with a host that sends said each I/O request to the data storage system providing physical storage for storing data of the storage group.

18. The non-transitory computer readable medium of claim 15 , wherein said additional processing further comprises:

receiving a second observed I/O response time measured for the storage group for the first subsequent time period;

determining whether the second observed I/O response time exceeds the delay removal threshold;

responsive to determining that the second observed I/O response time does not exceed the delay removal threshold, performing second processing comprising:

for each I/O directed to the storage group that is received within a second subsequent time period, determining a second response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O;

determining whether the second response time for said each I/O is less than the response time delay threshold; and

if the second response time for said each I/O is less than the response time delay threshold, applying a second response time delay to said each I/O prior to returning a response for said each I/O, said second response time delay being larger than a first response time delay equal to the response time delay.

19. A system comprising:

a processor; and

a memory comprising code stored thereon, that when executed, performs a method of controlling I/O response time comprising:

determining, for a time period, whether a first observed I/O response time for a storage group of one or more logical devices exceeds I/O performance specified by a service level objective, wherein the service level objective includes a minimum response time and a maximum response time; and

responsive to the first observed I/O response time exceeding I/O performance of the service level objective for the time period, performing first processing to decrease I/O performance for the storage group, said first processing comprising:

for each I/O directed to the storage group that is received by a data storage system within a first subsequent time period, determining a first response time denoting an amount of time taken to service said each I/O prior to returning a response for said each I/O, wherein the first response time is a total amount of elapsed time measured from a starting point in time when said each I/O is received by the data storage system to an end point in time when the data storage system is ready to return a response for said each I/O;

determining whether the first response time for said each I/O is less than a response time delay threshold, wherein the response time delay threshold is an I/O response time that is selected in accordance with the minimum response time of the service level objective and that is less than the minimum response time of the service level objective; and

if the first response time for said each I/O is less than the response time delay threshold that is an I/O response time less than the minimum response time of the service level objective, applying a response time delay to said each I/O prior to returning a response for said each I/O, wherein subsequent to the end point in time when the data storage system is ready to return a response for said each I/O, the data storage system waits an additional amount of time denoted by the response time delay prior to returning the response for said each I/O; and

wherein the first processing includes additional processing that applies a plurality of response time delays, in a number of successive time periods subsequent to the first subsequent time period, to I/Os that are directed to the storage group and have associated response times less than the response time delay threshold, wherein said additional processing is performed until an observed I/O response time measured for the storage group for a time period is more than a delay removal threshold, wherein the delay removal threshold is more than the minimum response time of the service level objective and is less than the maximum response time of the service level objective.

Assignments (11)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (045455/0001) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061753/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (040136/0001) Recorded Apr 26, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO ASAP SOFTWARE EXPRESS, INC.); DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC CORPORATION (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MAGINATICS LLC); EMC IP HOLDING COMPANY LLC (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO MOZY, INC.); SCALEIO LLC
Reel/Frame 061324/0001 →
RELEASE OF SECURITY INTEREST Recorded Nov 3, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL, L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058216/0001 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: EMC CORPORATION
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 040203/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
Reel/Frame 040136/0001 →
SECURITY AGREEMENT Recorded Sep 21, 2016
From: ASAP SOFTWARE EXPRESS, INC.; AVENTAIL LLC; CREDANT TECHNOLOGIES, INC.; DELL USA L.P.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL SOFTWARE INC.; DELL SYSTEMS CORPORATION; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; FORCE10 NETWORKS, INC.; MAGINATICS LLC; MOZY, INC.; SCALEIO LLC; SPANNING CLOUD APPS LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 040134/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 23, 2016
From: JUNG, JAEYOO; WILSON, JEFFREY
To: EMC CORPORATION
Reel/Frame 037800/0738 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 31, 2015
From: MARTIN, OWEN; WANG, HUI; DAVIDSON, JAMES L.
To: EMC CORPORATION
Reel/Frame 037390/0311 →
Cited By (1)
US 12,307,127