IP Library Granted Patent US 10,482,023
Granted Patent B1
US 10,482,023 · App. 15/279,787 · Granted Nov 19, 2019

I/O path optimization based on cache slot location

Inventors: Douglas E. LeCrone (Hopkinton, MA); Paul A. Linstead (Shrewsbury, MA)
Assignee: EMC IP Holding Company LLC
G06F12/0888G06F3/061G06F3/0605G06F3/067G06F3/0631G06F3/0647G06F3/0659G06F3/0665G06F12/0813H04L67/1097H04L67/2842G06F2212/1016G06F2212/152G06F2212/154G06F2212/263G06F2212/507G06F2212/604
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,482,023
App. No.
15/279,787
Granted
Nov 19, 2019
Kind
B1
Abstract

Processing an I/O operation may include the host selecting one of the available paths over which to send each I/O operation to the data storage system. The selected path may be to a particular director that has responsibility for cache slot allocation and locally accessing the cache slot predicted to include the data of the I/O operation. The host may understand the cache slot allocation algorithm used on the data storage system and how cache slots are allocated for particular logical devices and tracks or locations on the logical devices. The host may direct I/Os down a path to a particular director that has, or will allocate, the cache slot used for the I/Os. There may be multiple directors in a data storage system including a distributed global memory. Each director may locally access a group of cache slots and communicate over a fabric to access the distributed global memory.

Claims (96)

1. A method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein said predicting, said determining and said sending are performed responsive to the I/O operation being a request that reads or writes a first amount of data that does not exceed a maximum size, and wherein responsive to determining the I/O operation exceeds a maximum size, sending the I/O operation to any one of a plurality of directors of the data storage system selected independent of which particular director of the plurality of directors is predicted by the cache slot allocation algorithm to locally access the first cache slot.

2. The method of claim 1 , wherein said predicting and said determining are performed by an I/O driver of the host.

3. The method of claim 1 , wherein the target address denotes a logical address or location on the logical device.

4. The method of claim 3 , wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director.

5. The method of claim 4 , wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions.

6. The method of claim 5 , wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory.

7. A method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein each of the plurality of directors is configured to send all requests to access the distributed global memory over the fabric if processor utilization of said each director exceeds a maximum threshold utilization level.

8. The method of claim 7 , further comprising:

determining, by a first of the plurality of directors, that processor utilization of said first director exceeds the maximum threshold utilization level; and

responsive to determining that processor utilization of the first director exceeds the maximum threshold utilization level, issuing a request by said first director over the fabric to access first data stored in a first location of one of the plurality of global memory portions that is locally accessible to the first director.

9. The method of claim 8 , further comprising:

returning, over the fabric to said first director, first data stored in the first location of the one global memory portion that is locally accessible to the first director.

10. The method of claim 6 , wherein each of the plurality of directors is configured to send, over the fabric, requests that transfer an amount of data exceeding a defined maximum amount, wherein the requests access locations in the distributed global memory.

11. A method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory wherein each of the plurality of directors is configured to send, over the fabric, requests that transfer an amount of data exceeding a defined maximum amount, wherein the requests access locations in the distributed global memory, and wherein the method further comprises:

determining, by a first of the plurality of directors, that a first request to access a first location in the distributed global memory transfers a first amount of data exceeding the defined maximum amount, wherein the first location denotes a global memory location of the distributed global memory and the global memory location is locally accessible to the first director; and

responsive to determining that the first amount exceeds the defined maximum amount, issuing the first request by the first director over the fabric.

12. The method of claim 11 , further comprising:

returning, over the fabric to the first director, first data stored in the first location denoting the global memory location.

13. A method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein the method further comprises:

determining whether the I/O operation accesses data spanning multiple cache slots of the cache of the data storage system, wherein each of the multiple cache slots is locally accessible to a different one of the plurality of directors;

responsive to determining the I/O operation accesses data spanning multiple cache slots each locally accessible to a different one of the plurality of directors, partitioning the I/O operation into multiple I/O operations wherein each of the multiple I/O operations accesses data of a different one of the multiple cache slots; and

issuing each of the multiple I/O operations accessing said different one of the multiple cache slots over a particular path to one of the plurality of directors that locally accesses said different one of the multiple cache slots.

14. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs method of processing an I/O operation comprising:

receiving, on a host, the I/O operation that reads or writes first metadata of a logical device having storage provisioned on a data storage system;

predicting, in accordance with an algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store the first metadata for the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein said predicting, said determining and said sending are performed responsive to the I/O operation being a request that reads or writes a first amount of data that does not exceed a maximum size, and wherein responsive to determining the I/O operation exceeds a maximum size, sending the I/O operation to any one of a plurality of directors of the data storage system selected independent of which particular director of the plurality of directors is predicted by the cache slot allocation algorithm to locally access the first cache slot.

15. A system comprising:

a processor; and

a memory comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation that reads or writes first data stored on a data storage system;

predicting, in accordance with an algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store the first data;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein said predicting, said determining and said sending are performed responsive to the I/O operation being a request that reads or writes a first amount of data that does not exceed a maximum size, and wherein responsive to determining the I/O operation exceeds a maximum size, sending the I/O operation to any one of a plurality of directors of the data storage system selected independent of which particular director of the plurality of directors is predicted by the cache slot allocation algorithm to locally access the first cache slot.

16. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein each of the plurality of directors is configured to send all requests to access the distributed global memory over the fabric if processor utilization of said each director exceeds a maximum threshold utilization level.

17. A system comprising:

a processor; and

a memory comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein each of the plurality of directors is configured to send all requests to access the distributed global memory over the fabric if processor utilization of said each director exceeds a maximum threshold utilization level.

18. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory wherein each of the plurality of directors is configured to send, over the fabric, requests that transfer an amount of data exceeding a defined maximum amount, wherein the requests access locations in the distributed global memory, and wherein the method further comprises:

determining, by a first of the plurality of directors, that a first request to access a first location in the distributed global memory transfers a first amount of data exceeding the defined maximum amount, wherein the first location denotes a global memory location of the distributed global memory and the global memory location is locally accessible to the first director; and

responsive to determining that the first amount exceeds the defined maximum amount, issuing the first request by the first director over the fabric.

19. A system comprising:

a processor; and

a memory comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory wherein each of the plurality of directors is configured to send, over the fabric, requests that transfer an amount of data exceeding a defined maximum amount, wherein the requests access locations in the distributed global memory, and wherein the method further comprises:

determining, by a first of the plurality of directors, that a first request to access a first location in the distributed global memory transfers a first amount of data exceeding the defined maximum amount, wherein the first location denotes a global memory location of the distributed global memory and the global memory location is locally accessible to the first director; and

responsive to determining that the first amount exceeds the defined maximum amount, issuing the first request by the first director over the fabric.

20. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein the method further comprises:

determining whether the I/O operation accesses data spanning multiple cache slots of the cache of the data storage system, wherein each of the multiple cache slots is locally accessible to a different one of the plurality of directors;

responsive to determining the I/O operation accesses data spanning multiple cache slots each locally accessible to a different one of the plurality of directors, partitioning the I/O operation into multiple I/O operations wherein each of the multiple I/O operations accesses data of a different one of the multiple cache slots; and

issuing each of the multiple I/O operations accessing said different one of the multiple cache slots over a particular path to one of the plurality of directors that locally accesses said different one of the multiple cache slots.

21. A system comprising:

a processor; and

a memory comprising code stored thereon that, when executed, performs a method of processing an I/O operation comprising:

receiving, on a host, the I/O operation directed to a target location of a logical device having storage provisioned on a data storage system;

predicting, in accordance with a cache slot allocation algorithm used by the data storage system, a first cache slot of a cache of the data storage system, wherein the first cache slot is expected to be used to store cached data for the target location of the logical device;

determining, by the host, a director of the data storage system, wherein the director locally accesses the first cache slot of the cache of the data storage system; and

sending the I/O operation from the host to the data storage system over a path, said path being a physical connection from the host to the director of the data storage system, wherein the target address denotes a logical address or location on the logical device, wherein the cache slot allocation algorithm maps the logical device and logical address or location on the logical device to which the I/O operation is directed to the first cache slot, and wherein the first cache slot is included in a first global memory portion that is local with respect to the director, wherein the data storage system includes a plurality of directors comprising the director, the data storage system further including a distributed global memory comprising a plurality of global memory portions including the first global memory portion, each of the plurality of directors locally accessing a different one of the plurality of global memory portions, and wherein the cache of the data storage system comprises groups of multiple cache slots, each of the groups of multiple cache slots being included in a different one of the global memory portions, wherein each of the plurality of directors is configured to communicate over a fabric to access the distributed global memory, and wherein the method further comprises:

determining whether the I/O operation accesses data spanning multiple cache slots of the cache of the data storage system, wherein each of the multiple cache slots is locally accessible to a different one of the plurality of directors;

responsive to determining the I/O operation accesses data spanning multiple cache slots each locally accessible to a different one of the plurality of directors, partitioning the I/O operation into multiple I/O operations wherein each of the multiple I/O operations accesses data of a different one of the multiple cache slots; and

issuing each of the multiple I/O operations accessing said different one of the multiple cache slots over a particular path to one of the plurality of directors that locally accesses said different one of the multiple cache slots.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (052216/0758) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 060438/0680 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST AF REEL 052243 FRAME 0773 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0152 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 26, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052243/0773 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Mar 24, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 052216/0758 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2016
From: LECRONE, DOUGLAS E.; LINSTEAD, PAUL A.
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 039892/0759 →