Using storage system memory as host memory
Memory of a storage system is made available (i.e., exposed) for use as host memory of a host, for example, as an extension of the main memory of the host. The host may be directly connected to an internal fabric of the data storage system. Portions of the storage system memory (SSM) may be allocated for use as host memory, and this may be communicated to the host system. The host OS and applications executing thereon then may make use of the SSM as if it were memory of the host system, for example, as second-tier persistent memory. The amount of SSM made available may be dynamically increased and decreased. The SSM may be accessed by the host system as memory; i.e., in accordance with memory-based instructions, for example, using remote direct memory access instructions. The SSM may be write protected using mirroring, vaulting and other techniques.
1. For a data storage network including a data storage system and one or more host systems on which a plurality of applications execute, the data storage system including a memory, a method comprising:
allocating a first portion of the memory for use as host memory by a first of the one or more host systems, wherein the first portion of the memory is included in a global memory of the data storage system and wherein, prior to said allocating, the first portion is used for one or more data storage system internal operations including caching data of I/O operations received at the data storage system;
sending a communication to the first host system informing the first host system that the first portion is available for use by the first host system as host memory;
sending a memory instruction, that is initiated by and originates from the first host system, from a storage system interface of the first host system to the memory via a direct connection to an internal switching fabric of the data storage system, wherein the memory is connected to the internal switching fabric and wherein the memory instruction accesses the first portion of the memory to perform an operation for a first data, wherein the storage system interface of the first host directly sends the memory instruction to the memory over the direct connection and the internal switching fabric without using one or more directors or other compute resources of the data storage system, wherein the internal switching fabric is internal within the data storage system and is used for internal communications between components of the data storage system, wherein the components of the data storage system include the one or more directors and the memory that is connected to the internal switching fabric, and wherein the storage system interface is physically directly connected over the direct connection to the internal switching fabric of the data storage system, wherein the first host system uses the direct connection to the internal switching fabric to directly communicate with the memory to send the memory instruction and access the first portion of the memory of the data storage system; and
executing the memory instruction, that is initiated by and originates from the first host system, on the first portion for the first data.
2. The method of claim 1 , further comprising:
the memory of the data storage system receiving a remote direct memory access request from the first host system over the direct connection, the remote direct memory access request including the memory instruction for the first data.
3. The method of claim 1 , wherein said sending sends the memory instruction directly from the storage system interface over the direct connection to the internal switching fabric independent of IO protocol.
4. The method of claim 1 , further comprising:
mirroring the first data on the data storage system.
5. The method of claim 1 , further comprising:
allocating a second portion of the first memory for use as host memory by a second of the one or more host systems.
6. The method of claim 5 , wherein execution of a first application is distributed across a plurality of host systems, including the first host system and the second host system,
wherein the first data is data resulting from execution of the first application, and
wherein the method further comprises:
storing second data in the second portion of memory, wherein the second data results from execution of the first application on the second host system.
7. The method of claim 1 , further comprising:
dynamically modifying an amount of the memory allocated for use as host memory by the first host system.
8. For a data storage network including a data storage system and one or more host systems on which a plurality of applications execute, the data storage system including a memory, a system comprising:
a data storage system including memory; and
executable logic that implements a method including:
allocating a first portion of the memory for use as host memory by a first of the one or more host systems, wherein the first portion of the memory is included in a global memory of the data storage system and wherein, prior to said allocating, the first portion is used for one or more data storage system internal operations including caching data of I/O operations received at the data storage system;
sending a communication to the first host system informing the first host system that the first portion is available for use by the first host system as host memory;
sending a memory instruction, that is initiated by and originates from the first host system, from a storage system interface of the first host system to the memory via a direct connection to an internal switching fabric of the data storage system, wherein the memory is connected to the internal switching fabric and wherein the memory instruction accesses the first portion of the memory to perform an operation for a first data, wherein the storage system interface of the first host directly sends the memory instruction to the memory over the direct connection and the internal switching fabric without using one or more directors or other compute resources of the data storage system, wherein the internal switching fabric is internal within the data storage system and is used for internal communications between components of the data storage system, wherein the components of the data storage system include the one or more directors and the memory that is connected to the internal switching fabric, and wherein the storage system interface is physically directly connected over the direct connection to the internal switching fabric of the data storage system, wherein the first host system uses the direct connection to the internal switching fabric to directly communicate with the memory to send the memory instruction and access the first portion of the memory of the data storage system; and
executing the memory instruction, that is initiated by and originates from the first host system, on the first portion for the first data.
9. The system of claim 8 , wherein the method further includes:
the memory of the data storage system receiving a remote direct memory access request from the first host system over the direct connection, the remote direct memory access request including the memory instruction for the first data.
10. The system of claim 8 , wherein said sending sends the memory instruction directly from the storage system interface over the direct connection to the internal switching fabric independent of IO protocol.
11. The system of claim 8 , wherein the method further includes:
mirroring the first data on the data storage system.
12. The system of claim 8 , wherein the method further includes:
allocating a second portion of the first memory for use as host memory by a second of the one or more host systems.
13. The system of claim 12 , wherein execution of a first application is distributed across a plurality of host systems, including the first host system and the second host system,
wherein the first data is data resulting from execution of the first application, and
wherein the method further includes storing second data in the second portion of memory, wherein the second data results from execution of the first application on the second host system.
14. The system of claim 8 , wherein the method further includes:
dynamically modifying an amount of the memory allocated for use as host memory by the first host system.
15. For a data storage network including a data storage system and one or more host systems on which a plurality of applications execute, the data storage system including a memory, non-transitory computer-readable media having software stored thereon, the software comprising:
executable code that allocates a first portion of the memory for use as host memory by the first host system, wherein the first portion of the memory is included in a global memory of the data storage system and wherein, prior to allocating the first portion for use as host memory by the executable code that allocates, the first portion is used for one or more data storage system internal operations including caching data of I/O operations received at the data storage system;
executable code that sends a communication to the first host system informing the first host system that the first portion is available for use by the first host system a host memory;
executable code that sends a memory instruction, that is initiated by and originates from the first host system, from a storage system interface of the first host system to the memory via a direct connection to an internal switching fabric of the data storage system, wherein the memory is connected to the internal switching fabric and wherein the memory instruction accesses the first portion of the memory to perform an operation for a first data, wherein the storage system interface of the first host directly sends the memory instruction to the memory over the direct connection and the internal switching fabric without using one or more directors or other compute resources of the data storage system, wherein the internal switching fabric is internal within the data storage system and is used for internal communications between components of the data storage system, wherein the components of the data storage system include the one or more directors and the memory that is connected to the internal switching fabric, and wherein the storage system interface is physically directly connected over the direct connection to the internal switching fabric of the data storage system, wherein the first host system uses the direct connection to the internal switching fabric to directly communicate with the memory to send the memory instruction and access the first portion of the memory of the data storage system; and
executable code that executes the memory instruction, that is initiated by and originates from the first host system, on the first portion for the first data.
16. The non-transitory computer-readable media of claim 15 , the software further comprising:
executable code that controls the memory of the data storage system receiving a remote direct memory access request from the first host system over the direct connection, the remote direct memory access request including the memory instruction for the first data.
17. The non-transitory computer-readable media of claim 15 , wherein said sending sends the memory instruction directly from the storage system interface over the direct connection to the internal switching fabric independent of IO protocol.
18. The non-transitory computer-readable media of claim 15 , the software further comprising:
executable code that mirrors the first data on the data storage system.
19. The non-transitory computer-readable media of claim 15 , the software further comprising:
executable code that allocates a second portion of the first memory for use as host memory by a second of the one or more host systems.
20. The non-transitory computer-readable media of claim 15 , the software further comprising:
executable code that dynamically modifies an amount of the memory allocated for use as host memory by the first host system.