Storage system having storage engines with multi-initiator host adapter and fabric chaining
A storage system includes four storage engines, each storage engine including two compute nodes. Eight point-to-point connections are used to interconnect pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system. Atomic operations can be initiated by any compute node on any other compute node. Atomic operations received by a compute node on one of the point-to-point connections will be forwarded on the other point-to-point connection if the atomic operation is not directed to the compute node. During normal operation, atomic operations on a given compute node are performed on a host adapter associated with the compute node. Upon failure of the host adapter associated with the compute node, atomic operations may be performed on the compute node using the host adapter of the other compute node of the storage engine.
1. A storage system, comprising:
a storage engine having a first compute node, a second compute node, a first fabric adapter, and a second fabric adapter, the first compute node having a first memory and the second compute node having a second memory;
a first internal connection between the first compute node and the first fabric adapter;
a second internal connection between the first compute node and the second fabric adapter;
a third internal connection between the second compute node and the first fabric adapter; and
a fourth internal connection between the second compute node and the second fabric adapter;
wherein the first fabric adapter contains a first multi-initiating module configured to enable both the first compute node and the second compute node to directly initiate atomic memory access operations on the first memory through the first fabric adapter, and contains a first fabric chaining module configured to enable memory access operations received on a first external connection to be re-transmitted out a second external connection; and
wherein the second fabric adapter contains a second multi-initiating module configured to enable both the second compute node and the first compute node to directly initiate atomic memory access operations on the second memory through the second fabric adapter, and contains a second fabric chaining module configured to enable memory access operations received on a third external connection to be re-transmitted out a fourth external connection.
2. A storage engine, comprising:
a first compute node with an associated first fabric access module, the first fabric access module including a first host adapter, a first multi-initiating module, and a first fabric chaining module;
a second compute node with an associated second fabric access module, the second fabric access module including a second host adapter, a second multi-initiating module, and a second fabric chaining module; and
an interconnection system interconnecting the first compute node with the first fabric access module and with the second fabric access module, and interconnecting the second compute node with the second fabric access module and with the first fabric access module;
wherein each fabric chaining module is connected to two external fabric access ports and configured to forward a first subset of memory operations received on a first of the two external fabric access ports out the second of the two external fabric access ports; and
wherein the storage engine is implemented using a single physical enclosure.
3. The storage engine of claim 2 , wherein each fabric chaining module is further configured to forward a second subset of memory operations received on a first of the two external fabric access ports to the first compute node on the interconnection system.
4. A storage system, comprising:
four storage engines, each storage engine including two compute nodes; and
eight point-to-point connections, each point-to-point connection interconnecting pairs of compute nodes on different storage engines, such that each compute node is connected to exactly two other compute nodes of the storage system.
5. The storage system of claim 4 , wherein each compute node has a respective fabric access module including a host adapter, a multi-initiating module, and a fabric chaining module.
6. The storage system of claim 5 , wherein each storage engine comprises an internal interconnection system connecting each of the two compute nodes with both of the respective fabric access modules.
7. The storage system of claim 6 , wherein each compute node has a respective memory, and the respective host adapter of each compute node is configured to implement atomic operations on the respective memory of its compute node.
8. The storage system of claim 6 , wherein the multi-initiating module is configured to enable atomic operations to be initiated by either compute node on a memory of the other compute node.
9. The storage system of claim 7 , wherein the multi-initiating module of a first compute node of one of the storage engines is configured to enable atomic operations to be initiated on the memory of a second compute node of the one of the storage engines, only upon failure of a respective fabric access module of the first compute node.
10. The storage system of claim 5 , wherein each fabric access module is connected to two of the point-to-point connections.
11. The storage system of claim 10 , wherein each fabric chaining module is configured to enable an atomic operation received on one of the point-to-point connections to be redirected out onto the other of the point-to-point connections.