Front end traffic handling in modular switched fabric based data storage systems
Systems, methods, apparatuses, and software for data storage systems are provided herein. In one example, a data storage system is provided that includes storage drives each comprising a PCIe interface, and configured to store data and retrieve the data stored on associated storage media responsive to data transactions received over a switched PCIe fabric. The data storage system includes processors configured to each manage only an associated subset of the storage drives over the switched PCIe fabric. A first processor is configured to identify first data packets received over a network interface associated with the first processor within a network buffer of the first processor as comprising a storage operation associated with at least one of the plurality of storage drives managed by a second processor, and responsively transfer the first data packets into a network buffer of the second processor.
1. A data storage system, comprising:
a plurality of storage drives each comprising a Peripheral Component Interconnect Express (PCIe) interface, and configured to store data and retrieve the data stored on associated storage media responsive to storage operations received over a switched PCIe fabric;
a plurality of processors configured to each manage only an associated subset of the plurality of storage drives over the switched PCIe fabric, wherein each of the processors is associated with a network interface configured to receive data packets related to the storage operations for any of the plurality of storage drives;
a first processor configured to identify first data packets received over a network interface associated with the first processor within a layer-2 network buffer of the first processor as comprising a storage operation associated with at least one of the plurality of storage drives managed by a second processor, and responsively transfer the first data packets into a layer-2 network buffer of the second processor; and
the second processor configured to identify the first data packets within the layer-2 network buffer of the second processor and handle the storage operation associated with the first data packets as if a network interface associated with the second processor had received the first data packets by at least processing the first data packets up a network stack of the second processor for handling of the storage operation associated with the first data packets.
2. The data storage system of claim 1 , wherein the first data packets comprise Non-Volatile Memory Express (NVMe) data packets for an NVMe storage operation.
3. The data storage system of claim 1 , wherein the first data packets comprise iSCSI (Internet Small Computer System Interface) data packets for an iSCSI storage operation.
4. The data storage system of claim 3 , comprising:
the first processor configured to identify the first data packets as being associated with the at least one of the plurality of storage drives managed by a second processor by at least processing a logical block address (LBA) and a iSCSI target identifier of an iSCSI header associated with the iSCSI data packets.
5. The data storage system of claim 1 , comprising:
the first processor configured to identify first data frames received over an Non-Volatile Memory Express (NVMe) interface associated with the first processor within a layer 2 NVMe buffer of the first processor as comprising an NVMe storage operation associated with at least one of the plurality of storage drives managed by the second processor, and responsively transfer the first data frames into a layer 2 NVMe buffer of the second processor;
the second processor configured to identify the first data frames within the layer 2 NVMe buffer of the second processor and handle the NVMe storage operation associated with the first data frames as if an NVMe interface associated with the second processor had received the first data frames.
6. The data storage system of claim 1 , wherein the network interface associated with the first processor and the network interface associated with the second processor are each have different Internet Protocol (IP) address and are both associated with a first Internet Small Computer System Interface (iSCSI) portal group; and comprising:
the first processor configured to identify the first data packets as associated with the first iSCSI portal group and associated with the at least one of the plurality of storage drives managed by a second processor, and responsively transfer the first data packets for handling by the second processor associated with the first iSCSI portal group as if the second network interface had received the data packets.
7. The data storage system of claim 1 , comprising:
the first processor configured to transfer the first data packets into the layer-2 network buffer of the second processor over the switched PCIe fabric using a PCIe address space shared at least among the first processor and the second processor.
8. A method of operating a data storage system, the method comprising:
in a plurality of storage drives, storing data and retrieving the data stored on associated storage media responsive to storage operations received over a Peripheral Component Interconnect Express (PCIe) interface;
in each of a plurality of processors communicatively coupled to the plurality of storage drives over a PCIe fabric, managing only an associated subset of the plurality of storage drives over the switched PCIe fabric;
in a first processor, identifying first data packets received over a network interface associated with the first processor within a layer-2 network buffer of the first processor as comprising a storage operation associated with at least one of the plurality of storage drives managed by a second processor, and responsively transferring the first data packets into a layer-2 network buffer of the second processor;
the second processor configured to identify the first data packets within the layer-2 network buffer of the second processor and handle the storage operation associated with the first data packets as if a network interface associated with the second processor had received the first data packets by at least processing the first data packets up a network stack of the second processor for handling of the storage operation associated with the first data packets.
9. The method of claim 8 , wherein the first data packets comprise Non-Volatile Memory Express (NVMe) data packets for an NVMe storage operation.
10. The method of claim 8 , wherein the first data packets comprise iSCSI (Internet Small Computer System Interface) data packets for an iSCSI storage operation.
11. The method of claim 10 , further comprising:
in the first processor, identifying the first data packets as being associated with the at least one of the plurality of storage drives managed by a second processor by at least processing a logical block address (LBA) and a iSCSI target identifier of an iSCSI header associated with the iSCSI data packets.
12. The method of claim 8 , further comprising:
in the first processor, identifying first data frames received over an Non-Volatile Memory Express (NVMe) interface associated with the first processor within a layer 2 NVMe buffer of the first processor as comprising an NVMe storage operation associated with at least one of the plurality of storage drives managed by the second processor, and responsively transferring the first data frames into a layer 2 NVMe buffer of the second processor;
in the second processor, identifying the first data frames within the layer 2 NVMe buffer of the second processor and handling the NVMe storage operation associated with the first data frames as if an NVMe interface associated with the second processor had received the first data frames.
13. The method of claim 8 , wherein the network interface associated with the first processor and the network interface associated with the second processor are each have different Internet Protocol (IP) address and are both associated with a first Internet Small Computer System Interface (iSCSI) portal group; and further comprising:
in the first processor configured, identifying the first data packets as associated with the first iSCSI portal group and associated with the at least one of the plurality of storage drives managed by a second processor, and responsively transferring the first data packets for handling by the second processor associated with the first iSCSI portal group as if the second network interface had received the data packets.
14. The method of claim 8 , further comprising:
in the first processor, transferring the first data packets into the layer-2 network buffer of the second processor over the switched PCIe fabric using a PCIe address space shared at least among the first processor and the second processor.
15. A modular data storage system, comprising:
a plurality of storage drives each comprising a Peripheral Component Interconnect Express (PCIe) interface, and configured to store data and retrieve the data stored on associated storage media responsive to storage operations received over a switched PCIe fabric;
a plurality of processing modules comprising one or more processors configured to each manage only an associated subset of the plurality of storage drives over the switched PCIe fabric, wherein each of the processors is associated with a network interface over the switched PCIe fabric configured to receive data packets related to the storage operations for any of the plurality of storage drives;
in a first processor, identifying first data frames received over a Non-Volatile Memory Express (NVMe) interface associated with the first processor within a layer 2 NVMe buffer of the first processor as comprising a storage operation associated with at least one of the plurality of storage drives managed by a second processor, and responsively transferring the first data frames into a layer 2 NVMe buffer of the second processor; and
in the second processor, identifying the first data frames within the layer 2 NVMe buffer of the second processor and handling the storage operation associated with the first data frames as if an NVMe interface associated with the second processor had received the first data frames.