Bridge device data transfer for data storage device arrays
Systems, methods, and a bridge device for a data storage device array that enables data transfer between data storage devices without transferring data to the host system are described. The bridge device receives a data transfer command from the host system, such as a fused read and write command, and converts it to separate read and write commands for the source and destination storage devices. The bridge device may operate in a non-volatile memory express over fabric (NVMe-oF) environment and manage the command queues, connection verifications, and command conversions to enable data transfer among storage devices connected to a shared peripheral component interface express (PCIe) root complex without using host network bandwidth or memory.
1 . A system, comprising:
a storage interface configured to communicate with a plurality of data storage devices;
a host interface configured to communicate with at least one host system; and
a bridge device comprising at least one controller configured to, alone or in combination:
provide, through the host interface, a target device for a network storage protocol configured to receive encapsulated storage commands from the at least one host system;
establish, through the storage interface, communication with the plurality of data storage devices using a bus storage protocol, wherein the bridge device is configured as an initiator device for the plurality of data storage devices as target devices;
receive, from the at least one host system, a data transfer command identifying a target data unit, a source data storage device, and a destination data storage device, wherein:
the plurality of data storage devices includes the source data storage device and the destination data storage device; and
the data transfer command comprises a fused command that includes a first set of parameters for a modified read command for the source data storage device and a second set of parameters for a modified write command for the destination data storage device; and
convert the data transfer command into:
a read command to the source data storage device for the target data unit, wherein the source data storage device is configured to, responsive to the read command:
read the target data unit from a non-volatile storage medium of the source data storage device; and
return the target data unit to a buffer location for the target data unit; and
a write command to the destination data storage device for the target data unit, wherein:
converting the data transfer command comprises determining:
additional parameters added to the first set of parameters for the read command to the source data storage device; and
additional parameters added to the second set of parameters for the write command to the destination data storage device;
the source data storage device is configured to, responsive to the read command:
read the target data unit from a non-volatile storage medium of the source data storage device; and
return the target data unit to a buffer location for the target data unit; and
the destination data storage device is configured to, responsive to the write command:
read the target data unit for the write command from the buffer location; and
write the target data unit to a non-volatile storage medium of the destination data storage device.
2 . The system of claim 1 , wherein the at least one controller is further configured to, alone or in combination:
manage storage interface connections with the plurality of data storage devices using a shared interface root complex;
verify, responsive to receiving the data transfer command:
a first connection to the source data storage device; and
a second connection to the destination data storage device; and
send, responsive to verification being unsuccessful, an error message to the at least one host system.
3 . The system of claim 1 , wherein:
the data transfer command is received from the at least one host system using the network storage protocol to encapsulate the data transfer command;
converting the data transfer command further comprises removing encapsulation of the data transfer command prior to generating the read command and the write command; and
the at least one controller is further configured to, alone or in combination and using the bus storage protocol:
send the read command to the source data storage device; and
send the write command to the destination data storage device.
4 . The system of claim 1 , wherein:
converting the data transfer command further comprises:
generating the read command for the target data unit indicating the buffer location for read data; and
generating the write command for the target data unit indicating the buffer location for the read data;
the buffer location is selected from:
a bridge device memory; or
a controller memory buffer of the destination data storage device;
the buffer location is not in the at least one host system;
the buffer location is not indicated in the data transfer command; and
the at least one controller is further configured to, alone or in combination:
determine the buffer location for the data transfer command;
send the read command to the source data storage device; and
send the write command to the destination data storage device.
5 . The system of claim 4 , wherein determining the buffer location comprises at least one of:
determining at least one scatter gather list for the buffer location; and
determining at least one physical region page for the buffer location.
6 . The system of claim 4 , wherein:
receiving the data transfer command uses a first set of command queues in the bridge device configured for host commands; and
sending the read command and the write command uses a second set of command queues in the bridge device configured for data storage device commands.
7 . The system of claim 1 , wherein the destination data storage device:
comprises a controller memory buffer configured to include the buffer location; and
provides, for the source data storage device, direct memory access to the controller memory buffer through a shared interface root complex.
8 . The system of claim 7 , wherein the target data unit is not stored in a bridge device memory between reading the target data unit from the source data storage device and writing the target data unit to the non-volatile storage medium of the destination data storage device.
9 . The system of claim 1 , further comprising:
a switch configured to provide a bus connection between the bridge device and the plurality of data storage devices; and
the plurality of data storage devices, wherein each data storage device of the plurality of data storage devices comprises:
a non-volatile storage medium configured to store host data units;
a device storage interface configured for communication with the bridge device; and
a device controller configured to receive storage commands using the bus storage protocol.
10 . A computer-implemented method, comprising:
providing, by a bridge device and through a host interface, a target device for a network storage protocol configured to receive encapsulated storage commands from at least one host system;
establishing, by the bridge device and through a storage interface, communication with a plurality of data storage devices using a bus storage protocol, wherein the bridge device is configured as an initiator device for the plurality of data storage devices as target devices;
receiving, by the bridge device and from the at least one host system, a data transfer command identifying a target data unit, a source data storage device, and a destination data storage device, wherein:
the plurality of data storage devices includes the source data storage device and the destination data storage device; and
the data transfer command comprises a fused command that includes a first set of parameters for a modified read command for the source data storage device and a second set of parameters for a modified write command for the destination data storage device;
converting, by the bridge device, the data transfer command into:
a read command to the source data storage device for the target data unit; and
a write command to the destination data storage device for the target data unit, wherein converting the data transfer command comprises determining:
additional parameters added to the first set of parameters for the read command to the source data storage device; and
additional parameters added to the second set of parameters for the write command to the destination data storage device;
reading, by the source data storage device and responsive to the read command, the target data unit from a non-volatile storage medium of the source data storage device;
writing, by the source data storage device and responsive to reading the target data unit, the target data unit to a buffer location;
reading, by the destination data storage device and responsive to the write command, the target data unit from the buffer location; and
writing, by the destination data storage device and responsive to reading the target data unit, the target data unit to a non-volatile storage medium of the destination data storage device.
11 . The computer-implemented method of claim 10 , further comprising:
managing, by the bridge device, storage interface connections with the plurality of data storage devices using a shared interface root complex;
verifying, by the bridge device and responsive to receiving the data transfer command:
a first connection to the source data storage device; and
a second connection to the destination data storage device; and
sending, by the bridge device and responsive to verification being unsuccessful, an error message to the at least one host system.
12 . The computer-implemented method of claim 10 , further comprising:
sending, by the bridge device, the write command to the destination data storage device using the bus storage protocol, wherein:
the data transfer command is received from the at least one host system using the network storage protocol to encapsulate the data transfer command; and
converting the data transfer command further comprises removing encapsulation of the data transfer command prior to generating the write command.
13 . The computer-implemented method of claim 11 , further comprising:
determining, by the bridge device, the buffer location for the data transfer command, wherein:
the buffer location is selected from:
a bridge device memory; or
a controller memory buffer of the destination data storage device;
the buffer location is not in the at least one host system;
the buffer location is not indicated in the data transfer command; and
converting the data transfer command further comprises:
generating the read command for the target data unit indicating the buffer location for read data; and
generating the write command for the target data unit indicating the buffer location for the read data;
sending, by the bridge device, the read command to the source data storage device; and
sending, by the bridge device, the write command to the destination data storage device.
14 . The computer-implemented method of claim 13 , wherein determining the buffer location comprises at least one of:
determining at least one scatter gather list for the buffer location; and
determining at least one physical region page for the buffer location.
15 . The computer-implemented method of claim 13 , wherein:
receiving the data transfer command uses a first set of command queues in the bridge device configured for host commands; and
sending the read command and the write command uses a second set of command queues in the bridge device configured for data storage device commands.
16 . The computer-implemented method of claim 10 , further comprising:
providing, for the source data storage device, direct memory access to the buffer location in a controller memory buffer in the destination data storage device through a shared interface root complex.
17 . The computer-implemented method of claim 10 , wherein the target data unit is not stored in a bridge device memory between reading the target data unit from the source data storage device and writing the target data unit to the non-volatile storage medium of the destination data storage device.
18 . A bridge device comprising:
a storage interface configured to communicate with a plurality of data storage devices;
a host interface configured to communicate with at least one host system;
means for providing, through the host interface, a target device for a network storage protocol configured to receive encapsulated storage commands from the at least one host system;
means for establishing, through the storage interface, communication with the plurality of data storage devices using a bus storage protocol, wherein the bridge device is configured as an initiator device for the plurality of data storage devices as target devices;
means for receiving, from the at least one host system, a data transfer command identifying a target data unit, a source data storage device, and a destination data storage device, wherein:
the plurality of data storage devices includes the source data storage device and the destination data storage device; and
the data transfer command comprises a fused command that includes a first set of parameters for a modified read command for the source data storage device and a second set of parameters for a modified write command for the destination data storage device; and
means for converting the data transfer command into:
a read command to the source data storage device for the target data unit, wherein the source data storage device is configured to, responsive to the read command:
read the target data unit from a non-volatile storage medium of the source data storage device; and
return the target data unit to a buffer location for the target data unit; and
a write command to the destination data storage device for the target data unit, wherein:
converting the data transfer command comprises determining:
additional parameters added to the first set of parameters for the read command to the source data storage device; and
additional parameters added to the second set of parameters for the write command to the destination data storage device;
the source data storage device is configured to, responsive to the read command:
read the target data unit from a non-volatile storage medium of the source data storage device; and
return the target data unit to a buffer location for the target data unit; and
the destination data storage device is configured to, responsive to the write command:
read the target data unit for the write command from the buffer location; and
write the target data unit to a non-volatile storage medium of the destination data storage device.