Host multi-path layer with proxy volume redirection for enhanced scalability of distributed storage systems
An apparatus in an illustrative embodiment comprises at least one processing device that includes a processor coupled to a memory. The at least one processing device is configured to associate a logical storage volume with at least one path between a host and at least a given one of a plurality of storage nodes of a distributed storage system, and to associate a proxy volume with respective sets of one or more paths from the host to each of the plurality of storage nodes of the distributed storage system. The at least one processing device is further configured to obtain an input-output operation directed to the logical storage volume, and to redirect the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume. Such an arrangement can avoid one or more host-side restrictions, thereby enhancing scalability of the distributed storage system.
1. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to associate a logical storage volume with at least one path between a host and at least a given one of a plurality of storage nodes of a distributed storage system, each of the storage nodes storing corresponding portions of the logical storage volume and one or more additional logical storage volumes, a given such logical storage volume being stored in a distributed manner over the storage nodes with different address ranges of the given logical storage volume being stored in different ones of the storage nodes;
to associate a proxy volume with respective sets of one or more paths from the host to each of the plurality of storage nodes of the distributed storage system;
to obtain an input-output operation directed to the logical storage volume; and
to redirect the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume;
wherein the host is configured to expose a first number of paths to at least one application executing on the host, the first number of paths being determined collectively across all of the logical storage volumes accessible to the at least one application; and
wherein associating the proxy volume with the respective sets of one or more paths from the host to each of the plurality of storage nodes provides the host with access to a second number of paths that is greater than the first number of paths.
2. The apparatus of claim 1 wherein the at least one processing device comprises at least a portion of the host.
3. The apparatus of claim 1 wherein the host comprises a plurality of initiators and supports one or more paths between each of the initiators and one or more targets on each of the storage nodes.
4. The apparatus of claim 1 wherein the distributed storage system comprises a software-defined storage system and the storage nodes comprise respective software-defined storage server nodes of the software-defined storage system.
5. The apparatus of claim 1 wherein the host comprises a multi-path layer that includes at least one multi-path input-output driver configured to perform at least portions of the associating the logical storage volume, the associating the proxy volume, the obtaining the input-output operation and the redirecting the input-output operation.
6. The apparatus of claim 1 wherein the logical storage volume is associated with at least first and second paths between the host and the given storage node of the plurality of storage nodes.
7. The apparatus of claim 1 wherein redirecting the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume comprises:
determining a particular portion of the logical storage volume targeted by the input-output operation;
accessing at least one data structure to identify a particular one of the plurality of storage nodes that stores data of the particular portion of the logical storage volume; and
selecting a particular path from the set of one or more paths from the host to the identified storage node.
8. The apparatus of claim 7 wherein the at least one data structure comprises a table.
9. The apparatus of claim 7 wherein the at least one data structure stores mapping information that maps portions of the logical storage volume to respective ones of the storage nodes that store data of those portions.
10. The apparatus of claim 9 wherein the mapping information is obtained at least in part from one or more of the storage nodes.
11. The apparatus of claim 9 wherein the mapping information is periodically updated to reflect configuration changes relating to the distributed storage system.
12. The apparatus of claim 1 wherein the logical storage volume is associated with no more than a maximum number of paths per volume for the logical storage volume and the proxy volume is associated with a number of paths that is larger than the maximum number of paths per volume for the logical storage volume.
13. The apparatus of claim 12 wherein the maximum number of paths per volume for the logical storage volume is two paths.
14. The apparatus of claim 1 wherein the proxy volume is associated with a number of paths that is a function of the number of storage nodes in the plurality of storage nodes.
15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device comprising a processor coupled to a memory, causes the at least one processing device:
to associate a logical storage volume with at least one path between a host and at least a given one of a plurality of storage nodes of a distributed storage system, each of the storage nodes storing corresponding portions of the logical storage volume and one or more additional logical storage volumes, a given such logical storage volume being stored in a distributed manner over the storage nodes with different address ranges of the given logical storage volume being stored in different ones of the storage nodes;
to associate a proxy volume with respective sets of one or more paths from the host to each of the plurality of storage nodes of the distributed storage system;
to obtain an input-output operation directed to the logical storage volume; and
to redirect the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume;
wherein the host is configured to expose a first number of paths to at least one application executing on the host, the first number of paths being determined collectively across all of the logical storage volumes accessible to the at least one application; and
wherein associating the proxy volume with the respective sets of one or more paths from the host to each of the plurality of storage nodes provides the host with access to a second number of paths that is greater than the first number of paths.
16. The computer program product of claim 15 wherein the host comprises a multi-path layer that includes at least one multi-path input-output driver configured to perform at least portions of the associating the logical storage volume, the associating the proxy volume, the obtaining the input-output operation and the redirecting the input-output operation.
17. The computer program product of claim 15 wherein redirecting the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume comprises:
determining a particular portion of the logical storage volume targeted by the input-output operation;
accessing at least one data structure to identify a particular one of the plurality of storage nodes that stores data of the particular portion of the logical storage volume; and
selecting a particular path from the set of one or more paths from the host to the identified storage node.
18. A method comprising:
associating a logical storage volume with at least one path between a host and at least a given one of a plurality of storage nodes of a distributed storage system, each of the storage nodes storing corresponding portions of the logical storage volume and one or more additional logical storage volumes, a given such logical storage volume being stored in a distributed manner over the storage nodes with different address ranges of the given logical storage volume being stored in different ones of the storage nodes;
associating a proxy volume with respective sets of one or more paths from the host to each of the plurality of storage nodes of the distributed storage system;
obtaining an input-output operation directed to the logical storage volume; and
redirecting the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume;
wherein the host is configured to expose a first number of paths to at least one application executing on the host, the first number of paths being determined collectively across all of the logical storage volumes accessible to the at least one application;
wherein associating the proxy volume with the respective sets of one or more paths from the host to each of the plurality of storage nodes provides the host with access to a second number of paths that is greater than the first number of paths; and
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
19. The method of claim 18 wherein the host comprises a multi-path layer that includes at least one multi-path input-output driver configured to perform at least portions of the associating the logical storage volume, the associating the proxy volume, the obtaining the input-output operation and the redirecting the input-output operation.
20. The method of claim 18 wherein redirecting the input-output operation to the proxy volume over a selected one of the paths associated with the proxy volume comprises:
determining a particular portion of the logical storage volume targeted by the input-output operation;
accessing at least one data structure to identify a particular one of the plurality of storage nodes that stores data of the particular portion of the logical storage volume; and
selecting a particular path from the set of one or more paths from the host to the identified storage node.
21. The apparatus of claim 1 wherein the host is subject to a host-side restriction specifying a maximum number of paths that can be exposed to the at least one application executing on the host, the maximum number of paths being determined collectively across all of the logical storage volumes accessible to the at least one application, and wherein associating the proxy volume with the respective sets of one or more paths from the host to each of the plurality of storage nodes provides the host with access to a total number of paths that is greater than the specified maximum number of paths.