Distributed file system with reduced write and read latencies
A method for reducing write latency in a distributed file system. A write request that includes a volume identifier is received at a data management subsystem deployed on a node within a distributed storage system. The data management subsystem maps the volume identifier to a file system volume and maps the file system volume to a set of logical block addresses in a logical block device in a storage management subsystem deployed on the node. The storage management subsystem maps the logical block device to a metadata object for the logical block device on the node that is used to process the write request. The mapping of the file system volume to the set of logical block addresses in the logical block device enables co-locating the metadata object with the file system volume on the node, which reduces the write latency associated with processing the write request.
1 . A method for reducing write latency in a distributed file system, the method comprising:
for a given node of a plurality of nodes of a distributed storage system:
reducing write latency associated with processing of write requests by co-locating, by the distributed storage system, a metadata object with a file system volume on the given node, wherein a first mapping, maintained by a data management subsystem of the given node, of the file system volume to a set of logical block addresses in a logical block device of a storage management subsystem of the given node enables the co-locating; and
maintaining, by the storage management subsystem of the given node, a second mapping of the logical block device to the metadata object on the given node;
receiving a write request at the data management subsystem;
identifying, by the data management subsystem, the file system volume on the given node to which the write request is directed based on a volume identifier associated with the write request; and
servicing the write request by utilizing, by the data management subsystem and the storage management subsystem the first mapping and the second mapping.
2 . The method of claim 1 , wherein co-location of the file system volume, the logical block device, and the metadata object on the given node reduces the write latency associated with the write request by enabling local metadata updates.
3 . The method of claim 2 , further comprising sending, by the data management subsystem, a request that identifies a set of logical block addresses to which data from the write request is to be written in the logical block device over a local pathway to the storage management subsystem.
4 . The method of claim 1 , further comprising computing, by a metadata service in the storage management subsystem, a set of block identifiers based on a request received from the data management subsystem, the request identifying the set of logical block addresses to which data from the write request is to be written in the logical block device.
5 . The method of claim 1 , further comprising sending a response to a source of the write request after metadata and data associated with the write request are stored in a logging storage device associated with the given node.
6 . A node of a distributed storage system comprising:
a memory; and
one or more processors coupled to the memory programmed with executable instructions, the instructions including:
a storage management subsystem for maintaining a first mapping of a logical block device to a metadata object and contributing to processing of write requests received by the node based on the first mapping; and
a data management subsystem for maintaining a second mapping of a file system volume to a set of logical block addresses in the logical block device and for contributing to processing of the write requests based on the second mapping, wherein the second mapping enables co-locating of the metadata object with the file system volume on the node, and wherein co-location of the metadata object with the file system volume on the node reduces write latency associated with the processing of the write requests.
7 . The node of claim 6 , wherein the file system volume is mapped to the set of logical block addresses in the logical block device by mapping the file system volume to a logical aggregate and mapping the logical aggregate to the set of logical block addresses in the logical block device.
8 . The node of claim 6 , wherein the file system volume, the logical block device, and the metadata object being co-located on the node reduces the write latency associated with the write request by enabling local metadata updates.
9 . The node of claim 6 , wherein the data management subsystem is configured to send a request that identifies a set of logical block addresses to which data associated with a given write request is to be written in the logical block device over a local pathway to the storage management subsystem.
10 . The node of claim 6 , wherein the storage management subsystem includes a metadata service configured to compute a set of block identifiers based on a request received from the data management subsystem, the request identifying the set of logical block addresses to which data associated with a given write request is to be written in the logical block device received from the data management subsystem.
11 . The node of claim 6 , wherein the storage management subsystem includes a metadata service configured to store a cryptographic hash for data that is to be written to a selected logical block address in the logical block device based on a given write request in a key-value store.
12 . The node of claim 6 , wherein the storage management subsystem includes a metadata service configured to overwrite a previously computed cryptographic hash associated with a selected block address in the logical block device with a new cryptographic hash.
13 . A non-transitory machine readable medium storing instructions, which when executed by one or more central processing unit resources of a distributed storage system, cause the distributed storage system to:
for a given node of a plurality of nodes of the distributed storage system:
reduce write latency associated with processing of write requests by co-locating a metadata object with a file system volume on the given node, wherein a first mapping, maintained by a data management subsystem of the given node, of the file system volume to a set of logical block addresses in a logical block device of a storage management subsystem of the given node enables the co-locating; and
maintain within the storage management subsystem of the given node a second mapping of the logical block device to the metadata object on the given node;
receive a write request at the data management subsystem;
identify by the data management subsystem, the file system volume on the given node to which the write request is directed based on a volume identifier associated with the write request; and
service the write request by utilizing, by the data management subsystem and the storage management subsystem the first mapping and the second mapping.
14 . The non-transitory machine readable medium of claim 13 , wherein co-location of the file system volume, the logical block device, and the metadata object on the given node reduces the write latency associated with the write request by enabling local metadata updates.
15 . The non-transitory machine readable medium of claim 13 , wherein the instructions further cause the distributed storage system to send, by the data management subsystem, a request that identifies a set of logical block addresses to which data from the write request is to be written in the logical block device over a local pathway to the storage management subsystem.
16 . The non-transitory machine readable medium of claim 13 , wherein the instructions further cause the distributed storage system to compute, by a metadata service in the storage management subsystem, a set of block identifiers based on a request received from the data management subsystem, the request identifying the set of logical block addresses to which data from the write request is to be written in the logical block device.
17 . The non-transitory machine readable medium of claim 13 , wherein the instructions further cause the distributed storage system to:
compute a cryptographic hash for a data block formed by file system data and file system metadata that is to be written in the logical block device based on the write request; and
store the cryptographic hash as a key within a key-value store and the data block as a value within the key-value store.
18 . The non-transitory machine readable medium of claim 13 , wherein servicing of the write request includes overwriting a logical block address in the logical block device and overwriting a previously computed cryptographic hash associated with a selected block address in the logical block device with a new cryptographic hash.
19 . The non-transitory machine readable medium of claim 13 , wherein the instructions further cause the distributed storage system to send a response to a source of the write request after metadata and data associated with the write request are stored in a logging storage device associated with the given node.
20 . The non-transitory machine readable medium of claim 13 , wherein the instructions further cause the distributed storage system to:
compute a block identifier for a data block corresponding to write data and write metadata associated with the write request; and
store the block identifier in a set of indirect nodes in a buffer tree within the data management subsystem.