File system improvements for zoned storage device operations
The techniques disclosed herein enable systems to efficiently interface with zoned namespace (ZNS) storage devices through a specialized management entity. To achieve this, the management entity receives write requests from a file system containing data intended for storage at the ZNS device. In response, the management entity selects a zone from the ZNS device to write the file data to. Accordingly, the file data is written by appending the file data to the zone at a location indicated by a write pointer. When the write operation is completed, the offset of the file data within the zone is observed and recorded by the file system in file metadata. In contrast to typical systems which allocate locations at the storage device prior to writing, appending file data and then recording the location enables improved efficiency in file system operations. Namely, that write operations can be issued to the ZNS device non-serially.
1 . A method comprising:
receiving, at a management entity, a write request from a file system to a storage device containing a first plurality of zones and a second plurality of zones, the write request comprising file data and a data quantity;
in response to receiving the write request, selecting, by the management entity, a selected zone of the first plurality of zones for storing the file data of the write request;
appending, by the management entity, the file data of the write request to the selected zone;
determining, by the management entity, an offset of the file data within the selected zone based on the data quantity and a starting location of the selected zone;
updating, by the management entity, file metadata associated with the file data with an identifier of the selected zone and the offset of the file data;
receiving, by the management entity, a plurality of additional write requests from the file system;
selecting, by the management entity, the second plurality of zones;
simultaneously transmitting, by the management entity, the plurality of additional write requests to the storage device;
executing, by the management entity, a plurality of write operations comprising appending respective file data of each additional write request of the plurality of additional write requests to a corresponding one of the second plurality of zones; and
receiving, by the management entity, all zone identifiers and file data offsets for the plurality of additional write requests following completion of all the plurality of write operations.
2 . The method of claim 1 , wherein the storage device is a zoned namespace storage device.
3 . The method of claim 1 , wherein the plurality of additional write requests is a first plurality of additional write requests, the method further comprising:
receiving a second plurality of additional write requests;
scheduling an append operation for a respective file data of each additional write request of the second plurality of additional write requests for storage at the selected zone; and
receiving the identifier of the selected zone and a respective offset of the respective file data for each additional write request of the second plurality of additional write requests to the file system.
4 . The method of claim 1 , wherein:
the write request is a write request in a first plurality of write requests generated by a first application; and
the plurality of additional write requests is a second plurality of write requests generated by a second application, the method further comprising:
writing file data from the first plurality of write requests to the first plurality of zones; and
writing the file data from the second plurality of write requests to the second plurality of zones.
5 . The method of claim 1 , wherein file data of write requests originating from a same application are stored within a same zone of the storage device.
6 . The method of claim 1 , wherein the file metadata is stored on a storage device that is different from the storage device containing the plurality of zones.
7 . The method of claim 1 , further comprising:
calculating, by the management entity, a write request priority for each write request quantifying an importance of each write request; and
scheduling, by a scheduling engine, write operations for each zone based on the write request priority, wherein file data of a higher priority write request is scheduled and dispatched to the storage device prior to file data of a lower priority write request.
8 . A system comprising:
a processing system;
a computer-readable medium having encoded thereon, computer-readable instructions that, when executed by the processing system, cause the system to perform operations comprising:
receiving, at a management entity, a write request from a file system to a storage device containing a first plurality of zones and a second plurality of zones, the write request comprising file data and a data quantity;
in response to receiving the write request, selecting, by the management entity, a selected zone of the first plurality of zones for storing the file data of the write request;
appending, by the management entity, the file data of the write request to the selected zone;
determining, by the management entity, an offset of the file data within the selected zone based on the data quantity and a starting location of the selected zone;
updating, by the management entity, file metadata associated with the file data with an identifier of the selected zone and the offset of the file data;
receiving, by the management entity, a plurality of additional write requests from the file system;
selecting, by the management entity, the second plurality of zones;
simultaneously transmitting, by the management entity, the plurality of additional write requests to the storage device;
executing, by the management entity, a plurality of write operations comprising appending respective file data of each additional write request of the plurality of additional write requests to a corresponding one of the second plurality of zones; and
receiving, by the management entity, all zone identifiers and file data offsets for the plurality of additional write requests following completion of all the plurality of write operations.
9 . The system of claim 8 , wherein the storage device is a zoned namespace storage device.
10 . The system of claim 8 , wherein the plurality of additional write requests is a first plurality of additional write requests, the operations further comprising:
receiving a second plurality of additional write requests;
scheduling an append operation for a respective file data of each additional write request of the second plurality of additional write requests for storage at the selected zone; and
receiving the identifier of the selected zone and a respective offset of the respective file data for each additional write request of the second plurality of additional write requests to the file system.
11 . The system of claim 8 , wherein:
the write request is a write request in a first plurality of write requests generated by a first application; and
the plurality of additional write requests is a second plurality of write requests generated by a second application, the operations further comprising:
writing file data from the first plurality of write requests to the first plurality of zones; and
writing the file data from the second plurality of write requests to the second plurality of zones.
12 . The system of claim 8 , wherein file data of write requests originating from a same application are stored within a same zone of the storage device.
13 . The system of claim 8 , wherein the file metadata is stored on a storage device that is different from the storage device containing the plurality of zones.
14 . The system of claim 8 , the operations further comprising:
calculating, by the management entity, a write request priority for each write request quantifying an importance of each write request; and
scheduling, by a scheduling engine, write operations for each zone based on the write request priority, wherein file data of a higher priority write request is scheduled and dispatched to the storage device prior to file data of a lower priority write request.
15 . A computer-readable storage medium having encoded thereon, computer- readable instructions that, when executed by a system, cause the system to perform operations comprising:
receiving, at a management entity, a write request from a file system to a storage device containing a first plurality of zones and a second plurality of zones, the write request comprising file data and a data quantity;
in response to receiving the write request, selecting, by the management entity, a selected zone of the first plurality of zones for storing the file data of the write request;
appending, by the management entity, the file data of the write request to the selected zone;
determining, by the management entity, an offset of the file data within the selected zone based on the data quantity and a starting location of the selected zone;
updating, by the management entity, file metadata associated with the file data with an identifier of the selected zone and the offset of the file data;
receiving, by the management entity, a plurality of additional write requests from the file system;
selecting, by the management entity, the second plurality of zones;
simultaneously transmitting, by the management entity, the plurality of additional write requests to the storage device;
executing, by the management entity, a plurality of write operations comprising appending respective file data of each additional write request of the plurality of additional write requests to a corresponding one of the second plurality of zones; and
receiving, by the management entity, all zone identifiers and file data offsets for the plurality of additional write requests following completion of all the plurality of write operations.
16 . The computer-readable storage medium of claim 15 , wherein the plurality of additional write requests is a first plurality of additional write requests, the operations further comprising:
receiving a second plurality of additional write requests;
scheduling an append operation for a respective file data of each additional write request of the second plurality of additional write requests for storage at the selected zone; and
receiving the identifier of the selected zone and a respective offset of the respective file data for each additional write request of the second plurality of additional write requests to the file system.
17 . The computer-readable storage medium of claim 15 , wherein:
the write request is a write request in a first plurality of write requests generated by a first application; and
the plurality of additional write requests is a second plurality of write requests generated by a second application, the operations further comprising:
writing file data from the first plurality of write requests to the first plurality of zones; and
writing the file data from the second plurality of write requests to the second plurality of zones.
18 . The computer-readable storage medium of claim 15 , wherein file data of write requests originating from a same application are stored within a same zone of the storage device.
19 . The computer-readable storage medium of claim 15 , wherein the file metadata is stored on a storage device that is different from the storage device containing the plurality of zones.
20 . The computer-readable storage medium of claim 15 , the operations further comprising:
calculating, by the management entity, a write request priority for each write request quantifying an importance of each write request; and
scheduling, by a scheduling engine, write operations for each zone based on the write request priority, wherein file data of a higher priority write request is scheduled and dispatched to the storage device prior to file data of a lower priority write request.