Tiered storage in a distributed file system
A file server receives a request for data from a user device. The data is represented at the file server by a virtual cluster descriptor. The file server queries an identifier map using an identifier of the virtual cluster descriptor. Responsive to the identifier map indicating that the requested data is stored at a location remote from the file server, the file server accesses a cold tier translation table that stores a mapping between an identifier of each of a plurality of virtual cluster descriptors and a storage location of data associated with the respective virtual cluster descriptor. The cold tier translation table is queried using the identifier of the virtual cluster descriptor to identify a storage location of the requested data, and the data is loaded to the file server from the identified storage location.
1. A method comprising:
receiving at a file server, a request from a user device for data represented by a virtual cluster descriptor, the file server including a tiered file storage system;
querying an identifier map using an identifier of the virtual cluster descriptor; and
responsive to the identifier map indicating that the requested data is stored at a location remote from the file server:
sending a notification to the user device to cause the user device to re-request the requested data from the file server at a later time;
accessing a cold tier translation table that stores a mapping between an identifier of each of a plurality of virtual cluster descriptors and a storage location of data associated with the respective virtual cluster descriptor, wherein tiered file storage system includes a cold tier storage for data that is selected for offloading;
querying the cold tier translation table using the identifier of the virtual cluster descriptor associated with the requested data to identify a storage location of the requested data in the cold tier storage; and
loading the requested data to the file server from the identified storage location.
2. The method of claim 1 , wherein:
the notification is to cause the user device to resend the request for the requested data after a delay of a specified interval of time.
3. The method of claim 1 , further comprising:
identifying a set of data stored at the file server that is to be offloaded from the file server to new locations remote from the file server in the cold tier storage, the identified set of data associated with a second virtual cluster desriptor; and
updating the cold tier translation table to map an identifier of the second virtual cluster descriptor to the new locations remote from the file server.
4. The method of claim 1 , wherein the identifier map stores a mapping between an identifier of a virtual cluster descriptor and a physical storage location at the file server if data corresponding to the virtual cluster descriptor is stored at the file server, and wherein the identifier map stores a mapping between the identifier of the virtual cluster descriptor and an empty location if the data corresponding to the virtual cluster descriptor is stored remotely from the file server.
5. The method of claim 1 , wherein the notification is to cause the user device to re-request the requested data one or more times while the file server fetches the requested data asynchronously from the cold tier storage.
6. The method of claim 1 , further comprising:
after loading the requested data to the file server from the identified storage location of the cold tier storage:
receiving, at the file server, another request from the user device for the requested data represented by the virtual cluster descriptor; and
returning the requested data from the file server to the user device in response to the another request.
7. The method of claim 1 , further comprising:
responsive to the identifier map indicating that the requested data is stored locally at the file server, retrieving the requested data from the file server and providing the requested data to the user device.
8. A method comprising:
receiving, at a file server including a tiered file storage system, a request from a client for data stored at a cold tier storage location remote from the file server, wherein the cold tier storage includes data that is selected for offloading;
in response to the request, sending a notification to the client to cause the client to re-request the requested data from the file server at a later time;
accessing a cold tier translation table that stores a mapping between an identifier of each of a plurality of virtual cluster descriptors and a storage location of data associated with the respective virtual cluster descriptor;
querying the cold tier translation table using an identifier of a virtual cluster descriptor associated with the requested data to identify a storage location of the requested data in the cold tier storage; and
loading the requested data to the file server from the identified storage location.
9. The method of claim 8 , further comprising:
storing at the file server, an identifier map that stores a mapping between an identifier of a virtual cluster descriptor and a physical storage location at the file server if data corresponding to the virtual cluster descriptor is stored at the file server, and that stores a mapping between the identifier of the virtual cluster descriptor and an empty location if the data corresponding to the virtual cluster descriptor is stored remotely from the file server.
10. The method of claim 9 , further comprising:
querying the identifier map using the identifier of the virtual cluster descriptor associated with the requested data; and
querying the cold tier translation table responsive to the identifier map indicating that the requested data is stored at a location remote from the file server.
11. The method of claim 8 , wherein:
the notification is to cause the client to resend the request for the requested data after a delay of a specified interval of time.
12. The method of claim 11 , wherein the notification is to cause the client to resend the request for the requested data a preset number of times.
13. The method of claim 8 , further comprising:
identifying a set of data stored at the file server that is to be offloaded from the file server to new locations remote from the file server, the identified set of data associated with a second virtual cluster descriptor; and
updating the cold tier translation table to map an identifier of the second virtual cluster descriptor to the new locations remote from the file server.
14. The method of claim 8 , further comprising:
after loading the requested data to the file server from the identified storage location of the cold tier storage:
receiving, at the file server, another request from the client for the requested data; and
returning the requested data from the file server to the client in response to the another request.
15. A system comprising:
a processor;
at least one machine-readable storage medium comprising instructions executable by the processor to store translation tables that map identifiers of each of a plurality of virtual cluster descriptors to a physical storage location of data corresponding to each virtual cluster descriptor; and
a file server including a tiered file storage system that includes a cold tier storage for data that is selected for offloading, wherein the file server is to:
store an identifier map;
in response to a request from a client for data represented by a virtual cluster descriptor of the plurality of virtual cluster descriptors, query the identifier map using an identifier of the virtual cluster descriptor associated with the requested data;
responsive to the identifier map indicating that the requested data is stored locally at the file server, retrieve the requested data from the file server; and
responsive to the identifier map indicating that the requested data is stored at a location remote from the file server:
send a notification to the client to cause the client to re-request the requested data from the file server at a later time;
query the cold tier translator using an identifier of a virtual cluster descriptor associated with requested data to identify a storage location of the requested data in the cold tier storage; and
load the requested data to the file server from the identified storage location.
16. The system of claim 15 , wherein the instructions are executable by the processor to:
identify a set of data stored at the file server that is to be offloaded from the file server to new locations remote from the file server, the identified set of data associated with a second virtual cluster descriptor; and
update the cold tier translation table to map an identifier of the second virtual cluster descriptor to the new locations remote from the file server.
17. The system of claim 15 , wherein:
the notification is to cause the user device to resend the data request after a delay of a specified interval of time.
18. The system of claim 17 , wherein the notification is to cause the user device to resend the request for the requested data a preset number of times.
19. The system of claim 15 , wherein the file server is further configured to:
responsive to the identifier map indicating that the requested data is stored locally at the file server, retrieve the requested data from the file server and provide the requested data to the user device.
20. The system of claim 15 , wherein the file server is to:
after loading the requested data to the file server from the identified storage location of the cold tier storage:
in response to receiving another request from the client for the requested data, return the requested data from the file server to the client in response to the another request.