System and method for managing small-size files in an aggregated file system
View Patent ↗In an aggregated file system, a method of processing a user file retrieves user file metadata and user data from a metadata server and applies operations to the user data in accordance with a file open request from a client. At the end of the process, the method stores the processed user data at a location in accordance with a predefined rule and updates the metadata in the metadata server to reference the processed user data at the location. In some embodiments, the predefined rule is to choose a location between the metadata server and a separate storage server in accordance with the size of the processed user data. If the size is still smaller than a predetermined threshold, the user data is stored in the metadata server. Otherwise, the user data is stored in the storage server.
1. A method of processing a user file by a file switch in an aggregated file system having at least one storage sever and at least one metadata server including at least one user file, comprising:
retrieving, by the file switch, metadata and user data associated with the user file from a metadata server;
processing, by the file switch, the user data and the metadata associated with the user file in accordance with a request from a client;
storing, by the file switch, the processed metadata associated with the user file in the metadata server;
storing, by the file switch, the processed user data associated with the user file along with the stored processed metadata at a location within the metadata server designated by a predefined rule when a size of the processed user data is smaller than a predefined threshold;
storing, by the file switch, the processed user data at a location within a storage server separate from the metadata server designated by the predefined rule when the size of the processed user data is greater than or equal to the predefined threshold; and
updating, by the file switch, the metadata in the metadata server to reference the user data at the designated location.
2. The method of claim 1 , wherein the retrieving includes retrieving, by the file switch, the metadata and user data in response to a file open request associated with the user file.
3. The method of claim 1 , further comprising, prior to said storing and updating, notifying, by the file switch, the client of a completion of processing the user data in response to the client request.
4. The method of claim 1 , wherein the metadata includes a size of the user data.
5. The method of claim 4 , wherein updating the metadata includes updating, by the file switch, the size of the user data in accordance with the processed user data.
6. The method of claim 1 , wherein the predefined threshold associated with the user file is different from a predefined threshold associated with a different user file.
7. The method of claim 1 , wherein the user data of the user file remains in the separate storage server irrespective of whether the size of the user data is smaller than the predefined threshold or not.
8. The method of claim 1 further comprising retrieving, by the file switch, the processed user data and the metadata in a single transaction in response to a file open request from the client when the processed user data and the metadata are both stored in the metadata server.
9. A file switch for use in a computer network having one or more metadata servers including at least one user file, one or more storage servers, and one or more client computers, the file switch comprising:
at least one interface for exchanging information with the one or more metadata servers, the one or more storage servers and the one or more client computers;
one or more processors; and
a memory coupled to the one or more processors, the one or more processors configured to execute programmed instructions stored in the memory, the programmed instructions comprising:
retrieving metadata and user data associated with a user file from a metadata server;
processing the user data and the metadata associated with the user file in accordance with a request from a client computer;
storing the processed metadata associated with the user file in the metadata server;
storing the processed user data associated with the user file along with the stored processed metadata at a location within the metadata server designated by a predefined rule when a size of the processed user data is smaller than a predefined threshold;
storing the processed user data at a location within a storage server separate from the metadata server designated by the predefined rule when the size of the processed user data is greater than or equal to the predefined threshold; and
updating the metadata in the metadata server to reference the user data at the designated location.
10. The file switch of claim 9 , wherein the retrieving includes retrieving the metadata and user data in response to a file open request associated with the user file.
11. The file switch of claim 9 , further comprising, prior to said storing and updating, notifying the client computer of a completion of processing the user file in response to the request.
12. The file switch of claim 9 , wherein the metadata includes a size of the user data.
13. The file switch of claim 9 , wherein the predefined threshold of the user file is different from a predefined threshold of a different user file.
14. The file switch of claim 9 , wherein the user data of the user file remains in the separate storage server irrespective of whether the size of the user data is smaller than the predefined threshold or not.
15. The file switch of claim 9 further comprising retrieving the processed user data and the metadata in a single transaction in response to a file open request from the client computer when the processed user data and the metadata are both stored in the metadata server.
16. A non-transitory computer readable medium having stored thereon instructions for processing a user file by a file switch in an aggregated file system having at least one storage sever and at least one metadata server including at least one user file, comprising machine executable code which when executed by at least one processor, causes the processor to perform steps comprising:
retrieving metadata and user data associated with the user file from a metadata server;
processing the user data and the metadata associated with the user file in accordance with a request from a client;
storing the processed metadata associated with the user file in the metadata server;
storing the processed user data associated with the user file along with the stored processed metadata at a location within the metadata server designated by a predefined rule when a size of the processed user data is smaller than a predefined threshold;
storing the processed user data at a location within a storage server separate from the metadata server designated by the predefined rule when the size of the processed user data is greater than or equal to the predefined threshold; and
updating the metadata in the metadata server to reference the user data at the designated location.
17. The medium as set forth in claim 16 wherein the retrieving includes retrieving, by the file switch, the metadata and user data in response to a file open request associated with the user file.
18. The medium as set forth in claim 16 further comprising, prior to said storing and updating, notifying, by the file switch, the client of a completion of processing the user data in response to the client request.
19. The medium as set forth in claim 16 wherein the metadata includes a size of the user data.
20. The medium as set forth in claim 19 wherein updating the metadata includes updating, by the file switch, the size of the user data in accordance with the processed user data.
21. The medium as set forth in claim 16 wherein the predefined threshold associated with the user file is different from a predefined threshold associated with a different user file.
22. The medium as set forth in claim 16 wherein the user data of the user file remains in the separate storage server irrespective of whether the size of the user data is smaller than the predefined threshold or not.