Method and apparatus for coordinating service execution within a shared file system environment to optimize cluster performance
View Patent ↗A method and apparatus for coordinating service execution within a shared storage cluster file system environment to optimize cluster performance is disclosed. In one embodiment, the method includes accessing host information in memory comprising mappings between file data and a plurality of nodes, wherein the mappings indicate at least one portion of the file data that is accessed by at least one node of the plurality of nodes on behalf of at least one application and executing at least one service on the at least one node of the plurality of nodes using at least one processor, wherein the at least one service accesses the at least one portion of the file data in the memory.
1. A method comprising: identifying file data requested by a service; determining, in response to the file data being requested by the service, which node of a plurality of nodes is providing an application with access to the file data, wherein the determining comprises accessing memory, the memory comprises a mapping generated by a plurality of file drivers between one or more files within the file data and a node of a plurality of nodes, the plurality of nodes is configured to provide shared access to the file data, the mapping identifies the one or more files within the file data that are accessed via the node of the plurality of nodes on behalf of the application, the file data is stored by a shared storage device, and the shared storage device is configured to be accessed by the plurality of nodes; selecting the node from the plurality of nodes on which the service is to be executed, wherein at least two of the plurality of nodes are configured to execute the service, the service is configured to access the one or more files within the file data using the node, and the selecting is based both on the mapping identifying that the one or more files within the file data is are being accessed by via the node on behalf of the application, and the service being configured to access the one or more files within the file data using the node; and executing the service on the node using at least one processor, wherein the service accesses the one or more files within the file data in the shared storage device.
2. The method of claim 1 , wherein the executing comprises adjusting a schedule to execute the service on the node, the schedule comprises a plurality of services to be executed on the plurality of nodes, and the adjusting the schedule comprises re-ordering the schedule such that the service is executed on the node while the mapping generated by the plurality of file drivers indicates that the one or more files within the file data is are being accessed by the node on behalf of the application.
3. The method of claim 1 , wherein
the application is executed on a client,
the client is coupled to the node, and
the application is configured to access the one or more files within the file data through the node using one or more file system operations.
4. The method of claim 1 , wherein the plurality of file drivers that generate the mapping by detect which one node out of the plurality of nodes has exclusive access to the one or more files within the file data.
5. The method of claim 3 , wherein
the accessing further comprises processing a file change log to identify the node, and
the file change log indicates which application performs the one or more file system operations.
6. The method of claim 1 , wherein the accessing further comprises processing a plurality of file system operations using the plurality of file drivers to generate a plurality of mappings, the shared storage device comprises a plurality of files including the one or more files within the file data, each of the mappings identifies which node out of the plurality of nodes accesses each of the plurality of files, and the plurality of mappings comprise the mapping.
7. The method of claim 1 , wherein
the accessing is performed in response to the service being scheduled for execution on the node.
8. A non-transitory computer-readable storage medium storing program instructions executable to: identify file data requested by a service; determine, in response to the file data being requested by the service, which node of a plurality of nodes is providing an application with access to the file data, wherein determining which node comprises accessing memory storing a mapping generated by a plurality of file drivers between one or more files within the file data and a node of a plurality of nodes using at least one processor, the plurality of nodes is configured to provide shared access to the file data, the mapping identifies the one or more files within the file data via the node of the plurality of nodes on behalf of the application, the file data is stored by a shared storage device, and the shared storage device is operable to be accessed by the plurality of nodes; select the node from the plurality of nodes on which the service is to be executed, wherein the service is configured to access the file data using the node, and the node is selected based both on the mapping identifying that the one or more files within the file data is are being accessed by via the node on behalf of the application, and the service being configured to access the one or more files within the file data using the node; and execute the service on the node using at east one processor, wherein the service, when executed, accesses the one or more files within the file data in the shared storage device.
9. The non-transitory computer-readable storage medium of claim 8 , further comprising program instructions executable to:
adjust a schedule to execute the service on the node of the plurality of nodes, wherein
the schedule comprises a plurality of services to be executed on the plurality of nodes, and
the adjustment of the schedule comprises re-ordering the schedule such that the service is executed on the node while the mapping indicates that the one or more files within the file data are being accessed by the node on behalf of the application.
10. The non-transitory computer-readable storage medium of claim 9 , wherein
the application is executed on a client,
the client is coupled to the node, and
the application is configured to access the one or more files within the file data through the node using one or more file system operations.
11. The non-transitory computer-readable storage medium of claim 8 , wherein the plurality of file drivers that generate the mapping by detect which one node out of the plurality of nodes has exclusive access to the one or more files within the file data.
12. The non-transitory computer-readable storage medium of claim 8 , further comprising program instructions executable to:
process a file change log to identify the node, wherein
the file change log indicates which application performs the one or more file system operations on the one or more files within the file data.
13. The non-transitory computer-readable storage medium of claim 8 further comprising program instructions executable to:
process a plurality of file system operations to generate a plurality of mappings using the plurality of drivers, wherein
the shared storage device comprises a plurality of files including the one or more files within the file data,
each of the mappings identifies which node out of the plurality of nodes accesses each of the plurality of files, and
the plurality of mappings comprise the mapping.
14. The non-transitory computer-readable storage medium of claim 9 , wherein
the memory is accessed in response to scheduling the service to be executed.
15. A system comprising: a plurality of nodes; and a memory coupled to the plurality of nodes, wherein the memory stores program instructions executable to: identify file data requested by a service, determine, in response to the file data being requested by the service, which node of a plurality of nodes is providing an application with access to the file data, wherein determining which node comprises accessing a mapping generated by a plurality of file drivers between one or more files within the file data and a node of the plurality of nodes, the plurality of nodes is configured to provide shared access to the file data, the mapping identifies the one or more files within the file data via the node on behalf of the application, the file data is stored by a shared storage device, and the shared storage device is operable to be accessed by the plurality of nodes, select the node from the plurality of nodes on which the service is to be executed, wherein at least two of the plurality of nodes are capable of executing the service, the service is configured to access the file data using the node, and the node is selected based both on the mapping identifying that the one or more files within the file data is are being accessed by via the node on behalf of the application, and the service being configured to access the file data using the node, and execute the service on the node using at least one processor, wherein the service, when executed, accesses the one or more files within the file data in the shared storage device.
16. The system of claim 15 , further comprising program instructions executable to:
adjust a schedule to execute the service on the node of the plurality of nodes, wherein
the schedule comprises a plurality of services to be executed on the plurality of nodes, and
the adjustment of the schedule comprises re-ordering the schedule such that the service is executed on the node while the mapping indicates that the one or more files within the file data are being accessed by the node on behalf of the application.
17. The system of claim 16 , wherein
the application is executed on a client,
the client is coupled to the node, and
the application is configured to access the one or more files within the file data through the node using one or more file system operations.
18. The system of claim 15 wherein the plurality of file drivers that generate the mapping by detect which one node out of the plurality of nodes has exclusive access to the one or more files within the file data.
19. The system of claim 15 , further comprising program instructions executable to:
process a file change log to identify the node, wherein
the file change log indicates which application performs the one or more file system operations on the one or more files within the file data.
20. The system of claim 15 further comprising program instructions executable to: process a plurality of file system operations, and generate a plurality of mappings using the plurality of file drivers, wherein the shared storage device comprises a plurality of files including the one or more files within the file data, each of the mappings identifies which node out of the plurality of nodes accesses each of the plurality of files, and the plurality of mappings comprise the mapping.
21. The method of claim 5 , wherein a file driver of the plurality of the drivers utilizes the one or more file system operations to identify the one or more files within the file data, and identifies at least one node of the plurality of nodes that performs the one or more file system operations on behalf of the application.
22. The method of claim 21 , wherein the driver of the plurality of the drivers invokes one or more operating system function calls to generate the file change log.
23. The method of claim 2 , wherein
the plurality of services are scheduled by a coordinator node, and
the coordinator node operates independently of the plurality of nodes.