Simulation of asynchronous modifications of shared data objects by a distributed application
View Patent ↗Methods and apparatus are provided for simulating asynchronous modifications of shared data objects by a number of distributed processes within a distributed application. An exemplary method comprises obtaining a plurality of simulation parameters that define parameters of a simulated distributed application, parameters of a plurality of distributed compute nodes on which the simulated distributed application executes and a storage system employed by the simulated distributed application; and applying the plurality of simulation parameters to a simulation engine, wherein the simulation engine simulates both synchronous and asynchronous modifications of at least one shared data object by a plurality of distributed processes within the simulated distributed application. One or more exemplary techniques evaluate a degree to which an asynchronous shared data modification model can improve overall performance of a distributed application or reduce the required capabilities of the storage system needed to support the distributed application, relative to the same distributed application using a synchronous shared data modification model.
1. A method, comprising:
obtaining a plurality of simulation parameters that define parameters of a simulated distributed application, parameters of a plurality of distributed compute nodes on which the simulated distributed application executes and a storage system employed by the simulated distributed application;
applying, using at least one processing device, said plurality of simulation parameters to a simulation engine, wherein said simulation engine simulates both synchronous and asynchronous modifications of at least one shared data object by a plurality of distributed processes within said simulated distributed application, wherein said asynchronous modifications of said at least one shared data object by said plurality of distributed processes employ a transaction-based approach where said asynchronous modifications of said at least one shared data object are committed after said plurality of distributed processes have made said asynchronous modifications based on a reference counting of the plurality of distributed processes; and
evaluating whether said simulated distributed application will exhibit a performance improvement using said asynchronous modifications by said plurality of distributed processes, relative to said synchronous modifications by said plurality of distributed processes, based on one or more outputs from said simulation engine, wherein said performance improvement comprises one or more of a decreased execution time, a decreased requirement for compute resources, and a decreased requirement for storage resources.
2. The method of claim 1 , wherein said simulation engine selectively enables one or more barrier operations associated with synchronous modifications of said at least one shared data object by said plurality of distributed processes.
3. The method of claim 1 , wherein said simulation engine selectively disables one or more barrier operations associated with said asynchronous modifications of said at least one shared data object by said plurality of distributed processes.
4. The method of claim 1 , further comprising the step of assessing a throughput performance of said simulated distributed application in an asynchronous modification mode that performs said asynchronous modifications of said at least one shared data object.
5. The method of claim 1 , further comprising the step of varying a compute variation parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
6. The method of claim 1 , further comprising the step of varying a communication size parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
7. The method of claim 1 , further comprising the step of varying a hotspot movement parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
8. The method of claim 1 , further comprising the step of varying a runtime parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
9. The method of claim 1 , further comprising the step of varying a job size parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
10. The method of claim 1 , further comprising the step of evaluating required capabilities of the storage system in said asynchronous modification mode to achieve a substantially same performance as said simulated distributed process in a synchronous modification mode.
11. A system, comprising:
a memory; and
at least one processing device configured to:
obtain a plurality of simulation parameters that define parameters of a simulated distributed application, parameters of a plurality of distributed compute nodes on which the simulated distributed application executes and a storage system employed by the simulated distributed application;
apply, using at least one processing device, said plurality of simulation parameters to a simulation engine, wherein said simulation engine simulates both synchronous and asynchronous modifications of at least one shared data object by a plurality of distributed processes within said simulated distributed application, wherein said asynchronous modifications of said at least one shared data object by said plurality of distributed processes employ a transaction-based approach where said asynchronous modifications of said at least one shared data object are committed after said plurality of distributed processes have made said asynchronous modifications based on a reference counting of the plurality of distributed processes; and
evaluate whether said simulated distributed application will exhibit a performance improvement using said asynchronous modifications by said plurality of distributed processes, relative to said synchronous modifications by said plurality of distributed processes, based on one or more outputs from said simulation engine, wherein said performance improvement comprises one or more of a decreased execution time, a decreased requirement for compute resources, and a decreased requirement for storage resources.
12. The system of claim 11 , wherein said simulation engine selectively enables one or more barrier operations associated with synchronous modifications of said at least one shared data object by said plurality of distributed processes.
13. The system of claim 11 , wherein said simulation engine selectively disables one or more barrier operations associated with said asynchronous modifications of said at least one shared data object by said plurality of distributed processes.
14. The system of claim 11 , wherein said at least one processing device is further configured to assess a throughput performance of said simulated distributed application in an asynchronous modification mode that performs said asynchronous modifications of said at least one shared data object.
15. The system of claim 11 , further comprising the step of varying a parameter value of the simulated distributed application for a plurality of successive distributed state capture operations by the simulated distributed application.
16. The system of claim 11 , wherein said at least one processing device is further configured to evaluate required capabilities of the storage system in said asynchronous modification mode to achieve a substantially same performance as said simulated distributed process in a synchronous modification mode.
17. A non-transitory machine-readable recordable storage medium, wherein one or more software programs when executed by one or more processing devices implement the following steps:
obtaining a plurality of simulation parameters that define parameters of a simulated distributed application, parameters of a plurality of distributed compute nodes on which the simulated distributed application executes and a storage system employed by the simulated distributed application;
applying, using at least one processing device, said plurality of simulation parameters to a simulation engine, wherein said simulation engine simulates both synchronous and asynchronous modifications of at least one shared data object by a plurality of distributed processes within said simulated distributed application, wherein said asynchronous modifications of said at least one shared data object by said plurality of distributed processes employ a transaction-based approach where said asynchronous modifications of said at least one shared data object are committed after said plurality of distributed processes have made said asynchronous modifications based on a reference counting of the plurality of distributed processes; and
evaluating whether said simulated distributed application will exhibit a performance improvement using said asynchronous modifications by said plurality of distributed processes, relative to said synchronous modifications by said plurality of distributed processes, based on one or more outputs from said simulation engine, wherein said performance improvement comprises one or more of a decreased execution time, a decreased requirement for compute resources, and a decreased requirement for storage resources.
18. The storage medium of claim 17 , wherein said simulation engine selectively enables one or more barrier operations associated with synchronous modifications of said at least one shared data object by said plurality of distributed processes.
19. The storage medium of claim 17 , wherein said simulation engine selectively disables one or more barrier operations associated with said asynchronous modifications of said at least one shared data object by said plurality of distributed processes.
20. The storage medium of claim 17 , further comprising the step of assessing a throughput performance of said simulated distributed application in an asynchronous modification mode that performs said asynchronous modifications of said at least one shared data object.