Dynamic recycling algorithm to handle overlapping writes during synchronous replication of application workloads with large number of files
Techniques are provided for overlapping write handling. Overlapping write managers are used to maintain the order that write operations are executed at a first computing environment and replicated to a second computing environment. Overlapping write managers are pre-allocated as available for managing overlapping write operations. A mapping is used to track what overlapping write managers are currently allocated for particular file handles of files. Thus, if an incoming write operation targets a file handle of an already allocated overlapping write manager, then that overlapping write manager is used to execute and replicate the incoming write operation so that the order of execution of overlapping writes by the second computing environment is the same as at the first computing environment. If there is no allocated overlapping write manager for the file handle, then a new overlapping write manager is allocated and utilized.
1. A method comprising:
maintaining a mapping comprising entries used to track overlapping write managers that are allocated to be available for managing overlapping writes targeting files stored by a first computing environment, wherein an overlapping write manager, tracked by an entry within the mapping, is allocated for a file stored by the first computing environment; and
in response to determining that there are no outstanding write operations having range locks for the file, removing the entry from the mapping and deallocating the overlapping write manager.
2. The method of claim 1 , comprising:
releasing a range lock for the file.
3. The method of claim 1 , comprising:
releasing a range lock for the file in response to an incoming write operation completing execution upon the file and upon a replicated file maintained by a second computing environment as a replica of the file.
4. The method of claim 1 , comprising:
in response to determining that there are no outstanding write operations having references to the overlapping write manager, deallocating the overlapping write manager.
5. The method of claim 1 , comprising:
in response to determining that there are no outstanding write operations having references to the overlapping write manager, removing the entry from the mapping.
6. The method of claim 1 , comprising:
in response to determining that there are no outstanding write operations having range locks for the file, removing the entry from the mapping.
7. The method of claim 1 , comprising:
deallocating overlapping write managers until a number of allocated overlapping write managers is less than a threshold.
8. The method of claim 1 , wherein a total number of overlapping write managers is maintained within a pool, and the method comprising:
reducing a number of allocated overlapping write managers from the pool to free computing resources.
9. The method of claim 1 , comprising:
using the overlapping write manager to acquire a range lock for executing an incoming write operation upon the file.
10. The method of claim 1 , comprising:
replicating the overlapping writes to replicated files stored by a second computing environment.
11. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:
maintain a mapping comprising entries used to track overlapping write managers that are allocated to be available for managing overlapping writes targeting files stored by a first computing environment, wherein an overlapping write manager, tracked by an entry within the mapping, is allocated for a file stored by the first computing environment; and
in response to determining that there are no outstanding write operations having range locks for the file, remove the entry from the mapping and deallocate the overlapping write manager.
12. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
in response to removing the entry from the mapping, deallocate the overlapping write manager.
13. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
release a range lock for the file in response to an incoming write operation completing execution upon the file and upon a replicated file maintained by a second computing environment as a replica of the file.
14. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
in response to determining that there are no outstanding write operations having references to the overlapping write manager, deallocate the overlapping write manager.
15. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
in response to determining that there are no outstanding write operations having references to the overlapping write manager, remove the entry from the mapping.
16. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
in response to determining that there are no outstanding write operations having range locks for the file, remove the entry from the mapping.
17. The non-transitory machine readable medium of claim 11 , wherein the instructions cause the machine to:
deallocate overlapping write managers until a number of allocated overlapping write managers is less than a threshold.
18. A computing device comprising:
a memory comprising machine executable code for performing a method; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
maintain a mapping comprising entries used to track overlapping write managers that are allocated to be available for managing overlapping writes targeting files stored by a first computing environment, wherein an overlapping write manager, tracked by an entry within the mapping, is allocated for a file stored by the first computing environment; and
in response to determining that there are no outstanding write operations having range locks for the file, remove the entry from the mapping and deallocate the overlapping write manager.
19. The computing device of claim 18 , wherein the machine executable code causes the processor to:
use the overlapping write manager to acquire a range lock for executing an incoming write operation upon the file.
20. The computing device of claim 18 , wherein the machine executable code causes the processor to:
replicate the overlapping writes to replicated files stored by a second computing environment.