Parallel pipelined processing for snapshot data deletion
Techniques delete snapshot data. In accordance with certain techniques, a first sub-process of a snapshot deletion process on a first data block of the snapshot data is performed with a first thread. The snapshot deletion process includes at least the first sub-process and a second sub-process, the first and second sub-processes being performed sequentially. In response to an end of the first sub-process performed with the first thread, the second sub-process on the first data block is performed with the first thread. In parallel with performing, with the first thread, the second sub-process on the first data block, the first sub-process on a second data block of the snapshot data is performed with a second thread different from the first thread, the second data block being different from the first data block. Such techniques improve IO lock contention, system resource utilization rate and parallelism, response time and system overhead.
1. A method of deleting snapshot data using a snapshot deletion process that includes multiple sub-processes arranged for execution in a pipeline, comprising:
performing, with a first thread, a first sub-process of the snapshot deletion process on a first data block of the snapshot data, the snapshot deletion process comprising at least the first sub-process and a second sub-process, the first and second sub-processes being performed sequentially;
in response to an end of the first sub-process performed with the first thread, performing, with the first thread, the second sub-process on the first data block; and
in parallel with performing, with the first thread, the second sub-process on the first data block, performing, with a second thread different from the first thread, the first sub-process on a second data block of the snapshot data, the second data block being different from the first data block,
the snapshot deletion process proceeding with multiple threads operating on respective blocks in parallel and with different ones of the threads performing different sub-processes at a same time, thereby avoiding I/O (Input/Output) lock contention,
wherein the multiple sub-processes include a stage of calculating a weight value to be merged or returned when deleting the data block and a stage of updating block metadata (BMD) of the data block, wherein the weight value, which tracks sharing among blocks, is merged responsive to a determination to calculate a merged weight value and is returned responsive to the block being shared and there being no existing block.
2. The method according to claim 1 , further comprising:
in response to an end of the first sub-process performed with the second thread, performing, with the second thread, the second sub-process on the second data block; and
in parallel with performing, with the second thread, the second sub-process on the second data block, performing, with a third thread different from the second thread, the first sub-process on a third data block of the snapshot data, the third data block being different from the first and second data blocks.
3. The method according to claim 2 , wherein the third thread is different from the first thread.
4. The method according to claim 1 , further comprising:
allocating a predetermined number of threads to delete the snapshot data, the predetermined number of threads comprising at least the first thread and the second thread.
5. The method according to claim 1 , wherein the first thread is executed by a first core of a processing unit, and the second thread is executed by a second core of the processing unit, the first core being different from the second core.
6. The method of claim 1 , wherein the multiple sub-processes further include:
a stage of reading metadata of a data block being processed; and
a stage of writing a log that records the update to the block metadata.
7. The method of claim 1 , wherein performing different sub-processes at the same time with different ones of the threads includes performing no one sub-process by more than one of the threads at the same time.
8. A device for deleting snapshot data, comprising:
a processor; and
a memory storing computer-executable instructions which, the instructions, when executed by the processor, causing the device to perform a method of deleting snapshot data using a snapshot deletion process that includes multiple sub-processes arranged for execution in a pipeline, the method comprising:
performing, with a first thread, a first sub-process of a snapshot deletion process on a first data block of the snapshot data, the snapshot deletion process comprising at least the first sub-process and a second sub-process, the first and second sub-processes being performed sequentially;
in response to an end of the first sub-process performed with the first thread, performing, with the first thread, the second sub-process on the first data block; and
in parallel with performing, with the first thread, the second sub-process on the first data block, performing, with a second thread different from the first thread, the first sub-process on a second data block of the snapshot data, the second data block being different from the first data block,
the snapshot deletion process proceeding with multiple threads operating on respective blocks in parallel and with different ones of the threads performing different sub-processes at a same time, thereby avoiding I/O (Input/Output) lock contention,
wherein the multiple sub-processes include a stage of calculating a weight value to be merged or returned when deleting the data block and a stage of updating block metadata (BMD) of the data block, wherein the weight value, which tracks sharing among blocks, is merged responsive to a determination to calculate a merged weight value.
9. The device according to claim 6 , wherein the method further comprises:
in response to an end of the first sub-process performed with the second thread, performing, with the second thread, the second sub-process on the second data block; and
in parallel with performing, with the second thread, the second sub-process on the second data block, performing, with a third thread different from the second thread, the first sub-process on a third data block of the snapshot data, the third data block being different from the first and second data blocks.
10. The device according to claim 9 , wherein the third thread is different from the first thread.
11. The device according to claim 8 , wherein the method further comprises:
allocating a predetermined number of threads to delete the snapshot data, the predetermined number of threads comprising at least the first thread and the second thread.
12. The device according to claim 8 , wherein the first thread is executed by a first core of a processing unit, and the second thread is executed by a second core of the processing unit, the first core being different from the second core.
13. A method of deleting snapshot data, comprising:
allocating multiple threads;
assigning the threads to respective blocks, the threads configured to perform a snapshot deletion process on the respective blocks, the snapshot deletion process including multiple pipelined sub-processes arranged for execution during sequential processing cycles;
initiating the snapshot deletion process by the respective threads on different processing cycles, the threads then independently performing the sub-processes on the respective blocks in a parallel arrangement that performs different sub-processes by different threads and avoids I/O (Input/Output) lock contention,
wherein the multiple sub-processes include a stage of calculating a weight value to be merged or returned when deleting the data block and a stage of updating block metadata (BMD) of the data block, wherein the weight value, which tracks sharing among blocks, is returned responsive to the block being shared and there being no existing block.
14. The method of claim 13 , further comprising executing the threads on respective cores of a processing unit.
15. The method of claim 13 , wherein the multiple sub-processes include:
a stage of reading metadata of a data block being processed; and
a stage of writing a log that records the update to the block metadata.