Method and system for metadata churn absorption
According to some embodiment, a backup storage system determines whether one or more data segments associated with a file object have been updated, where the data segment(s) are stored in a first segment level in a multi-level hierarchy. For each updated data segment, the system applies a reference-based hierarchical data structure to track metadata associated with the updated data segment, where the metadata is stored in a second segment level in the multi-level hierarchy. The system determines whether the reference-based hierarchical data structure has reached a specific size. The system writes the metadata associated with the updated data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.
1. A computer-implemented method for absorbing churn of a cache memory device, the method comprising:
determining whether one or more data segments associated with a file object have been modified, wherein the data segment(s) are stored in a first segment level in a multi-level data structure;
for each modified data segment,
applying a reference-based hierarchical data structure to only track metadata associated with the modified data segment, wherein the metadata is stored in a second segment level in the multi-level data structure, the second segment level being a parent level segment to the first segment level in the multi-level data structure;
determining whether the reference-based hierarchical data structure has reached a specific size; and
writing the metadata associated with the modified data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.
2. The method of claim 1 , wherein applying the reference-based hierarchical data structure to track metadata associated with the modified data segment comprises dynamically allocating one or more nodes within the reference-based hierarchical data structure, wherein each of the nodes stores a reference to a data region within the second segment level.
3. The method of claim 1 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises writing the metadata in a first of a plurality of write-evict units (WEUs) stored in the SSD.
4. The method of claim 1 , wherein the specific size is indicative that the reference-based hierarchical data structure is full.
5. The method of claim 1 , wherein the reference-based hierarchical data structure is a binary tree, B-tree, B+ tree, or heap.
6. The method of claim 2 , wherein the one or more nodes include a root node, one or more intermediate nodes, and one or more leaf nodes.
7. The method of claim 1 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises performing only one read cycle and one write cycle on the SSD.
8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations to absorb churn of a cache memory device, the operations comprising:
determining whether one or more data segments associated with a file object have been modified, wherein the data segment(s) are stored in a first segment level in a multi-level data structure;
for each modified data segment,
applying a reference-based hierarchical data structure to only track metadata associated with the modified data segment, wherein the metadata is stored in a second segment level in the multi-level data structure, the second segment level being a parent level segment to the first segment level in the multi-level data structure;
determining whether the reference-based hierarchical data structure has reached a specific size; and
writing the metadata associated with the modified data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.
9. The non-transitory machine-readable medium of claim 8 , wherein applying the reference-based hierarchical data structure to track metadata associated with the modified data segment comprises dynamically allocating one or more nodes within the reference-based hierarchical data structure, wherein each of the nodes stores a reference to a data region within the second segment level.
10. The non-transitory machine-readable medium of claim 8 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises writing the metadata in a first of a plurality of write-evict units (WEUs) stored in the SSD.
11. The non-transitory machine-readable medium of claim 8 , wherein the specific size is indicative that the reference-based hierarchical data structure is full.
12. The non-transitory machine-readable medium of claim 8 , wherein the reference-based hierarchical data structure is a binary tree, B-tree, B+ tree, or heap.
13. The non-transitory machine-readable medium of claim 9 , wherein the one or more nodes include a root node, one or more intermediate nodes, and one or more leaf nodes.
14. The non-transitory machine-readable medium of claim 8 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises performing only one read cycle and one write cycle on the SSD.
15. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including:
determining whether one or more data segments associated with a file object have been modified, wherein the data segment(s) are stored in a first segment level in a multi-level data structure;
for each modified data segment,
applying a reference-based hierarchical data structure to only track metadata associated with the modified data segment, wherein the metadata is stored in a second segment level in the multi-level data structure, the second segment level being a parent level segment to the first segment level in the multi-level data structure;
determining whether the reference-based hierarchical data structure has reached a specific size; and
writing the metadata associated with the modified data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.
16. The data processing system of claim 15 , wherein applying the reference-based hierarchical data structure to track metadata associated with the modified data segment comprises dynamically allocating one or more nodes within the reference-based hierarchical data structure, wherein each of the nodes stores a reference to a data region within the second segment level.
17. The data processing system of claim 15 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises writing the metadata in a first of a plurality of write-evict units (WEUs) stored in the SSD.
18. The data processing system of claim 15 , wherein the specific size is indicative that the reference-based hierarchical data structure is full.
19. The data processing system of claim 15 , wherein the reference-based hierarchical data structure is a binary tree, B-tree, B+ tree, or heap.
20. The data processing system of claim 16 , wherein the one or more nodes include a root node, one or more intermediate nodes, and one or more leaf nodes.
21. The data processing system of claim 15 , wherein writing the metadata associated with the modified data segment(s) to the SSD comprises performing only one read cycle and one write cycle on the SSD.