IP Library Granted Patent US 12,339,805
Granted Patent B2
US 12,339,805 · App. 18/362,088 · Granted Jun 24, 2025

Facilitating access to fragmented snapshot data

Inventors: Yosef Shatsky (Karnei Shomron, IL); Doron Tal (Geva Carmel, IL)
Assignee: Dell Products L.P.
G06F16/122G06F11/3409G06F16/128G06F16/164
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,339,805
App. No.
18/362,088
Granted
Jun 24, 2025
Kind
B2
Abstract

Techniques are provided to facilitate access to fragmented snapshot data. For example, a storage control system generates a snapshot data structure of a storage volume. The snapshot data structure comprises plurality of nodes comprising a volume node and one or more snapshot nodes, the volume node is configured to store new and updated data that is written to logical offsets of the storage volume, and the one or more snapshot nodes comprise point-in-time copies of data of the storage volume taken at different times. The storage control system adds an entry for the volume node in a metadata structure associated with the snapshot data structure, wherein the entry comprises a reference to a physical location of data held by a given one of the snapshot nodes at a given logical offset of the storage volume.

Claims (49)

1. A method, comprising:

generating, by a storage control system, a snapshot data structure of a storage volume, wherein the snapshot data structure comprises plurality of nodes comprising a volume node and one or more snapshot nodes, wherein the volume node is configured to store data that is written to one or more logical offsets of the storage volume, and wherein the one or more snapshot nodes comprise point-in-time copies of data of the storage volume taken at different times;

for a given logical offset of the storage volume for which data has not been stored by the volume node, the storage control system adding a volume node entry for the given logical offset in a metadata structure associated with the snapshot data structure, wherein the volume node entry comprises a reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at the given logical offset of the storage volume, wherein the reference is determined by the storage control system using existing referencing metadata associated with the storage volume; and

utilizing, by the storage control system, the reference in the volume node entry of the volume node in the metadata structure to access the current version of the data block held by the given one of the snapshot nodes.

2. The method of claim 1 , wherein the snapshot data structure comprises a snapshot tree structure in which the one or more snapshot nodes comprise a chain of incremental snapshots taken of the storage volume at different times.

3. The method of claim 1 , wherein:

the metadata structure associated with the snapshot data structure comprises a mapping table data structure comprising a metadata entry for each logical offset in each logical address space of each node of the snapshot data structure which holds a data block of the storage volume; and

each metadata entry comprises a key-value pair, wherein the key comprises a combination of a node identifier and logical offset and the value comprises a physical location of a data block stored at the logical offset in the logical address space of the node.

4. The method of claim 1 , wherein the existing referencing metadata comprises deduplication reference metadata which is utilized by the storage control system to determine the reference to the physical location of data held by the given one of the snapshot nodes at the given logical offset of the storage volume.

5. The method of claim 1 , further comprising, for a given sequence of logical offsets of the storage volume for which data has not been stored by the volume node, the storage control system adding a sequence of volume node entries for the given sequence of logical offsets in the metadata structure associated with the snapshot data structure, wherein each volume node entry of the sequence of volume node entries comprises a respective reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at a respective logical offset of the given sequence of logical offsets of the storage volume, wherein each reference is determined by the storage control system using the existing referencing metadata associated with the storage volume.

6. The method of claim 5 , further comprising performing, by the storage control system, a sequential read operation to sequentially access the data blocks in the sequence of logical offsets of the storage volume, which are held by the one or more snapshot nodes, by accessing the sequence of volume node entries for the volume node in the metadata structure to determine the physical location of the data blocks.

7. The method of claim 5 , further comprising:

determining, by the storage control system, a current input/output (I/O) workload associated with the storage volume;

adding the sequence of entries for the volume node in the metadata structure, in response to determining the current I/O workload comprises a read I/O workload; and

refraining from adding the sequence of volume node entries for the volume node in the metadata structure, in response to determining the current I/O workload comprises a write intensive I/O workload.

8. The method of claim 5 , further comprising:

determining, by the storage control system, a fragmentation level of the snapshot data structure; and

adding the sequence of volume node entries for the volume node in the metadata structure, in response to determining that the fragmentation level of the snapshot data structure exceeds a threshold.

9. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to implement a storage control system that is configured to perform a snapshot management process which comprises:

generating a snapshot data structure of a storage volume, wherein the snapshot data structure comprises plurality of nodes comprising a volume node and one or more snapshot nodes, wherein the volume node is configured to store data that is written to one or more logical offsets of the storage volume, and wherein the one or more snapshot nodes comprise point-in-time copies of data of the storage volume taken at different times;

for a given logical offset of the storage volume for which data has not been stored by the volume node, adding a volume node entry for the given logical offset in a metadata structure associated with the snapshot data structure, wherein the volume node entry comprises a reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at the given logical offset of the storage volume, wherein the reference is determined by the storage control system using existing referencing metadata associated with the storage volume; and

utilizing the reference in the volume node entry of the volume node in the metadata structure to access the current version of the data block held by the given one of the snapshot nodes.

10. The computer program product of claim 9 , wherein the snapshot data structure comprises a snapshot tree structure in which the one or more snapshot nodes comprise a chain of incremental snapshots taken of the storage volume at different times.

11. The computer program product of claim 9 , wherein:

the metadata structure associated with the snapshot data structure comprises a mapping table data structure comprising a metadata entry for each logical offset in each logical address space of each node of the snapshot data structure which holds a data block of the storage volume; and

each metadata entry comprises a key-value pair, wherein the key comprises a combination of a node identifier and logical offset and the value comprises a physical location of a data block stored at the logical offset in the logical address space of the node.

12. The computer program product of claim 9 , wherein the existing referencing metadata comprises deduplication reference metadata which is utilized to determine the reference to the physical location of data held by the given one of the snapshot nodes at the given logical offset of the storage volume.

13. The computer program product of claim 9 , for a given sequence of logical offsets of the storage volume for which data has not been stored by the volume node, further comprising program code for adding a sequence of volume node entries for the given sequence of logical offsets in the metadata structure associated with the snapshot data structure, wherein each volume node entry of the sequence of volume node entries comprises a respective reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at a respective logical offset of the given sequence of logical offsets of the storage volume, wherein each reference is determined by the storage control system using the existing referencing metadata associated with the storage volume.

14. The computer program product of claim 13 , further comprising program code for performing a sequential read operation to sequentially access the data blocks in the sequence of logical offsets of the storage volume, which are held by the one or more snapshot nodes, by accessing the sequence of volume node entries for the volume node in the metadata structure to determine the physical location of the data blocks.

15. The computer program product of claim 13 , further comprising program code for:

determining a current input/output (I/O) workload associated with the storage volume;

adding the sequence of volume node entries for the volume node in the metadata structure, in response to determining the current I/O workload comprises a read I/O workload; and

refraining from adding the sequence of volume node entries for the volume node in the metadata structure, in response to determining the current I/O workload comprises a write intensive I/O workload.

16. The computer program product of claim 13 , further comprising program code for:

determining a fragmentation level of the snapshot data structure; and

adding the sequence of volume node entries for the volume node in the metadata structure, in response to determining that the fragmentation level of the snapshot data structure exceeds a threshold.

17. An apparatus, comprising:

at least one processing device, and memory to store program instructions that are executed by the at least one processing device to implement a storage control system that is configured to:

generate snapshot data structure of a storage volume, wherein the snapshot data structure comprises plurality of nodes comprising a volume node and one or more snapshot nodes, wherein the volume node is configured to store data that is written to one or more logical offsets of the storage volume, and wherein the one or more snapshot nodes comprise point-in-time copies of data of the storage volume taken at different times;

for a given logical offset of the storage volume for which data has not been stored by the volume node, add a volume node entry for the given logical offset in a metadata structure associated with the snapshot data structure, wherein the volume node entry comprises a reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at the given logical offset of the storage volume, wherein the reference is determined by the storage control system using existing referencing metadata associated with the storage volume; and

utilize the reference in the volume node entry of the volume node in the metadata structure to access the current version of the data block held by the given one of the snapshot nodes.

18. The apparatus of claim 17 , wherein:

the snapshot data structure comprises a snapshot tree structure in which the one or more snapshot nodes comprise a chain of incremental snapshots taken of the storage volume at different times;

the metadata structure associated with the snapshot data structure comprises a mapping table data structure comprising a metadata entry for each logical offset in each logical address space of each node of the snapshot data structure which holds a data block of the storage volume; and

each metadata entry comprises a key-value pair, wherein the key comprises a combination of a node identifier and logical offset, and wherein the value comprises a physical location of a data block stored at the logical offset in the logical address space of the node.

19. The apparatus of claim 17 , wherein the existing referencing metadata comprises deduplication reference metadata which is utilized to determine the reference to the physical location of data held by the given one of the snapshot nodes at the given logical offset of the storage volume.

20. The apparatus of claim 17 , wherein:

for a given sequence of logical offsets of the storage volume for which data has not been stored by the volume node, the storage control system is configured to add a sequence of volume node entries for the given sequence of logical offsets in the metadata structure associated with the snapshot data structure, wherein each volume node entry of the sequence of volume node entries comprises a respective reference which points to a physical location of a current version of a data block held by a given one of the snapshot nodes other than the volume node at a respective logical offset of the given sequence of logical offsets of the storage volume, wherein each reference is determined by the storage control system using the existing referencing metadata associated with the storage volume; and

the storage control system is configured to perform a sequential read operation to sequentially access the data blocks in the sequence of logical offsets of the storage volume, which are held by the one or more snapshot nodes, by accessing the sequence of volume node entries for the volume node in the metadata structure to determine the physical location of the data blocks.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 31, 2023
From: SHATSKY, YOSEF; TAL, DORON
To: DELL PRODUCTS L.P.
Reel/Frame 064433/0797 →
Continuity (1)
Related Publication 20250045242A1 · Feb 6, 2025
References Cited (125)
US 5381539A · Yanai et al. · 1995 [cited by applicant]
US 5551003A · Mattson et al. · 1996 [cited by applicant]
US 5764880A · Gerdt et al. · 1998 [cited by applicant]
US 6052799A · Li et al. · 2000 [cited by applicant]
US 6941420B2 · Butterworth et al. · 2005 [cited by applicant]
US 7617358B1 · Liikanen · 2009 [cited by examiner]
US 7987156B1 · Chatterjee · 2011 [cited by examiner]
US 8843676B2 · Rajamanickam et al. · 2014 [cited by applicant]
US 9372751B2 · McNutt · 2016 [cited by applicant]
US 9514014B2 · Webman et al. · 2016 [cited by applicant]
US 9892045B1 · Douglis et al. · 2018 [cited by applicant]
US 10078598B1 · Wallace et al. · 2018 [cited by applicant]
US 10331561B1 · Shilane et al. · 2019 [cited by applicant]
US 10445180B2 · Butterworth et al. · 2019 [cited by applicant]
US 10986174B1 · Sharma et al. · 2021 [cited by applicant]
US 11119668B1 · Keller et al. · 2021 [cited by applicant]
US 11144399B1 · Yarimi et al. · 2021 [cited by applicant]
US 11163479B2 · Lieblich et al. · 2021 [cited by applicant]
US 11163699B2 · Keller et al. · 2021 [cited by applicant]
US 11221975B2 · Puder et al. · 2022 [cited by applicant]
US 11262933B2 · Matosevich et al. · 2022 [cited by applicant]
US 11301162B2 · Matosevich et al. · 2022 [cited by applicant]
US 11307935B2 · Keller et al. · 2022 [cited by applicant]
US 11372810B2 · Keller et al. · 2022 [cited by applicant]
US 11416396B2 · Shatsky et al. · 2022 [cited by applicant]
US 11418589B1 · Spiegelman · 2022 [cited by applicant]
US 11487432B2 · Aharoni et al. · 2022 [cited by applicant]
US 11487460B2 · Keller et al. · 2022 [cited by applicant]
US 11513997B2 · Keller et al. · 2022 [cited by applicant]
US 11550479B1 · Shatsky et al. · 2023 [cited by applicant]
US 11573736B2 · Matosevich et al. · 2023 [cited by applicant]
US 11606429B2 · Aharoni et al. · 2023 [cited by applicant]
US 11609854B1 · Shatsky et al. · 2023 [cited by applicant]
US 11630773B1 · Shatsky et al. · 2023 [cited by applicant]
US 11636089B2 · Aharoni et al. · 2023 [cited by applicant]
US 11650920B1 · Shatsky et al. · 2023 [cited by applicant]
US 11675789B2 · Shatsky et al. · 2023 [cited by applicant]
US 11687536B2 · Sharma et al. · 2023 [cited by applicant]
US 11704053B1 · Tal et al. · 2023 [cited by applicant]
US 11704160B2 · Shatsky et al. · 2023 [cited by applicant]
US 20020032835A1 · Li et al. · 2002 [cited by applicant]
US 20080021853A1 · Modha et al. · 2008 [cited by applicant]
US 20090204761A1 · Caprioli et al. · 2009 [cited by applicant]
US 20090276593A1 · Jacobson et al. · 2009 [cited by applicant]
US 20120278560A1 · Benzion · 2012 [cited by examiner]
US 20130042056A1 · Shats · 2013 [cited by examiner]
US 20130305002A1 · Hallak et al. · 2013 [cited by applicant]
US 20140215147A1 · Pan · 2014 [cited by applicant]
US 20140215262A1 · Li et al. · 2014 [cited by applicant]
US 20140244935A1 · Ezra et al. · 2014 [cited by applicant]
US 20150134879A1 · Zheng · 2015 [cited by examiner]
US 20160103764A1 · Banerjee et al. · 2016 [cited by applicant]
US 20180113640A1 · Fernandez et al. · 2018 [cited by applicant]
US 20180267893A1 · Barzik et al. · 2018 [cited by applicant]
US 20180300075A1 · Fernandez et al. · 2018 [cited by applicant]
US 20180356989A1 · Meister · 2018 [cited by examiner]
US 20190163587A1 · Anna et al. · 2019 [cited by applicant]
US 20190227845A1 · Sridhar et al. · 2019 [cited by applicant]
US 20200133503A1 · Sun et al. · 2020 [cited by applicant]
US 20200356442A1 · Agarwal · 2020 [cited by examiner]
US 20210073079A1 · Venkatesan · 2021 [cited by examiner]
US 20210279187A1 · Puder et al. · 2021 [cited by applicant]
US 20210294505A1 · Keller et al. · 2021 [cited by applicant]
US 20210294774A1 · Keller · 2021 [cited by examiner]
US 20210294775A1 · Keller et al. · 2021 [cited by applicant]
US 20210303160A1 · Lieblich et al. · 2021 [cited by applicant]
US 20210303169A1 · Tagar et al. · 2021 [cited by applicant]
US 20210303202A1 · Ben Zeev et al. · 2021 [cited by applicant]
US 20210303401A1 · Keller et al. · 2021 [cited by applicant]
US 20210303407A1 · Keller · 2021 [cited by examiner]
US 20210303480A1 · Yarimi et al. · 2021 [cited by applicant]
US 20210342297A1 · Gupta · 2021 [cited by examiner]
US 20210373796A1 · Matosevich et al. · 2021 [cited by applicant]
US 20220004320A1 · Matosevich et al. · 2022 [cited by applicant]
US 20220035788A1 · Aharoni et al. · 2022 [cited by applicant]
US 20220113867A1 · Aharoni et al. · 2022 [cited by applicant]
US 20220114184A1 · Sharma et al. · 2022 [cited by applicant]
US 20220116454A1 · Aharoni et al. · 2022 [cited by applicant]
US 20220121458A1 · Moran et al. · 2022 [cited by applicant]
US 20220129380A1 · Shatsky et al. · 2022 [cited by applicant]
US 20220171567A1 · Matosevich et al. · 2022 [cited by applicant]
US 20220187991A1 · Keller et al. · 2022 [cited by applicant]
US 20220222113A1 · Shatsky et al. · 2022 [cited by applicant]
US 20220342758A1 · Tal et al. · 2022 [cited by applicant]
US 20220350497A1 · Matosevich et al. · 2022 [cited by applicant]
US 20220358018A1 · Bar Shalom et al. · 2022 [cited by applicant]
US 20220405254A1 · Shatsky et al. · 2022 [cited by applicant]
US 20220414102A1 · Shatsky et al. · 2022 [cited by applicant]
US 20230127321A1 · Shatsky et al. · 2023 [cited by applicant]
US 20230236966A1 · Yarimi et al. · 2023 [cited by applicant]
US 20230237029A1 · Tal et al. · 2023 [cited by applicant]
US 20230334011A1 · Shatsky · 2023 [cited by examiner]
US 20240143554A1 · Kaushik · 2024 [cited by examiner]
WO 2015108670A1 · 2015 [cited by applicant]
WO 2020204880A1 · 2020 [cited by applicant]
WO 2020204882A1 · 2020 [cited by applicant]
International Search Report and Written Opinion of PCT/US2019/024885 dated Jan. 7, 2020, 13 pages. [cited by applicant]
International Search Report and Written Opinion of PCT/US2019/024900 dated Jan. 7, 2020, 12 pages. [cited by applicant]
DELL EMC, “EMC ScaleIO Basic Architecture Documentation,” Technical White Paper, Mar. 2017, 22 pages. [cited by applicant]
EMC2, “EMC ScaleIO Design Considerations and Best Practices,” Technical White Paper, Jun. 2016, 30 pages. [cited by applicant]
I. Koltsidas et al., “SoftwAre Log-Structured Array (SALSA)—A Unified Stack for SSDs and SMR Disks,” IBM Research Report, Dec. 2, 2015, 13 pages. [cited by applicant]
S. M. Rumble et al., “Log-Structured Memory for DRAM-Based Storage,” Proceedings of the 12th USENIX Conference on File and Storage Technologies, Santa Clara, CA, Feb. 17-20, 2014, 17 pages. [cited by applicant]
Dell EMC, “Dell EMC VxFlex Family Overview,” Technical White Paper, May 2019, 44 pages. [cited by applicant]
J. Nakano et al., “ReVivel/O: Efficient Handling of I/O in Highly-Available Rollback-Recovery Servers,” IEEE Symposium on High-Performance Computer Architecture, Feb. 11-15, 2006, pp. 200-211. [cited by applicant]
Wikipedia, “Raft (Computer Science),” https://en.wikipedia.org/wiki/Raft_(computer_science), Feb. 10, 2020, 4 pages. [cited by applicant]
Wikipedia, “Paxos (Computer Science),” https://en.wikipedia.org/wiki/Paxos_(computer_science), Dec. 6, 2019, 21 pages. [cited by applicant]
Wikipedia, “State Machine Replication,” https://en.wikipedia.org/wiki/State_machine_replication, Dec. 14, 2019, 9 pages. [cited by applicant]
Dell Technologies, “Dell EMC PowerFlex: Secure Snapshots,” Technical White Paper, Jul. 2020, 17 pages. [cited by applicant]
Dell Technologies, “Dell EMC PowerFlex: Protected Maintenance Mode,” Technical White Paper, Jul. 2020, 20 pages. [cited by applicant]
Dell Technologies, “Dell EMC PowerFlex: Introduction to Replication,” Technical White Paper, Jun. 2020, 34 pages. [cited by applicant]
Dell Technologies, “Dell EMC PowerFlex: Networking Best Practices and Design Considerations,” Best Practices, Jun. 2020, 64 pages. [cited by applicant]
Dell EMC, “Getting To Know Dell EMC PowerFlex,” Version 3.5.x, Rev. 02, Jan. 2021, 66 pages. [cited by applicant]
Dell EMC, “Dell EMC VxRack FLEX,” Dell EMC Product Overview, Apr. 2018, 5 pages. [cited by applicant]
G. Soundararajan et al., “Dynamic Resource Allocation for Database Servers Running on Virtual Storage,” FAST 2009: Proceedings of the 7th conference on File and storage technologies, Feb. 2009, pp. 71-84. [cited by applicant]
U.S. Appl. No. 17/681,449 filed in the name of Yosef Shatsky et al. Feb. 25, 2022, and entitled “Optimization for Garbage Collection in a Storage System.” [cited by applicant]
U.S. Appl. No. 17/726,853 filed in the name of Irit Brener-Shalem et al. Apr. 22, 2022, and entitled “Intelligent Load Scheduling in a Storage System.” [cited by applicant]
U.S. Appl. No. 17/729,219 filed in the name of Yosef Shatsky et al. Apr. 26, 2022, and entitled “Load Distribution in a Data Storage System.” [cited by applicant]
U.S. Appl. No. 17/853,364 filed in the name of Yosef Shatsky et al. Jun. 29, 2022, and entitled “Managing Lookup Operations of a Metadata Structure for a Storage System.” [cited by applicant]
U.S. Appl. No. 17/864,579 filed in the name of Yosef Shatsky et al. Jul. 14, 2022, and entitled “Managing Granularity of a Metadata Structure for a Storage System.” [cited by applicant]
U.S. Appl. No. 17/868,045 filed in the name of Yosef Shatsky et al. Jul. 19, 2022, and entitled “Managing Insert Operations of a Metadata Structure for a Storage System.” [cited by applicant]
U.S. Appl. No. 17/969,875 filed in the name of Yosef Shatsky et al. Oct. 20, 2022, and entitled “Multiple-Instance Write Cache for a Storage System.” [cited by applicant]
U.S. Appl. No. 18/090,792 filed in the name of Christopher Trudel et al. Dec. 29, 2022, and entitled “Cluster Management in Large-Scale Storage Systems.” [cited by applicant]
U.S. Appl. No. 18/092,516 filed in the name of Igal Moshkovich et al. Jan. 3, 2023, and entitled “Managing Data on Shutdown of Storage System.” [cited by applicant]
U.S. Appl. No. 18/138,415 filed in the name of Yosef Shatsky et al. Apr. 24, 2023, and entitled “Dynamic Reserve Capacity in Storage Systems.” [cited by applicant]
U.S. Appl. No. 18/138,415 filed in the name of Doron Tal et al. Jul. 13, 2023, and entitled “Multi-Modal Write Cache for Data Storage System.” [cited by applicant]