IP Library Granted Patent US 12,086,116
Granted Patent B2
US 12,086,116 · App. 17/976,946 · Granted Sep 10, 2024

Object and sequence number management

Inventors: Ananthan Subramanian (San Ramon, CA); Sridevi Jantli (San Jose, CA); Anil Paul Thoppil (Sunnyvale, CA); Cheryl Marie Thompson (Sunnyvale, CA); Qinghua Zheng (San Jose, CA)
Assignee: NetApp, Inc.
G06F16/215G06F16/22G06F16/2379
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,086,116
App. No.
17/976,946
Granted
Sep 10, 2024
Kind
B2
Abstract

Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Claims (63)

1. A method, comprising:

storing data of a volume into an object within a remote object store, wherein an object name is assigned to the object based upon a volume identifier of the volume stored within a storage system external to the remote object store;

for each object name within a list of object names of objects stored within the remote object store:

in response to determining that the object name of the object matches the volume identifier within a list of volume identifiers of volumes stored within the storage system external to the remote object store, designating the object as a valid object storing the data of the volume having the volume identifier; and

in response to determining that the object name does not match at least one volume identifier within the list of volume identifiers, designating the object as an orphan object that stores stale data no longer referenced by the volumes; and

deleting the orphan object from the remote object store to reclaim storage space consumed by the orphan object.

2. The method of claim 1 , comprising:

generating an orphan object list of orphan objects determined from the list of object names;

generating an orphan volume identifier list of volume identifiers of the orphan objects; and

displaying the orphan object list and the orphan volume identifier list to a user.

3. The method of claim 1 , comprising:

in response to the storage system receiving a delete command to delete the orphan object, transmitting a delete operation to the remote object store to delete the orphan object.

4. The method of claim 1 , comprising:

in response to the storage system receiving a delete all command to delete the orphan objects within an orphan object list of orphan objects determined from the list of object names, transmitting delete operations to the remote object store to delete the orphan objects within the orphan object list.

5. The method of claim 1 , comprising:

in response to the storage system receiving a delete all command to delete orphan objects associated with the volume identifier, transmitting delete operations to the remote object store to delete the orphan objects associated with the volume identifier.

6. The method of claim 1 , comprising:

receiving a command from a requestor to retrieve data from the orphan object;

retrieving the orphan object from the remote object store; and

providing the data of the orphan object to the requestor.

7. The method of claim 1 , comprising:

issuing a first batch command to the remote object store for a first portion of the list of object names.

8. The method of claim 7 , comprising:

issuing a second batch command to the remote object store for a second portion of the list of object names.

9. The method of claim 1 , comprising:

storing the list of volume identifiers within a cluster-wide database accessible to a plurality of nodes of the storage system separate from the remote object store that is hosted by a third party provider; and

in response to a set of objects being deleted from the remote object store, removing volume identifiers, corresponding to the set of objects, from the cluster-wise database.

10. The method of claim 1 , comprising:

creating the object name for the object to include a prefix derived from the volume identifier of the volume.

11. The method of claim 1 , comprising:

creating the object name for the object based upon the volume identifier and a sequence number assigned to the object.

12. The method of claim 1 , comprising:

creating the object name for the object based upon a hash of the volume identifier and a sequence number assigned to the object.

13. A computing device comprising:

a memory comprising machine executable code; and

a processor coupled to the memory, the processor configured to execute the machine executable code to cause the computing device to:

store data of a volume into an object within a remote object store, wherein an object name is assigned to the object based upon a sequence number assigned to the object and a volume identifier of the volume stored within a storage system external to the remote object store;

for each object name within a list of object names of objects stored within the remote object store:

in response to determining that the object name of the object matches the volume identifier within a list of volume identifiers of volumes stored within the storage system external to the remote object store, designating the object as a valid object storing the data of the volume having the volume identifier; and

in response to determining that the object name does not match at least one volume identifier within the list of volume identifiers, designating the object as an orphan object that stores stale data no longer referenced by the volumes; and

delete the orphan object from the remote object store to reclaim storage space consumed by the orphan object.

14. The computing device of claim 13 , wherein the machine executable code cause the computing device to:

generate an orphan object list of orphan objects determined from the list of object names;

generate an orphan volume identifier list of volume identifiers of the orphan objects

display the orphan object list and the orphan volume identifier list to a user.

15. The computing device of claim 13 , wherein the machine executable code cause the computing device to:

store the list of volume identifiers within a cluster-wide database accessible to a plurality of nodes of the storage system separate from the remote object store that is hosted by a third party provider; and

in response to a set of objects being deleted from the remote object store, removing volume identifiers, corresponding to the set of objects, from the cluster-wise database.

16. The computing device of claim 13 , wherein the machine executable code cause the computing device to:

create the object name for the object to include a prefix derived from the volume identifier of the volume.

17. The computing device of claim 13 , wherein the machine executable code cause the computing device to:

create the object name for the object based upon a hash of the volume identifier.

18. The computing device of claim 13 , wherein the machine executable code cause the computing device to:

create the object name for the object based upon a hash of the volume identifier and the sequence number assigned to the object.

19. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:

store data of a volume into an object within a remote object store, wherein an object name is assigned to the object based upon a sequence number assigned to the object and a hash of a volume identifier of the volume stored within a storage system external to the remote object store;

for each object name within a list of object names of objects stored within the remote object store:

in response to determining that the object name of the object matches the volume identifier within a list of volume identifiers of volumes stored within the storage system external to the remote object store, designate the object as a valid object storing the data of the volume having the volume identifier; and

in response to determining that the object name does not match at least one volume identifier within the list of volume identifiers, designate the object as an orphan object that stores stale data no longer referenced by the volumes; and

delete the orphan object from the remote object store to reclaim storage space consumed by the orphan object.

20. The non-transitory machine readable medium of claim 19 , wherein the instructions cause the machine to:

issue a first batch command to the remote object store for a first portion of the list of object names; and

issue a second batch command to the remote object store for a second portion of the list of object names.

Continuity (3)
Continuation 16751766 · Jan 24, 2020
Continuation 15581447 · Apr 28, 2017
Related Publication 20230052732A1 · Feb 16, 2023
Cited By (3)
US 12,346,324 US 12,675,410 US 12,688,171