Storage system with decoupling of reference count updates
A storage system comprises a plurality of storage devices and an associated storage controller. The storage controller is configured to receive a request to copy a first range of logical addresses to a second range of logical addresses, determine at least one physical block of the storage devices to which the first range of logical addresses is mapped, map the second range of logical addresses to the determined at least one physical block, and add at least one content-based signature associated with the determined at least one physical block to a pending increment data structure that includes content-based signatures corresponding to physical blocks for which an increment of an associated reference count is pending. The storage controller is further configured to execute a pending increment of a reference count associated with a given physical block corresponding to at least one of the content-based signatures in the pending increment data structure.
1. An apparatus comprising:
a content addressable storage system comprising a plurality of storage devices and an associated storage controller, the storage controller being configured:
to implement a logical storage layer of the content addressable storage system by mapping logical addresses associated with user data pages to content-based signatures respectively derived from a content of those user data pages;
to implement a physical storage layer of the content addressable storage system by mapping the content-based signatures to respective physical blocks in the storage devices where the corresponding user data pages are stored;
to implement a pending increment data structure associated with the storage controller, the pending increment data structure comprising a first plurality of entries, each entry in the pending increment data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which an increment of a corresponding reference count is pending, each corresponding reference count comprising an indication of a number of logical addresses that are mapped to the corresponding content-based signature in the logical storage layer; and
to implement a pending decrement data structure associated with the storage controller, the pending decrement data structure comprising a second plurality of entries, each entry in the pending decrement data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which a decrement of a corresponding reference count is pending;
to receive a request to copy user data pages associated with a first range of logical addresses to a second range of logical addresses;
to determine a content-based signature corresponding to at least a portion of the first range of logical addresses based at least in part on the mapping of the logical storage layer;
to map at least a portion of the second range of logical addresses to the determined content-based signature in the logical storage layer;
to add the determined content-based signature to a first entry of the pending increment data structure;
to determine that the pending decrement data structure comprises a second entry comprising the determined content-based signature;
to execute a pending decrement of the reference count corresponding to the determined content-based signature based at least in part on the determination that the pending decrement data structure comprises the second entry comprising the determined content-based signature, the execution of the pending decrement comprising:
determining that a given set of entries of the first plurality of entries of the pending increment data structure each comprise the determined content-based signature;
determining that a first timestamp of the pending increment data structure that corresponds to each entry in the given set of entries is earlier than a second timestamp of the pending decrement data structure that corresponds to an entry of the pending decrement data structure that comprises the determined content-based signature;
executing at least one pending increment of the reference count corresponding to at least one entry in the given set of entries based at least in part on the determination that the first timestamp is earlier than the second timestamp; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after the execution of the at least one pending increment;
wherein the storage controller is implemented using at least one processing device comprising a processor coupled to a memory.
2. The apparatus of claim 1 , wherein the pending increment data structure and the pending decrement data structure are the same data structure.
3. The apparatus of claim 1 , wherein the first timestamp of the pending increment data structure represents a time at which the determined content-based signature was added to an entry of the given set of entries of the pending increment data structure.
4. The apparatus of claim 3 , wherein the first timestamp of the pending increment data structure represents an earliest time at which the determined content-based signature was added to one of the given set of entries in the pending increment data structure.
5. The apparatus of claim 4 , wherein:
the determined content-based signature was added to a given entry of the given set of entries of the pending increment data structure later than the second timestamp;
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries including the given entry; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the at least one pending increment comprises executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the given entry.
6. The apparatus of claim 3 , wherein the second timestamp of the pending decrement data structure represents a time at which the determined content-based signature was added to an entry of the pending decrement data structure.
7. The apparatus of claim 6 , wherein the second timestamp of the pending decrement data structure represents a latest time at which the determined content-based signature was added to an entry of the pending decrement data structure.
8. The apparatus of claim 1 , wherein:
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries; and
the execution of a given pending increment of the plurality of pending increments comprises: removing the determined content-based signature from both a given entry corresponding to the given pending increment in the pending increment data structure and the second entry comprising the determined content-based signature in the pending decrement data structure without incrementing or decrementing the reference count corresponding to the determined content-based signature.
9. A method comprising:
receiving a request to copy user data pages associated with a first range of logical addresses to a second range of logical addresses;
determining a content-based signature corresponding to at least a portion of the first range of logical addresses based at least in part on a mapping of a logical storage layer of a content addressable storage system, the content addressable storage system comprising a plurality of storage devices and an associated storage controller, the storage controller being configured:
to implement the logical storage layer of the content addressable storage system by mapping logical addresses associated with user data pages to content-based signatures respectively derived from a content of those user data pages;
to implement a physical storage layer of the content addressable storage system by mapping the content-based signatures to respective physical blocks in the storage devices where the corresponding user data pages are stored;
to implement a pending increment data structure associated with the storage controller, the pending increment data structure comprising a first plurality of entries, each entry in the pending increment data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which an increment of a corresponding reference count is pending, each corresponding reference count comprising an indication of a number of logical addresses that are mapped to the corresponding content-based signature in the logical storage layer; and
to implement a pending decrement data structure associated with the storage controller, the pending decrement data structure comprising a second plurality of entries, each entry in the pending decrement data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which a decrement of a corresponding reference count is pending;
mapping at least a portion of the second range of logical addresses to the determined content-based signature in the logical storage layer;
adding the determined content-based signature to a first entry of the pending increment data structure;
determining that the pending decrement data structure comprises a second entry comprising the determined content-based signature;
executing a pending decrement of the reference count corresponding to the determined content-based signature based at least in part on the determination that the pending decrement data structure comprises a second entry comprising the determined content-based signature, the execution of the pending decrement comprising:
determining that a given set of entries of the first plurality of entries of the pending increment data structure each comprise the determined content-based signature;
determining that a first timestamp of the pending increment data structure that corresponds to each entry in the given set of entries is earlier than a second timestamp of the pending decrement data structure that corresponds to an entry of the pending decrement data structure that comprises the determined content-based signature;
executing at least one pending increment of the reference count corresponding to at least one entry in the given set of entries based at least in part on the determination that the first timestamp is earlier than the second timestamp; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after the execution of the at least one pending increment;
wherein the method is implemented by at least one processing device comprising a processor coupled to a memory.
10. The method of claim 9 , wherein the first timestamp of the pending increment data structure represents a time at which the determined content-based signature was added to an entry of the given set of entries of the pending increment data structure.
11. The method of claim 10 , wherein the second timestamp of the pending decrement data structure represents a time at which the determined content-based signature was added to an entry of the pending decrement data structures;
wherein the second timestamp of the pending decrement data structure represents a latest time at which the determined content-based signature was added to an entry of the pending decrement data structure.
12. The method of claim 10 , wherein the first timestamp of the pending increment data structure represents an earliest time at which the determined content-based signature was added to one of the given set of entries in the pending increment data structure.
13. The method of claim 12 , wherein:
the determined content-based signature was added to a given entry of the given set of entries of the pending increment data structure later than the second timestamp;
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries including the given entry; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the at least one pending increment comprises executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the given entry.
14. The method of claim 9 , wherein:
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries; and
the execution of a given pending increment of the plurality of pending increments comprises removing the determined content-based signature from a given entry corresponding to the given pending increment in the pending increment data structure and the second entry comprising the determined content-based signature in the pending decrement data structure without incrementing or decrementing the reference count corresponding to the determined content-based signature.
15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device to:
receive a request to copy user data pages associated with a first range of logical addresses to a second range of logical addresses;
determine a content-based signature corresponding to at least a portion of the first range of logical addresses based at least in part on a mapping of a logical storage layer of a content addressable storage system, the content addressable storage system comprising a plurality of storage devices and an associated storage controller, the storage controller being configured:
to implement the logical storage layer of the content addressable storage system by mapping logical addresses associated with user data pages to content-based signatures respectively derived from a content of those user data pages;
to implement a physical storage layer of the content addressable storage system by mapping the content-based signatures to respective physical blocks in the storage devices where the corresponding user data pages are stored;
to implement a pending increment data structure associated with the storage controller, the pending increment data structure comprising a first plurality of entries, each entry in the pending increment data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which an increment of a corresponding reference count is pending, each corresponding reference count comprising an indication of a number of logical addresses that are mapped to the corresponding content-based signature in the logical storage layer; and
to implement a pending decrement data structure associated with the storage controller, the pending decrement data structure comprising a second plurality of entries, each entry in the pending decrement data structure being configured to store a content-based signature that is mapped in the physical storage layer to a physical block for which a decrement of a corresponding reference count is pending;
map at least a portion of the second range of logical addresses to the determined content-based signature in the logical storage layer;
add the determined content-based signature to a first entry of the pending increment data structure;
determine that the pending decrement data structure comprises a second entry comprising the determined content-based signature;
execute a pending decrement of the reference count corresponding to the determined content-based signature based at least in part on the determination that the pending decrement data structure comprises a second entry comprising the determined content-based signature, the execution of the pending decrement comprising:
determining that a given set of entries of the first plurality of entries of the pending increment data structure each comprise the determined content-based signature;
determining that a first timestamp of the pending increment data structure that corresponds to each entry in the given set of entries is earlier than a second timestamp of the pending decrement data structure that corresponds to an entry of the pending decrement data structure that comprises the determined content-based signature;
executing at least one pending increment of the reference count corresponding to at least one entry in the given set of entries based at least in part on the determination that the first timestamp is earlier than the second timestamp; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after the execution of the at least one pending increment.
16. The computer program product of claim 15 , wherein:
the first timestamp of the pending increment data structure represents a time at which the determined content-based signature was added to an entry of the given set of entries of the pending increment data structure.
17. The computer program product of claim 16 , wherein:
the second timestamp of the pending decrement data structure represents a time at which the determined content-based signature was added to an entry of the pending decrement data structure;
the second timestamp of the pending decrement data structure represents a latest time at which the determined content-based signature was added to an entry of the pending decrement data structure.
18. The computer program product of claim 16 , wherein the first timestamp of the pending increment data structure represents an earliest time at which the determined content-based signature was added to one of the given set of entries in the pending increment data structure.
19. The computer program product of claim 18 , wherein:
the determined content-based signature was added to a given entry of the given set of entries of the pending increment data structure later than the second timestamp;
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries including the given entry; and
executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the at least one pending increment comprises executing the decrement of the reference count corresponding to the second entry in the pending decrement data structure that comprises the determined content-based signature after execution of the given entry.
20. The computer program product of claim 15 , wherein:
executing the at least one pending increment of the reference count corresponding to the at least one entry in the given set of entries comprises executing the pending increments of the reference count corresponding to each entry in the given set of entries; and
the execution of a given pending increment of the plurality of pending increments comprises removing the determined content-based signature from both a given entry corresponding to the given pending increment in the pending increment data structure and the second entry comprising the determined content-based signature in the pending decrement data structure without incrementing or decrementing the reference count corresponding to the determined content-based signature.