Direct data placement
Aspects of disclosure provide methods, systems, and computer-readable media for direct data displacement. The methods include in view of a write request pertaining to a first object, determining, based on a first signature of the first object, whether an index of a storage system comprises an existing object record pertaining to the first object; and in response to determining that the index of the storage system does not include an existing object record pertaining to the first signature, creating a first object record in the index comprising the first signature and a reference count. The methods further include in response to determining that the index of the storage system includes an existing object record pertaining to the first signature, updating a reference count associated with the first object stored in the existing object.
1. A method, comprising:
n view of a write request pertaining to a first object, determining, based on a first signature of the first object, whether an index of a storage system comprises an existing object record pertaining to the first object, wherein each of a plurality of entries of the index corresponds to a respective portion of a storage pool comprising a plurality of storage devices; and
in response to determining that the index of the storage system does not include an existing object record pertaining to the first signature, creating a first object record in the index, wherein the first object record comprises the first signature and a reference count, wherein two or more of the plurality of entries of the index correspond to a plurality of hashes of the first signature, and wherein creating the first object record comprises:
placing the first signature of the first object in a first entry of the plurality of the entries of the index corresponding to a first hash of the first signature;
in response to determining that the index of the storage system includes an existing object record pertaining to the first signature, updating, by a processor, a reference count associated with the first object, wherein the exiting object record pertaining to the first signature comprises the reference count associated with the first object.
2. The method of claim 1 , wherein creating the first object record comprises:
displacing a second object record stored in the first entry of the index, comprising:
generating a second hash for a second signature associated with a second object using a second hash function, wherein the second object record comprises the second signature; and
inserting the second signature in a second entry of the index corresponding to the second hash.
3. The method of claim 1 , wherein determining, based on the first signature of the first object, whether the index of the storage system comprises the existing object record pertaining to the first object comprises:
looking up a prefix of the first signature in prefix table of the storage system.
4. The method of claim 1 , wherein determining, based on the first signature of the first object, whether the index of the storage system comprises an existing object record pertaining to the first object comprises:
generating the plurality of hashes by processing the first signature using a plurality of hash functions in a predetermined order.
5. The method of claim 1 , wherein updating the reference count comprises incrementing the reference count associated with the first object.
6. The method of claim 1 , wherein the first signature of the first object is a hash of at least a portion of content of the first object.
7. A system comprising:
a memory; and
a processor operatively coupled to the memory, the processor to:
in view of a write request pertaining to a first object, determine, based on a first signature of the first object, whether an index of a storage system comprises an existing object record pertaining to the first object, wherein each of a plurality of entries of the index corresponds to a respective portion of a storage pool comprising a plurality of storage devices; and
in response to determining that the index of the storage system does not include an existing object record pertaining to the first signature, create a first object record in the index comprising the first signature and a reference count, wherein two or more of the plurality of entries of the index correspond to a plurality of hashes of the first signature, and wherein, to create the first object record, the processor is further to:
store the first signature of the first object in a first entry of the plurality of the entries of the index corresponding to a first hash of the first signature;
wherein the processor is further to: in response to determining that the index of the storage system includes an existing object record pertaining to the first signature, update a reference count associated with the first object, wherein the exiting object record pertaining to the first signature comprises the reference count associated with the first object.
8. The system of claim 7 , wherein, to create the first object record, the processor is further to:
generate a second hash for a second signature associated with a second object using a second hash function, wherein the second object record comprises the second signature; and
insert the second signature in a second entry of the index corresponding to the second hash.
9. The system of claim 7 , wherein, to determine, based on the first signature of the first object, whether the index of the storage system comprises an existing object record pertaining to the first object, the processor is further to:
look up a prefix of the first signature in prefix table of the storage system.
10. The system of claim 7 , wherein, to determine, based on the first signature of the first object, whether the index of the storage system comprises the existing object record pertaining to the first object, the processor is further to:
generate the plurality of hashes by processing the first signature using a plurality of hash functions in a predetermined order.
11. The system of claim 7 , wherein, to update the reference count, the processor is further to increment the reference count associated with the first object.
12. The system of claim 11 , wherein the signature of the first object is a hash of at least a portion of content of the first object.
13. A non-transitory machine-readable storage medium including instructions that, when accessed by a processor, cause the processor to:
in view of a write request pertaining to a first object, determine, based on a first signature of the first object, whether an index of a storage system comprises an existing object record pertaining to the first object, wherein each of a plurality of entries of the index corresponds to a respective portion of a storage pool comprising a plurality of storage devices; and
in response to determining that the index of the storage system does not include an existing object record pertaining to the first signature, create a first object record in the index comprising the first signature and a reference count, wherein two or more of the plurality of entries of the index correspond to a plurality of hashes of the first signature, and wherein, to create the first object record, the processor is further to:
store the first signature of the first object in a first entry of the plurality of the entries of the index corresponding to a first hash of the first signature, wherein the first signature of the first object is a hash of at least a portion of content of the first object;
in response to determining that the index of the storage system includes an existing object record pertaining to the first signature, update a reference count associated with the first object, wherein the exiting object record pertaining to the first signature comprises the reference count.
14. The non-transitory machine-readable storage medium of claim 13 , wherein, to create the first object record, the processor is further to:
generate a second hash for a second signature associated with a second object using a second hash function, wherein the second object record comprises the second signature; and
insert the second signature in a second entry of the index corresponding to the second hash.
15. The non-transitory machine-readable storage medium of claim 13 , wherein, to determine, based on the first signature of the first object, whether the index of the storage system comprises an existing object record pertaining to the first object, the processor is further to:
look up a prefix of the first signature in prefix table of the storage system.
16. The non-transitory machine-readable storage medium of claim 13 , wherein, to determine, based on the first signature of the first object, whether the index of the storage system comprises an existing object record pertaining to the first object, the processor is further to:
generate the plurality of hashes by processing the first signature using a plurality of hash functions in a predetermined order.
17. The non-transitory machine-readable storage medium of claim 13 , wherein updating the reference count comprises incrementing the reference count associated with the first object.