Method and apparatus for managing data objects of a data storage system
Techniques for managing data objects of a data storage system are described herein. According to one embodiment, a hash function is generated for a plurality of data objects by analyzing each of the data objects stored in the data storage system. For each of the data objects, a hash operation is performed on the data object using the hash function, generating a hash value. The hash value is associated with a predetermined attribute of the data object, such that the predetermined attribute of the data object is uniquely identified using the hash function subsequently in response to a request for accessing the predetermined attribute of the data object. The data object is then stored at a persistent storage location of the data storage system, wherein the persistent storage location is identifiable based on a hash value obtained from the hash function.
1. A computer-implemented method for managing data objects of a data storage system, the method comprising:
generating identifiers for a plurality of data objects stored in the data storage system based on content of each of the data objects, the identifiers uniquely identifying the data objects, each data object associated with an attribute;
generating a hash function based on the identifiers and a total number of the data objects, the hash function to map the data objects into unique values within a value range determined by the total number of the data objects;
storing an attribute associated with each data object at a location of the data storage system according to the hash function; and
in response to a request for accessing a first attribute associated with a first data object subsequently, locating the first attribute from a first location using the hash function applied to the first data object, wherein the value range allows the locating to be performed quickly.
2. The method of claim 1 , wherein the data storage system is a deduplicating storage system.
3. The method of claim 1 , wherein an identifier of a data object is generated by hashing at least a portion of the content of the data object.
4. The method of claim 1 , wherein the hash function is a minimal hash function that maps a set of N elements to a range of N values, and wherein an identifier of a data object is used as an input to the minimal hash function.
5. The method of claim 1 , wherein the hash function is a perfect minimal hash function.
6. The method of claim 1 , wherein the predetermined attribute of a data object includes a storage location of the data object.
7. The method of claim 1 , wherein the predetermined attribute of a data object includes an access control attribute of the data object.
8. A non-transitory computer-readable medium having instructions stored therein, which when executed by a processor, cause the processor perform operations, the operations comprising:
generating identifiers for a plurality of data objects stored in the data storage system based on content of each of the data objects, the identifiers uniquely identifying the data objects, each data object associated with an attribute;
generating a hash function based on the identifiers and a total number of the data objects, the hash function to map the data objects into unique values within a value range determined by the total number of the data objects;
storing an attribute associated with each data object at a location of the data storage system according to the hash function; and
in response to a request for accessing a first attribute associated with a first data object subsequently, locating the first attribute from a first location using the hash function applied to the first data object, wherein the value range allows the locating to be performed quickly.
9. The non-transitory computer-readable medium of claim 8 , wherein the data storage system is a deduplicating storage system.
10. The non-transitory computer-readable medium of claim 8 , wherein an identifier of a data object is generated by hashing at least a portion of the content of the data object.
11. The non-transitory computer-readable medium of claim 8 , wherein the hash function is a minimal hash function that maps a set of N elements to a range of N values, and wherein an identifier of a data object is used as an input to the minimal hash function.
12. The non-transitory computer-readable medium of claim 8 , wherein the hash function is a perfect minimal hash function.
13. The non-transitory computer-readable medium of claim 8 , wherein the predetermined attribute of a data object includes a storage location of the data object.
14. The non-transitory computer-readable medium of claim 8 , wherein the predetermined attribute of a data object includes an access control attribute of the data object.
15. A data storage system, comprising:
a processor; and
memory coupled to the processor to store instructions, which when executed from the memory, cause the processor to
generate identifiers for a plurality of data objects stored in the data storage system based on content of each of the data objects, the identifiers uniquely identifying the data objects, each data object associated with an attribute;
generate a hash function based on the identifiers and a total number of the data objects, the hash function to map the data objects into unique values within a value range determined by the total number of the data objects;
store an attribute associated with each data object at a location of the data storage system according to the hash function, and
in response to a request for accessing a first attribute associated with a first data object subsequently, locate the first attribute from a first location using the hash function applied to the first data object, wherein the value range allows the locating to be performed quickly.
16. The system of claim 15 , wherein the data storage system is a deduplicating storage system.
17. The system of claim 15 , wherein an identifier of a data object is generated by hashing at least a portion of the content of the data object.
18. The system of claim 15 , wherein the hash function is a minimal hash function that maps a set of N elements to a range of N values, and wherein an identifier of a data object is used as an input to the minimal hash function.
19. The system of claim 15 , wherein the hash function is a perfect minimal hash function.
20. The system of claim 15 , wherein the predetermined attribute of a data object includes a storage location of the data object.
21. The system of claim 15 , wherein the predetermined attribute of a data object includes an access control attribute of the data object.