Method and apparatus for differentiated data placement
Method and apparatus for locating data on disk storage, wherein multiple instances of data can be stored at different locations to satisfy different use requirements such as read access, write access, and data security. The method allows a data storage system, such as a file system, to provide both read optimized and write optimized performance on disk storage of different types (e.g., sizes and speed).
1. A storage system comprising:
an interface component of a file-based storage system or a block storage manager for placing, by one or more processors, data for storage at physical locations on disk storage, wherein the interface component references each data element in the data by a globally unique identifier (GUID) derived from the content of the data element such that the GUID must change when content of the data element changes, and wherein the GUID does not change based on where multiple instances of the data element are stored at different physical locations on the disk storage;
a mapping index, accessible by the one or more processors, that stores, for a single GUID, multiple pointers to different physical locations on the disk storage for multiple instances of the data element; and
a location strategizer for dynamically determining, by the one or more processors, the different physical locations on the disk storage for storing the multiple instances of the data element as the data element is referenced by the interface component during storage system activity, and wherein the location strategizer stores the multiple instances of the data element within different data sequences at the different locations on the disk storage for read optimization and write optimization, the different data sequences having different orders of the stored data elements.
2. The storage system of claim 1 , wherein the interface component comprises a file-based storage system.
3. The storage system of claim 1 , wherein the interface component comprises a block storage manager.
4. The storage system of claim 1 , wherein the location strategizer places multilple instances for data security.
5. The storage system of claim 1 , wherein the location strategizer dynamically determines the different locations as the data is referenced by the interface component based on anticipated use of the data, condition of disks, or as other system parameters change over time.
6. The storage system of claim 1 , wherein the mapping index is implemented by programmable logic.
7. The storage system of claim 1 , wherein the mapping index is implemented by executable computer program instructions.
8. A non-transitory computer medium containing executable program instructions for executing a method comprising:
referencing each data element by a globally unique identifier (GUID) derived from the content of the data element such that the GUID must change when content of the data element changes, and wherein the GUID does not change based on where multiple instances of the data element are stored at different physical locations on a disk storage;
maintaining a mapping index that stores, for a single GUID, multiple pointers to different physical locations on the disk storage for storing multiple instances of the data element;
dynamically determining, as the data element is referenced during storage system activity, the physical locations for storing the multiple instances of the data element at the different locations on the disk storage to benefit I/O profiles for both read access and write access; and
storing the multiple instances of the data element at the determined different locations within different data sequences at the different locations on the disk storage for read optimization and write optimization, the different data sequences having different orders of the stored data elements.
9. The non-transitory computer medium of claim 8 , wherein the method includes placing multiple instances of data elements, in the different data sequences, for data security.
10. The non-transitory computer medium of claim 8 , wherein the method includes placing multiple instances on different disks for read optimization and write optimization.
11. The non-transitory computer medium of claim 8 , wherein the method includes placing multiple instances on different disks for read optimization, write optimization and data security.
12. The non-transitory computer medium of claim 8 , wherein the index includes a data structure for each data element containing pointers to physical block addresses where the multiple instances of the data element are stored on the disk storage.
13. The non-transitory computer medium of claim 8 , wherein the GUID comprises a hash of the content of the data element.
14. The non-transitory computer medium of claim 8 , wherein the GUID comprises a cryptographic hash or collision resistant hash of the content of the data element.
15. The non-transitory computer medium of claim 8 , wherein the index maps to physical locations on a plurality of disks.
16. The non-transitory computer medium of claim 15 , wherein the disks are of different sizes and/or access speeds.
17. The non-transitory computer medium of claim 8 , wherein:
the data elements comprises data and/or metadata, and collections of data elements have their own GUID derived from the contents of the collection such that a change to one or more data elements of the collection changes the collection GUID.
18. The non-transitory computer medium of claim 8 , wherein the disk storage includes multiple disks and the method includes placing multiple instances of the data element for read optimization and write optimization on the multiple disks.
19. The non-transitory computer medium of claim 18 , wherein the disks are of different sizes and/or access speeds.