Media aware distributed data layout
A storage system comprises a plurality of vdisks, with each vdisk containing a plurality of storage segments, and each segment providing a specific class of service (CoS). Each vdisk stores files with data and meta data distributed among its storage segments. A storage system includes a memory having multiple classes of service. The system includes an interface for storing a file as blocks of data associated with a class of service in the memory. The interface chooses the class of service for a block on a block by block basis.
1. A file system for storing a file comprising:
a plurality of vdisks, each vdisk having a plurality of non-transient storage segments, each segment providing a specific class of service (CoS) for storage, each vdisk having a plurality of inodes stored in an inode table, each inode describing a file in the vdisk, each inode of each vdisk storing data on one or more of a plurality of segments, each segment having a different class of service for storage where a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks in the storage system, the vdisk chosen to hold the newly created file or directory differs from the vdisk holding the newly created object's parent directory; and
a controller having at least one physical processor which stores data of a file in an inode of a vdisk, in one or more arbitrary segments of that vdisk.
2. A system as described in claim 1 where the controller includes a plurality of physical processors, each processor serving one or more of the vdisks.
3. A system as described in claim 1 including at least one directory storing mappings of file names to (vdisk, inode) pairs.
4. A system as described in claim 3 wherein each inode has one or more back pointers to all of the directory entries pointing to the inode.
5. A system as described in claim 4 wherein one of the vdisks is a root vdisk.
6. A system as described in claim 5 wherein one of the inodes in the root vdisk is a root inode of the file system.
7. A system as described in claim 6 wherein each segment has a plurality of chunks.
8. A system as described in claim 7 , where the predetermined mapping is a round robin assignment algorithm.
9. A system as described in claim 7 , where the predetermined mapping chooses the vdisk with a largest available space, or a largest percentage of available space.
10. A system as described in claim 7 , including a plurality of processors, either virtual or physical, and wherein the predetermined mapping chooses the vdisk served by a least loaded processor.
11. A system as described in claim 7 , wherein each vdisk is an arbitrary collection of directories and files, and at least one file is stored in at least two segments.
12. A system as described in claim 7 , wherein each vdisk is an arbitrary collection of directories and files, and at least two vdisks hold at least one file.
13. A storage system as described in claim 7 , wherein each vdisk contains meta data disk block pointers only to blocks within the same vdisk.
14. A system as described in claim 13 , including an interface to initiate a file system consistency check on an individual vdisk, triggered by an indication of an inconsistency in a specific vdisk.
15. A system as described in claim 1 , including:
a controller including a plurality of processors, each processor serving one or more of the vdisks.
16. A system as described in claim 15 , wherein each segment has a plurality of chunks.
17. A system as described in claim 16 , wherein each segment has chunks added to it dynamically over time.
18. A method for storing a file comprising the steps of:
receiving the file at an interface; and
storing data of the file in an inode of a vdisk with a controller in multiple segments of one vdisk of a plurality of vdisks arbitrarily, each vdisk having a plurality of inodes, at least two of the inodes of one vdisk storing data in different segments, each segment having a different class of service for storage where a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks in the storage system, the vdisk chosen to hold the newly created file or directory differs from the vdisk holding the newly created object's parent directory.