Media aware distributed data layout
A storage system comprises a plurality of vdisks, with each vdisk containing a plurality of storage segments, and each segment providing a specific class of service (CoS). Each vdisk stores files with data and meta data distributed among its storage segments. A storage system comprises a memory having multiple classes of service. The system comprises an interface for storing a file as blocks of data associated with a class of service in the memory. The interface chooses the class of service for a block on a block by block basis. A file system for storing a file comprises a plurality of vdisks, with each vdisk having a plurality of inodes. Each inode of each vdisk stores data on one or more segments, with each segment having a different class of service. The system comprises a controller which stores data of a file in an inode of a vdisk, in one or more segments of that vdisk. A file system for storing a file comprises a plurality of vdisks, and each vdisk having a plurality of inodes. The system comprises a controller including a plurality of processors, with each processor serving one or more of the vdisks. A file system for storing comprises a plurality of vdisks, with each vdisk having a plurality of inodes, a plurality of inodes of at least one vdisk storing data on a plurality of segments, each segment having a different class of service. The system comprises a controller which stores data of the file in one or more segments of one vdisk. A method for storing a file.
1. A storage system comprising:
a processor coupled to a memory;
a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (COS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, each file's meta data including a search tree indexed by logical block number, each file described by a single inode, a first portion of a file's data is stored in a first segment, and a second portion of the file is stored in a second segment, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks.
2. A storage system as described in claim 1 , where the predetermined mapping is a round robin assignment algorithm.
3. A storage system as described in claim 1 , where the predetermined mapping chooses the vdisk with a largest available space, or a largest percentage of available space.
4. A storage system as described in claim 1 , including a plurality of processors and wherein the predetermined mapping chooses the vdisk served by a least loaded processor.
5. A storage system as described in claim 1 , wherein each vdisk contains an arbitrary collection of directories and files, and at least two vdisks hold at least one file.
6. A storage system as described in claim 1 , wherein each vdisk contains meta data disk block pointers only to blocks within a same vdisk.
7. A storage system as described in claim 6 , including an interface to initiate a file system consistency check on an individual vdisk, triggered by an indication of an inconsistency in a specific vdisk.
8. A storage system as described in claim 1 , wherein at least one file's data is stored in one segment, and at least one other file's data is stored in a second segment.
9. A storage system as described in claim 8 , including a plurality of vdisks, and where a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks.
10. A storage system as described in claim 9 , where the predetermined mapping is a round robin assignment algorithm.
11. A storage system as described in claim 9 , where the predetermined mapping chooses the vdisk with a largest available space, or a largest percentage of available space.
12. A storage system as described in claim 9 , including a plurality of processors and wherein the predetermined mapping chooses the vdisk served by a least loaded processor.
13. A storage system as described in claim 1 , wherein each vdisk is an arbitrary collection of directories and files, and at least one file is stored in at least two segments.
14. A storage system as described in claim 8 , wherein each vdisk contains an arbitrary collection of directories and files, and at least two vdisks hold at least one file.
15. A storage system as described in claim 8 , wherein each vdisk contains meta data disk block pointers only to blocks within the same vdisk.
16. A storage system as described in claim 15 , including an interface to initiate a file system consistency check on an individual vdisk, triggered by an indication of an inconsistency in a specific vdisk.
17. A system as described in claim 1 , wherein each segment consists of a plurality of chunks.
18. A method for storing a file comprising the steps of:
receiving the file at an interface;
storing a first portion of the file in a first segment of a memory in communication with the interface and a second portion of the file in a second segment of the memory having a different class of service for storage than the first segment's class of service; and
retrieving the file by reading the first portion and the second portion from the first and second segment, respectively, each file's meta data including a search tree indexed by logical block number, each file described by a single inode, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks.
19. A storage system comprising:
a processor coupled to a memory;
a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (COS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments and a persistent operations table that implements directory modifying operations atomically, the meta data in at least one segment describes file data stored in any other segment, each file described by a single inode, a first portion of a file's data is stored in a first segment, and a second portion of the file is stored in a second segment, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks.
20. A storage system comprising:
a processor coupled to a memory;
a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (COS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, each segment having disk blocks, the disk blocks being allocated to files and being marked as in use by a corresponding bit being set in the bitmap allocation table, indexed by physical block address, each file described by a single inode, a first portion of a file's data is stored in a first segment, and a second portion of the file is stored in a second segment, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks.
21. A storage system comprising:
a processor coupled to a memory;
a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (COS) different from the CoS of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, a first portion of a file's data is stored in a first segment, and a second portion of the file is stored in either the first segment or a second segment, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks, where the predetermined mapping is a round robin assignment algorithm, each file's meta data including a search tree indexed by logical block number, each file described by a single node.
22. A storage system comprising:
a processor coupled to a memory;
a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (COS) different from the CoS of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, at least one file's data is stored in one segment, and at least one other file's data is stored in a second segment, including a plurality of vdisks, and where a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks, where the predetermined mapping is a round robin assignment algorithm, each file's meta data including a search tree indexed by logical block number, each file described by a single node.