Media aware distributed data layout

1. A storage system, comprising:

a computer;

a plurality of vdisks in communication with the computer, each vdisk containing a plurality of persistent storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks; and

a policy module for vdisk and segment choice in communication with the computer, a first portion of a file's data is stored in a first segment, and a second portion of the file's data is stored in either the first segment or a second segment, the predetermined mapping is a round robin assignment algorithm.

2. A storage system as described in claim 1 , where the predetermined: mapping cheeses the vdisk with a largest available space, or a largest percentage of available space.

3. A storage system, as described in claim 1 , including a plurality of processors and wherein, the predetermined mapping chooses the vdisk served by a least loaded processor.

4. A storage system as described in claim 1 , wherein each vdisk contains an arbitrary collection of directories and files, and at least two vdisks hold at least one file.

5. A storage system as described in claim 1 , wherein at least one file's data is stored in one segment, and at least one other file's data is stored in a second segment.

6. A storage system as described in claim 5 , where the predetermined mapping chooses the vdisk with a largest available space, or a largest percentage of available space.

7. A storage system as described in claim 5 , including a plurality of processors and wherein the predetermined mapping chooses the vdisk served by a least loaded processor.

8. A storage system as described in claim 1 , wherein, each vdisk is an arbitrary collection of directories and files, and at least one file is stored in at least two segments.

9. A storage system as described in claim 5 , wherein each vdisk contains an arbitrary collection of directories and files, and at least two vdisks hold at least one file.

10. A system as described in claim 1 wherein each segment consists of a plurality of chunks.

11. A method for storing a file comprising the steps of:

receiving the file at a communication interface;

storing a first portion of the file in a first segment of a persistent memory in communication with the interface and a second portion of the file in a second segment of the memory having a different class of service for storage than the first segment's class of service; and

retrieving by a computer the file by reading the first portion and the second portion from the first and second segment, respectively, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks, a first portion of a file's data is stored in a first segment, and a second portion of the file's data is stored in either the first segment or a second segment, each vdisk contains meta data disk block pointers only to blocks within a same vdisk.

12. A storage system comprising:

a computer;

a plurality of vdisks in communication with the computer, each vdisk containing a plurality of persistent storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments, each vdisk storing one or more files with data and meta data distributed among its storage segments and a persistent operations table that implements directory modifying operations atomically, each vdisk is an arbitrary collection of directories and files, and at least one file is stored in at least two segments, a first portion of a file's data is stored in a first segment, and a second portion of the file's data is stored in either the first segment or a second segment, each vdisk contains meta data disk block pointers only to blocks within a same vdisk.

13. A storage system comprising:

a plurality of vdisks, each vdisk containing a plurality of storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, each segment having disk blocks, the disk blocks being allocated to files and being marked as in use by a corresponding bit being set in the bitmap allocation table, indexed by physical block address; and

a persistent operations table which tracks file system operations that affect a single vdisk, a first portion of a file's data is stored in a first segment, and a second portion of the file's data is stored in either the first segment or a second segment, each vdisk contains meta data disk block pointers only to blocks within a same vdisk.

14. A storage system, comprising:

a computer;

a plurality of vdisks in communication with the computer, each vdisk containing a plurality of persistent storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks; and

a policy module for vdisk and segment choice in communication with the computer, a first portion of a file's data is stored in a first segment, and a second portion of the file's data is stored in either the first segment or a second segment, each vdisk contains meta data disk block pointers only to blocks within a same vdisk.

15. A storage system as described in claim 14 , including an interface to initiate a file system consistency check on an individual vdisk, triggered by an indication of an inconsistency in a specific vdisk.

16. A storage system comprising:

a computer;

a plurality of vdisks in communication with the computer, each vdisk containing a plurality of persistent storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks; and

a policy module for vdisk and segment choice in communication with the computer, at least one file's data is stored in one segment, and at least one other file's data is stored in a second segment, the predetermined mapping is a round robin assignment algorithm.

17. A storage system comprising:

a computer;

a plurality of vdisks in communication with the computer, each vdisk containing a plurality of persistent storage segments, each segment providing a specific class of service (CoS) for storage different from the CoS for storage of the other segments; each vdisk storing one or more files with data and meta data distributed among its storage segments, a vdisk to hold a newly created file or directory is chosen from the plurality of vdisks based on a predetermined mapping into the plurality of vdisks; and

a policy module for vdisk and segment choice in communication with the computer, at least one file's data is stored in one segment, and at least one other file's data is stored in a second segment, each vdisk contains meta data disk block pointers only to blocks within the same vdisk.

18. A storage system as described in claim 17 , including an interface to initiate a file system consistency check on an individual vdisk, triggered by an indication of an inconsistency in a specific vdisk.