Sparse file access

1. A method comprising:

a storage system receiving a storage command to access a sparse file;

assembling a combined content of a set of materialized blocks and a header that identifies a plurality of fixed-size non-materialized blocks, wherein:

the combined content does not comprise a content of the plurality of fixed-size non-materialized blocks, and

content of the plurality of fixed-size non-materialized blocks occupy no physical capacity of said storage system;

transferring, responsive to the assembling, the combined content, including respective identifiers of said plurality of fixed-size non-materialized blocks, from the storage system to a computer system.

2. The method of claim 1 further comprising, responsive to the transferring, the computer system processing the header to generate a sparse pattern, wherein the storage command comprises a read command, wherein the transferring comprises transferring the combined content from the storage system to the computer system, wherein the assembling is responsive to the receiving.

3. The method of claim 1 wherein the storage command comprises a write command, wherein the transferring comprises transferring the combined content from the computer system to the storage system, wherein the write command comprises the combined content, wherein the receiving is responsive to the assembling.

4. The method of claim 1 wherein receiving the storage command comprises receiving a first write command and a second write command, wherein the method further comprises:

the storage system notifying the computer system that the first write command failed due to insufficient physical space;

if executing the second write command does not require an additional physical space, then successfully executing the second write command after the notifying.

5. The method of claim 1 wherein the storage command is a request to perform one of: a write or a format, wherein the sparse file is thin provisioned in excess of available physical capacity of said storage system, wherein the storage command does not cause a block to become materialized.

6. The method of claim 1 further comprising, responsive to the receiving, processing storage metadata that maps virtual block identifiers, each of which identifies a block across multiple storage devices, to logical block identifiers, wherein the storage metadata resides only in volatile memory.

7. The method of claim 6 wherein the storage metadata is generated from durable metadata that maps logical blocks to virtual blocks.

8. The method of claim 7 wherein the durable metadata resides in durable storage during a reboot of the storage system.

9. The method of claim 6 wherein the storage metadata resides in a hash table comprising a plurality of buckets, wherein each bucket of the plurality of buckets comprises a segmented array.

10. One or more non-transitory computer readable media comprising instructions that when executed by one or more processors cause:

a storage system receiving a storage command to access a sparse file;

assembling a combined content of a set of materialized blocks and a header that identifies a plurality of fixed-size non-materialized blocks, wherein:

the combined content does not comprise a content of the plurality of fixed-size non-materialized blocks, and

content of the plurality of fixed-size non-materialized blocks occupy no physical capacity of said storage system;

transferring, responsive to the assembling, the combined content, including respective identifiers of said plurality of fixed-size non-materialized blocks, from the storage system to a computer system.

11. The one or more non-transitory computer readable media of claim 10 wherein the instructions further cause, responsive to the transferring, the computer system processing the header to generate a sparse pattern, wherein the storage command comprises a read command, wherein the transferring comprises transferring the combined content from the storage system to the computer system, wherein the assembling is responsive to the receiving.

12. The one or more non-transitory computer readable media of claim 10 wherein the storage command comprises a write command, wherein the transferring comprises transferring the combined content from the computer system to the storage system, wherein the write command comprises the combined content, wherein the receiving is responsive to the assembling.

13. The one or more non-transitory computer readable media of claim 10 wherein receiving the storage command comprises receiving a first write command and a second write command, wherein the instructions further cause:

the storage system notifying the computer system that the first write command failed due to insufficient physical space;

if executing the second write command does not require an additional physical space, then successfully executing the second write command after the notifying.

14. The one or more non-transitory computer readable media of claim 10 wherein the storage command is a request to perform one of: a write or a format, wherein the sparse file is thin provisioned in excess of available physical capacity of said storage system, wherein the storage command does not cause a block to become materialized.

15. The one or more non-transitory computer readable media of claim 10 wherein the instructions further cause, responsive to the receiving, processing storage metadata that maps virtual block identifiers, each of which identifies a block across multiple storage devices, to logical block identifiers, wherein the storage metadata resides only in volatile memory.

16. The one or more non-transitory computer readable media of claim 15 wherein the storage metadata is generated from durable metadata that maps logical blocks to virtual blocks.

17. The one or more non-transitory computer readable media of claim 16 wherein the durable metadata resides in durable storage during a reboot of the storage system.

18. The one or more non-transitory computer readable media of claim 15 wherein the storage metadata resides in a hash table comprising a plurality of buckets, wherein each bucket of the plurality of buckets comprises a segmented array.

19. A storage system comprising:

one or more block-oriented storage devices configured to store one or more sparse files,

wherein each of the one or more sparse files is composed of a set of materialized blocks and a set of fixed-size non-materialized blocks;

connected to the one or more block-oriented storage devices, a processor configured to:

receive a read command to retrieve a plurality of fixed-size non-materialized blocks and a set of materialized blocks from one of the one or more sparse files;

assemble a combined content of the set of materialized blocks and a header that identifies the plurality of fixed-size non-materialized blocks, wherein:

the combined content does not comprise a content of the plurality of fixed-size non-materialized blocks, and

content of the plurality of fixed-size non-materialized blocks occupy no physical capacity of said storage system;

transfer, responsive to the assembling, the combined content, including respective identifiers of said plurality of fixed-size non-materialized blocks, to a computer system.

20. The storage system of claim 19 wherein the processor is further configured to:

receive a write command to store a combined content of a set of materialized blocks and a header that identifies one or more fixed-size non-materialized blocks, wherein the combined content does not comprise a content of the one or more fixed-size non-materialized blocks;

store, responsive to the receiving the write command and into the one or more block-oriented storage devices, the set of materialized blocks and metadata that describes the one or more fixed-size non-materialized blocks.