Metadata management for virtual volumes
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, manage metadata for virtual volumes. In some implementations, a method and system include defining multiple metadata blocks in a persistent storage, including information that links a virtual address space to the storage system, where the defining includes, for at least one of the multiple metadata blocks, determining multiple output addresses corresponding to the storage system, and writing the multiple output addresses and an identifier corresponding to the multiple metadata blocks in a metadata block in the persistent storage. In some implementations, a method and system include reading the multiple metadata blocks into the memory from the persistent storage, including identifying the metadata block based on the identifier; receiving an input address of the virtual address space; and obtaining a corresponding output address to the storage system using the multiple metadata blocks in the memory.
1. A system comprising: a processor; a memory coupled with the processor; a storage system coupled with the memory; and a computer-readable medium embodying a program operable to cause the processor to perform operations comprising: defining multiple metadata blocks in a persistent storage, the multiple metadata blocks comprising information that links a virtual address space to the storage system, where the defining includes, for each one of the multiple metadata blocks, determining multiple output addresses corresponding to the storage system, and writing the multiple output addresses, which correspond to the one metadata block, and an identifier corresponding to the multiple metadata blocks, in the one metadata block in the persistent storage; where the memory comprises a hash table data structure to access and manage multiple sets of the metadata blocks corresponding to the multiple virtual volumes; and where the hash table data structure comprises multiple locations, a location including a field that saves a persistent address for a given metadata block, a field that saves an in-memory pointer to an in-memory copy of the given metadata block, and a field that saves least recently used (LRU) information.
2. The system of claim 1 , the operations comprising:
reading the multiple metadata blocks into the memory from the persistent storage, the reading comprising identifying each of the multiple metadata blocks based on the identifier corresponding to the multiple metadata blocks;
receiving an input address of the virtual address space; and
obtaining an output address corresponding to the storage system for the input address using the multiple metadata blocks in the memory.
3. The system of claim 2 , where the storage system comprises a RAID system comprising the persistent storage.
4. The system of claim 2 , where the writing comprises writing a virtual volume identifier for the metadata block, and the defining comprises writing the multiple metadata blocks to a defined area of the persistent storage without writing to the persistent storage a data structure linking the multiple metadata blocks together.
5. The system of claim 4 , where the reading comprises performing a single, sequential read from the defined area of the persistent storage.
6. A method comprising: defining multiple metadata blocks in a persistent storage, the multiple metadata blocks comprising information that links a virtual address space to a storage system, where the defining includes, for each one of the multiple metadata blocks, determining multiple output addresses corresponding to the storage system, and writing the multiple output addresses, which correspond to the one metadata block, and an identifier corresponding to the multiple metadata blocks, in the one metadata block in the persistent storage; reading the multiple metadata blocks into a memory from the persistent storage, the reading comprising identifying each of the multiple metadata blocks based on the identifier corresponding to the multiple metadata blocks; receiving an input address of the virtual address space; and obtaining an output address corresponding to the storage system for the input address using the multiple metadata blocks in the memory; where the reading comprises: generating hash values from information corresponding to each of the multiple metadata blocks; and saving each of the multiple metadata blocks in a hash table data structure in accordance with the hash values; where the hash table data structure comprises multiple locations, a location including a field that saves a persistent address for a given metadata block, a field that saves an in-memory pointer to an in-memory copy of the given metadata block, and a field that saves least recently used (LRU) information; and the obtaining comprises updating the LRU information.
7. The method of claim 6 , where the storage system comprises the persistent storage, and reading the multiple metadata blocks comprises reading the multiple metadata blocks into the memory from the storage system.
8. The method of claim 6 , where the writing comprises writing a virtual volume identifier for the metadata block, and the defining comprises writing the multiple metadata blocks to a defined area of the persistent storage without writing to the persistent storage a data structure linking the multiple metadata blocks together.
9. The method of claim 8 , where the reading comprises performing a single, sequential read from the defined area of the persistent storage.
10. The method of claim 6 , comprising saving multiple sets of metadata blocks corresponding to multiple virtual volumes in the hash table data structure.
11. A system comprising: a processor; a memory coupled with the processor; a storage system coupled with the memory; and a computer-readable medium embodying a program operable to cause the processor to perform operations comprising: reading multiple metadata blocks into the memory from a persistent storage, the reading comprising identifying each respective metadata block of the multiple metadata blocks based on an identifier included in the metadata block along with multiple output addresses corresponding to the storage system to link a virtual address space to the storage system, the identifier corresponding to the multiple metadata blocks, receiving an input address of the virtual address space, and obtaining an output address corresponding to the storage system for the input address using the multiple metadata blocks in the memory; where the reading comprises: generating hash values from information corresponding to each of the multiple metadata blocks, and saving each of the multiple metadata blocks in a hash table data structure in accordance with the hash values; where the hash table data structure comprises multiple locations, a location including a field that saves a persistent address for a given metadata block, a field that saves an in-memory pointer to an in-memory copy of the given metadata block, and a field that saves least recently used (LRU) information; and the obtaining comprises updating the LRU information.
12. The system of claim 11 , where the storage system comprises the persistent storage, and reading the multiple metadata blocks comprises reading the multiple metadata blocks into the memory from the storage system.
13. The system of claim 11 , where the metadata block includes a virtual volume identifier, and the multiple metadata blocks are located in a defined area of the persistent storage, which doesn't include a data structure linking the multiple metadata blocks together.
14. The system of claim 13 , where the reading comprises performing a single, sequential read from the defined area of the persistent storage.
15. The system of claim 11 , where the hash table data structure includes multiple sets of metadata blocks corresponding to multiple virtual volumes.