Systems and methods for rebuilding a cache index
Systems and methods for rebuilding an index for a flash cache are provided. The index is rebuilt by reading headers of containers stored in the cache and inserting information from the headers into the index. The index is enabled while being rebuild such that lookup operations can be performed using the index even when the index is incomplete. New containers can be inserted into used or unused regions of the cache while the index is being rebuilt.
1. A method for operating a flash memory cache while rebuilding an index for the flash memory cache, the method comprising:
rebuilding the index by:
reading headers of containers stored in the flash memory cache, each of the containers storing segments, wherein the headers contain sufficient information to index the segments stored in the containers without accessing the segments stored in the containers; and
updating the index with information extracted from the headers of the containers, wherein the information includes identifiers of the segments stored in the containers and locations of the segments stored in the containers; and
adding a new segment to the flash memory cache in parallel with updating the index.
2. The method of claim 1 , further comprising adding information including an identifier for the new segment and a location of the new segment in a container to the index.
3. The method of claim 1 , further comprising operating the flash memory cache in a normal manner with a partially rebuilt index, wherein the normal manner includes at least evicting segments or containers from the flash memory cache using the partially rebuilt index and performing lookup operations using the partially rebuilt index.
4. The method of claim 1 , further comprising adding the new segment to a portion of the flash memory cache that has not been used or that does not currently store any containers or segments.
5. The method of claim 1 , further comprising adding the new segment to a portion of the flash memory cache that has been used but not yet indexed while rebuilding the index.
6. The method of claim 1 , further comprising adding the new segment to the flash memory cache in accordance with first policies while the index is rebuilt and operating the flash memory cache in accordance with normal polices after the index is rebuilt, wherein the first policies relate to selecting a used region of the flash memory cache in which to add the new segment, wherein the first policies include at least one of least recently used or lowest container number.
7. The method of claim 1 , further comprising performing lookup operations into the flash memory cache while rebuilding the index.
8. The method of claim 1 , further comprising reading the flash memory cache with multiple threads, wherein each thread is assigned a range of containers to read.
9. The method of claim 1 , further comprising storing a cache state in a reserved region of the flash memory cache.
10. The method of claim 9 , wherein the cache state identifies a highest container identifier stored in the flash memory cache and identifies which regions of the flash memory cache are unused, wherein the highest container identifier ensures that all containers are read when rebuilding the index.
11. The method of claim 1 , further comprising adding a new container to the flash memory cache while rebuilding the index, wherein the new container is added to an unused region of the flash memory cache or wherein the new container replaces an existing container in a used region of the flash memory cache that has not been indexed, wherein a header of the new container in the used region is not read during the rebuilding process.
12. A computing system configured to rebuild an index into flash memory, the computing system comprising:
a flash memory configured to cache data segments in the flash memory;
a processor configured to execute computer executable instructions for performing a method for rebuilding the index, the method comprising:
rebuilding the index into the flash memory, wherein the index associates identifiers of the data segments with locations of the data segments in the flash memory, by:
reading headers of containers stored in the flash memory, each of the containers stored in the flash memory and storing at least one of the data segments in the flash memory, wherein the headers contain sufficient information to index the data segments stored in the containers stored in the flash memory without accessing the data segments stored in the containers stored in the flash memory; and
updating the index with information extracted from the headers of the containers, wherein the information includes identifiers of the data segments stored in the containers and locations of the data segments in the containers; and
adding a new data segment to the flash memory in parallel with updating the index.
13. The computing system of claim 12 , further comprising adding information including an identifier for the new data segment and a location of the new data segment within one of the containers stored in the flash memory to the index.
14. The computing system of claim 12 , further comprising a memory that is separate from the flash memory, wherein the index is stored in the memory, in the flash memory, or in both the memory and the flash memory.
15. The computing system of claim 12 , further comprising adding the new data segment to a new container that is added to a portion of the flash memory that has not been used.
16. The computing system of claim 12 , further comprising adding the new data segment to a portion of the flash memory that has been used but not yet indexed while rebuilding the index, wherein the new data segment being added is added to a new container stored in the flash memory in accordance with first policies while the index is being rebuilt and operating the flash memory in accordance with normal policies after the index is rebuilt.
17. The computing system of claim 12 , further comprising performing lookup operations into the flash memory while rebuilding the index.
18. The computing system of claim 12 , further comprising storing a cache state in a reserved region of the flash memory.
19. The computing system of claim 18 , wherein the cache state identifies a highest container identifier stored in the flash memory and identifies which regions of the flash memory are unused, wherein identifying the highest container identifier ensures that all containers stored in the flash memory are read while rebuilding the index.
20. The computing system of claim 12 , further comprising inserting a new container into a portion of a used region of the flash memory that is not yet indexed without indexing any containers replaced by the insertion of the new container and without removing entries that correspond to data segments stored in the containers being replaced from the index.