Cache consistency optimization
Various methods and systems for optimizing cache consistency are disclosed. For example, one method involves writing data to a file during a write transaction. The file is stored in a persistent storage device and cached in a non-volatile storage device. The method determines if an in-memory flag associated with the persistent storage device set. If the in-memory flag is not set, the method increases a generation count associated with the persistent storage device before a write transaction is performed on the file. The method then sets the in-memory flag before performing the write transaction on the file. In other examples, the method involves using a persistent flag associated with the non-volatile storage device to maintain cache consistency during a data freeze related to the taking of a snapshot by synchronizing generation counts associated with the persistent storage device and the non-volatile storage device.
1. A method comprising:
writing data to a file as part of a write transaction, wherein
the file is stored in a persistent storage device (PSD), and at least a portion of the file is stored in a non-volatile storage device (NVSD), and
the writing the data results in one or more modifications to the file;
determining that a PSD flag is not set, wherein
the PSD flag is associated with in-memory information; and
based on the determination that the PSD flag is not set,
increasing a generation count associated with the PSD, and
storing the in-memory information in the PSD before writing the data to the PSD and the NVSD as part of the write transaction, wherein
the PSD and the NVSD implement a write-through cache.
2. The method of claim 1 , comprising
in response to detecting a subsequent write transaction associated with the file,
inhibiting the increase of the generation count associated with the PSD by virtue of accessing PSD metadata, wherein
the PSD metadata indicates that the PSD flag is set.
3. The method of claim 2 , comprising
setting the PSD flag in an inode of the PSD prior to writing data to the file stored in the PSD and in the NVSD.
4. The method of claim 2 , comprising
inhibiting the increase of a generation count associated with the NVSD after the write transaction and the subsequent write transaction.
5. The method of claim 2 , comprising
accessing PSD metadata during the subsequent write transaction;
determining whether the PSD metadata indicates that the PSD flag is set;
if the PSD metadata indicates that the PSD flag is not set,
only increasing the generation count associated with the PSD while maintaining the generation count associated with the NVSD, and
setting the PSD flag; and
if the PSD metadata indicates that the PSD flag is set,
proceeding with the subsequent write transaction without increasing the generation count associated with the PSD or the generation count associated with the NVSD.
6. The method of claim 2 , wherein
the write transaction and the subsequent write transaction are both write-through cache transactions.
7. The method of claim 1 , wherein
the generation count associated with the PSD tracks a number of write transactions associated with the file stored in the PSD, except when the increase of the generation count associated with the PSD is inhibited by setting of the PSD flag.
8. The method of claim 3 , comprising
copying the generation count from the inode of the PSD to an inode of the NVSD, wherein
the copying occurs,
after the subsequent write transaction, and
during deactivation of the inode of the PSD; and
clearing the PSD flag in the inode of the PSD, wherein
the clearing the PSD flag permits an increase of the generation count associated with the PSD upon a future write transaction.
9. The method of claim 8 , wherein
the inode of the PSD is deactivated after a last close of the file.
10. The method of claim 8 , comprising
detecting a data freeze of data in the PSD;
determining whether the data freeze due to a snapshot;
setting a NVSD flag in an inode of the NVSD if the data freeze is the result of a snapshot, wherein
the NVSD flag is configured to indicate persistent information;
upon restoration of the snapshot,
detecting the NVSD flag in the inode of the NVSD, and
serving data from the PSD instead of the NVSD; and
clearing the PSD flag and the NVSD flag after copying the generation count associated with the PSD from an inode of the PSD to the inode of the NVSD.
11. The method of claim 10 , wherein
detecting the NVSD flag in the inode of the NVSD indicates inconsistency between data stored in the PSD and data stored in the NVSD.
12. The method of claim 10 , comprising
detecting a cache transaction, wherein
the cache transaction increases the generation count associated with the PSD, but not the generation count associated with the NVSD;
setting the PSD flag in the inode of the PSD;
detecting another snapshot of data in the PSD, wherein
the another snapshot is taken after the cache transaction,
the taking of the another snapshot causes a file system structure associated with the PSD and the NVSD to experience a data freeze,
synchronizing the generation count associated with the PSD and the generation count associated with the NVSD, and
clearing the PSD flag and the NVSD flag;
detecting one or more other cache transactions after the cache transaction, wherein
the one or more cache transactions causes the increase of the generation count associated with the PSD; and
setting another NVSD flag in the inode of the NVSD.
13. A non-transitory computer readable storage medium storing program instructions executable to:
write data to a file as part of a write transaction, wherein
the file is stored in a persistent storage device (PSD), and at least a portion of the file is stored in a non-volatile storage device (NVSD), and
the writing the data results in one or more modifications to the file;
determine that a PSD flag is not set, wherein
the PSD flag is associated with in-memory information; and
based on the determination that the PSD flag is not set,
increase a generation count associated with the PSD, and
store the in-memory information in the PSD before writing the data to the PSD and the NVSD as part of the write transaction, wherein
the PSD and the NVSD implement a write-through cache.
14. The non-transitory computer readable storage medium of claim 13 , wherein
the PSD flag is an in-memory flag.
15. The non-transitory computer readable storage medium of claim 13 , comprising
in response to detecting a subsequent write transaction associated with the file,
inhibiting the increase of the generation count associated with the PSD by virtue of accessing PSD metadata, wherein
the PSD metadata indicates that the PSD flag is set.
16. The non-transitory computer readable storage medium of claim 15 , comprising
detecting a data freeze of data in the PSD;
determining whether the data freeze due to a snapshot;
setting a NVSD flag in an inode of the NVSD if the data freeze is a result of the snapshot, wherein
the NVSD flag is configured to indicate persistent information;
upon restoration of the snapshot,
detecting the NVSD flag in the inode of the NVSD, and
serving data from the PSD instead of the NVSD; and
clearing the PSD flag and the NVSD flag after copying the generation count associated with the PSD from an inode of the PSD to the inode of the NVSD.
17. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to:
write data to a file as part of a write transaction, wherein
the file is stored in a persistent storage device (PSD), and at least a portion of the file is stored in a non-volatile storage device (NVSD), and
the writing the data results in one or more modifications to the file;
determine that a PSD flag is not set, wherein
the PSD flag is associated with in-memory information; and
based on the determination that the PSD flag is not set,
increase a generation count associated with the PSD, and
store the in-memory information in the PSD before writing the data to the PSD and the NVSD as part of the write transaction, wherein
the PSD and the NVSD implement a write-through cache.
18. The system of claim 17 , comprising
in response to detecting a subsequent write transaction associated with the file,
inhibiting the increase of the generation count associated with the PSD by virtue of accessing PSD metadata, wherein
the PSD metadata indicates that the PSD flag is set.
19. The system of claim 18 , comprising
detecting a data freeze of data in the PSD;
determining whether the data freeze due to a snapshot;
setting a NVSD flag in an inode of the NVSD if the data freeze is a result of the snapshot, wherein
the NVSD flag is configured to indicate persistent information;
upon restoration of the snapshot,
detecting the NVSD flag in the inode of the NVSD, and
serving data from the PSD instead of the NVSD; and
clearing the PSD flag and the NVSD flag after copying the generation count associated with the PSD from an inode of the PSD to the inode of the NVSD.
20. The method of claim 1 , wherein
storing the in-memory information in the PSD before writing the data to the PSD and the NVSD as part of the write transaction prevents increase of a generation count associated with the NVSD while the data is being written to the PSD and the NVSD.