Peer-to-peer based caching for network file system
Systems and techniques relating to network file systems are described. A described technique includes associating with a process that exports a network file system, where the process manages the network file system, associating with one or more peers that use the network file system, where the one or more peers cache data associated with the network file system on one or more non-volatile memory structures, receiving a file system request to read or write at least a portion of a file that is associated with the network file system, and processing the file system request by selectively communicating with the one or more peers to retrieve cached data associated with the file when the file system request indicates a read or inform the one or more peers of a write to the file when the file system request indicates a write.
1. A system comprising:
a server system running a process that exports a network file system, wherein the process manages the network file system; and
a device that is communicatively coupled with the server system via a network, the device comprising: (i) a non-volatile memory structure to cache data associated with the network file system, and (ii) processor electronics configured to
associate with the network file system,
associate with one or more peers that are configured to cache data associated with the network file system, the one or more peers having respective one or more connections with the network file system,
receive a file system request to read or write at least a portion of a file that is associated with the network file system,
selectively communicate with the one or more peers to retrieve cached data associated with the file when the file system request indicates a read, and
perform a write file operation when the file system request indicates a write, the write file operation to receive file data from an application for writing to the file, selectively communicate with the one or more peers to inform the one or more peers of a write to the file, selectively perform data de-duplication to invalidate any remotely cached versions of the file that are cached by the one or more peers, and cache the file data in the non-volatile memory structure,
wherein the processor electronics are configured to commit the file data to the network file system.
2. The system of claim 1 , wherein the file system request comprises a write request, and wherein the processor electronics are configured to
commit the file data in aggregate with additional file data to the network file system after the write file operation is successfully completed, the additional file data being associated with an additional write file operation.
3. The system of claim 1 , wherein the file system request comprises a read request, and wherein the processor electronics are configured to
check for the file in the non-volatile memory structure,
selectively identify a peer of the one or more peers having a cached version of the file,
selectively send, based on a lack of the file in the non-volatile memory structure, a request to the identified peer to retrieve at least a portion of a remotely cached version of the file, and
selectively communicate, based on a lack of locally and remotely cached versions of the file, with a process exporting the network file system to retrieve at least a portion of the file.
4. The system of claim 1 , wherein the processor electronics are configured to
cache the file on the non-volatile memory structure, and
perform data de-duplication to invalidate one or more remotely cached versions of the file.
5. The system of claim 1 , wherein the processor electronics are configured to run virtual machines that comprise a first virtual machine and a second virtual machine, wherein the first virtual machine runs a first peer process to access the network file system, wherein the second virtual machine runs a second peer process to access the network file system, and wherein the first peer process and second peer process cache data associated with the network file system on the non-volatile memory structure.
6. The system of claim 5 , wherein the first peer process caches a first version of the file on the non-volatile memory structure, wherein the second peer process caches a second version of the file on the non-volatile memory structure and causes the first peer process to invalidate the first version.
7. The system of claim 1 , wherein the device comprises a solid state drive that comprises the non-volatile memory structure, wherein the non-volatile memory structure comprises a flash memory structure.
8. An apparatus comprising:
a non-volatile memory structure to cache data associated with a network file system; and
processor electronics configured to
associate with the network file system,
associate with one or more peers that are configured to cache data associated with the network file system, the one or more peers having respective one or more connections with the network file system,
receive a file system request to read or write at least a portion of a file that is associated with the network file system,
selectively communicate with the one or more peers to retrieve cached data associated with the file, when the file system request indicates a read, and
perform a write file operation when the file system request indicates a write, the write file operation to receive file data from an application for writing to the file, selectively communicate with the one or more peers to inform the one or more peers of a write to the file, selectively perform data de-duplication to invalidate any remotely cached versions of the file that are cached by the one or more peers, and cache the file data in the non-volatile memory structure,
wherein the processor electronics are configured to commit the file data to the network file system.
9. The apparatus of claim 8 , wherein the file system request comprises a write request, and wherein the processor electronics are configured to
commit the file data in aggregate with additional file data to the network file system after the write file operation is successfully completed, the additional file data being associated with an additional write file operation.
10. The apparatus of claim 8 , wherein the file system request comprises a read request, and wherein the processor electronics are configured to
check for the file in the non-volatile memory structure,
selectively identify a peer of the one or more peers having a cached version of the file,
selectively send, based on a lack of the file in the non-volatile memory structure, a request to the identified peer to retrieve at least a portion of a remotely cached version of the file, and
selectively communicate, based on a lack of locally and remotely cached versions of the file, with a process exporting the network file system to retrieve at least a portion of the file.
11. The apparatus of claim 8 , wherein the processor electronics are configured to
cache the file on the non-volatile memory structure, and
perform data de-duplication to invalidate one or more remotely cached versions of the file.
12. The apparatus of claim 8 , wherein the processor electronics are configured to run virtual machines that comprise a first virtual machine and a second virtual machine, wherein the first virtual machine runs a first peer process to access the network file system, wherein the second virtual machine runs a second peer process to access the network file system, and wherein the first peer process and second peer process cache data associated with the network file system on the non-volatile memory structure.
13. The apparatus of claim 12 , wherein the first peer process caches a first version of the file on the non-volatile memory structure, wherein the second peer process caches a second version of the file on the non-volatile memory structure and causes the first peer process to invalidate the first version.
14. The apparatus of claim 8 , comprising:
a solid state drive that comprises the non-volatile memory structure, wherein the non-volatile memory structure comprises a flash memory structure.
15. A method comprising:
associating with a process that exports a network file system, wherein the process manages the network file system;
associating with one or more peers that use the network file system, wherein the one or more peers cache data associated with the network file system on one or more non-volatile memory structures;
receiving a file system request to read or write at least a portion of a file that is associated with the network file system; and
processing the file system request by selectively communicating with the one or more peers to retrieve cached data associated with the file when the file system request indicates a read and performing a write file operation when the file system request indicates a write, the write file operation comprising: receiving file data from an application for writing to the file, selectively informing the one or more peers of a write to the file, selectively performing data de-duplication to invalidate any remotely cached versions of the file that are cached by the one or more peers, and caching the file data in a local non-volatile memory structure, wherein processing the file system request comprises committing the file data to the network file system.
16. The method of claim 15 , wherein the file system request comprises a write request, and wherein
the file data is committed in aggregate with additional file data to the network file system after the write file operation is successfully completed, the additional file data being associated with an additional write file operation.
17. The method of claim 15 , wherein the file system request comprises a read request, and wherein processing the file system request comprises:
checking for the file in the local non-volatile memory structure;
selectively identifying a peer of the one or more peers having a cached version of the file;
selectively sending, based on a lack of the file in the local non-volatile memory structure, a request to the identified peer to retrieve at least a portion of a remotely cached version of the file; and
selectively communicating, based on a lack of locally and remotely cached versions of the file, with a process exporting the network file system to retrieve at least a portion of the file.
18. The method of claim 15 , wherein processing the file system request comprises:
caching the file on the local non-volatile memory structure; and
performing data de-duplication to invalidate one or more remotely cached versions of the file.
19. The method of claim 15 , comprising:
running virtual machines that comprise a first virtual machine and a second virtual machine, wherein the first virtual machine runs a first peer process to access the network file system, wherein the second virtual machine runs a second peer process to access the network file system, and wherein the first peer process and second peer process cache data associated with the network file system on the one or more non-volatile memory structures.
20. The method of claim 19 , wherein the first peer process caches a first version of the file on the one or more non-volatile memory structures, wherein the second peer process caches a second version of the file on the one or more non-volatile memory structures and causes the first peer process to invalidate the first version.