Wide area network file system with low latency write command processing
View Patent ↗Methods, apparatuses, and systems directed to write command processing in distributed file caching systems. Implementations of the invention allow for write operations to identified files to proceed, while information regarding the identified file is fetched from a remote host and a locally cached version of the file is constructed. Implementations of the present invention can be configured to improve the performance of wide area network file systems, while preserving file consistency.
1. An apparatus comprising
one or more network interfaces;
a memory;
one or more processors;
computer-readable instructions stored in the memory operable to cause the one or more processors to
initiate, responsive to a file open command received from a requesting application, a fetch of a file identified in the file open command from a remote host, the file located remotely from the one or more processors;
forward, while the fetch is in process, write commands identifying the file to the remote host for remote execution of the write commands;
locally execute the write commands on a cached version of the file, wherein the cached version of the file is local to the one or more processors;
construct the cached version of the file as file construction commands are received from the remote host, wherein the file construction commands are selectively processed to avoid overwriting data of the cached version of the file corresponding to the locally executed write commands.
2. The apparatus of claim 1 wherein the computer readable instructions are further operable to cause the one or more processors to
access, responsive to the file open command, a local cache to identify if a cached version of the identified file is stored in the local cache;
compute, if a cached version of the identified file exists in the local cache, one or more data signatures for one or more blocks of the cached version of the data file; and
transmit the one or more data signatures to the remote host.
3. The apparatus of claim 1 wherein the computer readable instructions are further operable to cause the one or more processors to
maintain a write history for the cached version of the file;
conditionally add offset and length data associated with the received write commands to the write history based on progress of the file fetch.
4. The apparatus of claim 3 wherein the computer readable instructions are further operable to cause the one or more processors to
access the write history of the file to conditionally execute the file construction commands on portions of the cached version of the file outside of data segments identified in the write history.
5. The apparatus of claim 3 wherein the write history is maintained as an ordered set of write commands sorted by offset value.
6. The apparatus of claim 3 wherein the computer readable instructions are further operable to cause the one or more processors to
collapse contiguous or overlapping write commands in the write history into one write command entry.
7. A method comprising
initiating, responsive to a file open command received from a requesting application, a fetch of a file identified in the file open command from a remote host, the file located remotely;
forwarding, while the fetch is in process, write commands identifying the file to the remote host for remote execution of the write commands;
locally executing the write commands on a cached version of the file, wherein the cached version of the file is local;
constructing the cached version of the file as file construction commands are received from the remote host, wherein the file construction commands are selectively processed to avoid overwriting data of the cached version of the file corresponding to the locally executed write commands.
8. The method of claim 7 further comprising
accessing, responsive to the file open command, a local cache to identify if a cached version of the identified file is stored in the local cache;
computing, if a cached version of the identified file exists in the local cache, one or more data signatures for one or more blocks of the cached version of the data file; and
transmitting the one or more data signatures to the remote host.
9. The method of claim 7 further comprising
maintaining a write history for the cached version of the file; and
conditionally adding offset and length data associated with the received write commands to the write history based on progress of the file fetch.
10. The method of claim 9 further comprising
accessing the write history of the file to conditionally execute the file construction commands on portions of the cached version of the file outside of data segments identified in the write history.
11. The method of claim 9 wherein the write history is maintained as an ordered set of write commands sorted by offset value.
12. The method of claim 9 further comprising
collapsing contiguous or overlapping write commands in the write history into one write command entry.
13. A system comprising
an edge file cache node and a server cache node;
wherein the edge file cache node comprises
one or more network interfaces;
a memory;
one or more processors; and
computer-readable instructions stored in the memory operable to cause the one or more processors to: initiate, responsive to a file open command received from a requesting application, a fetch of a file identified in the file open command from the server cache node; forward, while the fetch is in process, write commands identifying the file to the server cache node for remote execution; locally execute the write commands on a cached version of the file, the cached version of the file stored in the memory; and construct the cached version of the file as file construction commands are received from the server cache node, wherein the file construction commands are selectively processed to avoid overwriting data of the cached version of the file corresponding to the locally executed write commands;
wherein the server cache node comprises
a server cache node
one or more network interfaces;
a memory;
one or more processors; and
computer-readable instructions stored in the memory operable to cause the one or more processors to: retrieve the file from a remote host, responsive to the fetch initiated by the edge file cache node; forward write commands to the file from the edge file cache node to the remote host; and generate and transmit to the edge file cache node file construction commands for the file.
14. The system of claim 13 wherein the computer readable instructions are further operable to cause the one or more processors to
access, responsive to the file open command, a local cache to identify if a cached version of the identified file is stored in the local cache;
compute, if a cached version of the identified file exists in the local cache, one or more data signatures for one or more blocks of the cached version of the data file; and
transmit the one or more data signatures to the remote host.
15. The system of claim 13 wherein the computer readable instructions are further operable to cause the one or more processors to
maintain a write history for the cached version of the file;
conditionally add offset and length data associated with the received write commands to the write history based on progress of the file fetch.
16. The system of claim 15 wherein the computer readable instructions are further operable to cause the one or more processors to
access the write history of the file to conditionally execute the file construction commands on portions of the cached version of the file outside of data segments identified in the write history.
17. The system of claim 15 wherein the write history is maintained as an ordered set of write commands sorted by offset value.
18. The system of claim 15 wherein the computer readable instructions are further operable to cause the one or more processors to
collapse contiguous or overlapping write commands in the write history into one write command entry.