Granted Patent B1

US 12,222,903 · App. 18/799,431 · Granted Feb 11, 2025

Inventors: Thomas Gregory Rothschilds (Issaquah, WA); Aaron James Passey (San Rafael, CA); Edward Carpenter (Seattle, WA)

Assignee: Qumulo, Inc.

G06F16/1774G06F16/172G06F16/182

Abstract

Claims (149)

1. A method for managing data in a file system over a network using one or more processors to execute instructions that are configured to cause performance of actions, comprising:

sharing one or more portions of a hub file system that includes one or more nodes with one or more spoke file systems that includes one or more other nodes, wherein the hub file system shares a plurality of file system items with the one or more spoke file systems; and

employing a request from a client of the hub file system to access one or more file system items that are in the hub file system to perform further actions, including:

determining a portion of the one or more spoke file systems that have cached data associated with the one or more file system items;

updating the one or more file system items in the hub file system based on an execution of one or more commands associated with the request from the hub file system client;

generating one or more invalidation log entries on the hub file system based on an execution of the one or more commands;

communicating the one or more invalidation log entries to the portion of the one or more spoke file systems, wherein the portion of the one or more spoke file systems employs the one or more invalidation log entries to invalidate each cached portion of the data associated with the one or more file system items; and

employing another request from the client for access to one or more other file system items of the plurality of shared file system items to perform further actions, including:

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems, and wherein each log entry is associated with a logical timestamp that corresponds to a sequence that the one or more commands were executed in the other portion of the one or more spoke file systems;

executing the one or more commands in the sequence again in the hub file system for each logical timestamp of the one or more log entries to update the one or more other file system items in the hub file system; and

granting a lock on the one or more other file system items in the hub file system that enables the client of the one or more spoke file systems to access the one or more updated file system items from the other portion of one or more spoke file systems.

2. The method of claim 1 , further comprising:

employing another request from a client of a spoke file system to access one or more other file system items that are in the hub file system to perform further actions, including:

obtaining a lock on the one or more other file system items for the client of the spoke file system based on the other request;

generating one or more log entries based on an execution of one or more other commands associated with the other request, wherein the one or more log entries are stored in the spoke file system to complete the other request; and

communicating the one or more other log entries from the spoke file system to the hub file system based on one or more of a message from the hub file system or a policy of the hub file system.

3. The method of claim 1 , further comprising:

obtaining one or more log entries from a portion one or more spoke file systems, wherein the one or more log entries are associated with one or more commands that were previously executed on the portion of the one the one or more spoke file systems, and wherein the portion of the one or more spoke file system have one or more locks associated with a portion of the one or more file system items;

executing the one or more commands in the hub file system; and

directing the portion of the one or more spoke file systems to release each lock associated with the portion of one or more file system entries.

4. The method of claim 1 , further comprising:

employing an execution of one or more commands on the hub file system, to perform, further actions, including:

determining another portion of the one or more spoke file systems that have an existing lock on the one or more other file system items;

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems;

associating a logical timestamp of a log entry corresponding to a most recent executed command with the one or more file system items; and

disregarding one or more subsequent commands that are associated with another log entry that has a logical timestamp that is equal to or less than the logical timestamp associated with the one or more file system items.

5. The method of claim 1 , further comprising:

determining one or more meta-data updates associated with one or more other file system items stored in the hub file system;

determining another portion of the one or more spoke file systems that are shared a portion of the one or more other file system items;

generating one or more meta-data update log entries based on the one or more meta-data updates; and

communicating the one or more meta-data update log entries to the other portion of the one or more spoke file systems, wherein meta-data for each of the one or more other file system items is updated on the portion of one or more spoke file systems.

6. The method of claim 1 , further comprising:

determining a plurality of log entries that are stored in the portion of the one or more spoke file systems; and

determining one or more log entries from the plurality of log entries based on one or more file system item identifiers included in the one or more log entries, wherein one or more other log entries of the plurality of log entries that omit the one or more file system identifiers are excluded from the one or more log entries.

7. The method of claim 1 , further comprising:

generating a plurality of invalidation log entries on the hub file system based on an execution of a plurality of commands on the hub file system; and

communicating the plurality of invalidation log entries to the one or more spoke file systems, wherein each spoke file system is associated with a cursor that iterates independently through the plurality of invalidation log entries, and wherein each cursor enables its associated spoke file system to consume invalidation log entries at a different rate.

8. A system for managing data in a file system comprising:

a network computer, comprising:

a memory that stores at least instructions; and

one or more processors that execute instructions that are configured to cause performance of actions, including:

sharing one or more portions of a hub file system that includes one or more nodes with one or more spoke file systems that includes one or more other nodes, wherein the hub file system shares a plurality of file system items with the one or more spoke file systems; and

employing a request from a client of the hub file system to access one or more file system items that are in the hub file system to perform further actions, including:

determining a portion of the one or more spoke file systems that have cached data associated with the one or more file system items;

updating the one or more file system items in the hub file system based on an execution of one or more commands associated with the request from the hub file system client;

generating one or more invalidation log entries on the hub file system based on an execution of the one or more commands;

communicating the one or more invalidation log entries to the portion of the one or more spoke file systems, wherein the portion of the one or more spoke file systems employs the one or more invalidation log entries to invalidate each cached portion of the data associated with the one or more file system items; and

employing another request from the client for access to one or more other file system items of the plurality of shared file system items to perform further actions, including:

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems, and wherein each log entry is associated with a logical timestamp that corresponds to a sequence that the one or more commands were executed in the other portion of the one or more spoke file systems;

executing the one or more commands in the sequence again in the hub file system for each logical timestamp of the one or more log entries to update the one or more other file system items in the hub file system; and

granting a lock on the one or more other file system items in the hub file system that enables the client of the one or more spoke file systems to access the one or more updated file system items from the other portion of one or more spoke file systems; and

one or more client computers, comprising:

a memory that stores at least instructions; and

one or more processors that execute instructions that are configured to cause performance of other actions, including:

communicating the request to the spoke file system.

9. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising:

employing another request from a client of a spoke file system to access one or more other file system items that are in the hub file system to perform further actions, including:

obtaining a lock on the one or more other file system items for the client of the spoke file system based on the other request;

generating one or more log entries based on an execution of one or more other commands associated with the other request, wherein the one or more log entries are stored in the spoke file system to complete the other request; and

communicating the one or more other log entries from the spoke file system to the hub file system based on one or more of a message from the hub file system or a policy of the hub file system.

10. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising:

obtaining one or more log entries from a portion one or more spoke file systems, wherein the one or more log entries are associated with one or more commands that were previously executed on the portion of the one the one or more spoke file systems, and wherein the portion of the one or more spoke file system have one or more locks associated with a portion of the one or more file system items;

executing the one or more commands in the hub file system; and

directing the portion of the one or more spoke file systems to release each lock associated with the portion of one or more file system entries.

11. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising: employing an execution of one or more commands on the hub file system, to perform, further actions, including:

determining another portion of the one or more spoke file systems that have an existing lock on the one or more other file system items;

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems;

associating a logical timestamp of a log entry corresponding to a most recent executed command with the one or more file system items; and

disregarding one or more subsequent commands that are associated with another log entry that has a logical timestamp that is equal to or less than the logical timestamp associated with the one or more file system items.

12. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising: determining one or more meta-data updates associated with one or more other file system items stored in the hub file system;

determining another portion of the one or more spoke file systems that are shared a portion of the one or more other file system items;

generating one or more meta-data update log entries based on the one or more meta-data updates; and

communicating the one or more meta-data update log entries to the other portion of the one or more spoke file systems, wherein meta-data for each of the one or more other file system items is updated on the portion of one or more spoke file systems.

13. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising:

determining a plurality of log entries that are stored in the portion of the one or more spoke file systems; and

determining one or more log entries from the plurality of log entries based on one or more file system item identifiers included in the one or more log entries, wherein one or more other log entries of the plurality of log entries that omit the one or more file system identifiers are excluded from the one or more log entries.

14. The system of claim 8 , wherein the one or more network computer processors execute instructions that are configured to cause performance of actions, further comprising:

generating a plurality of invalidation log entries on the hub file system based on an execution of a plurality of commands on the hub file system; and

communicating the plurality of invalidation log entries to the one or more spoke file systems, wherein each spoke file system is associated with a cursor that iterates independently through the plurality of invalidation log entries, and wherein each cursor enables its associated spoke file system to consume invalidation log entries at a different rate.

15. A processor readable non-transitory storage media that includes instructions for managing data in a file system over a network, wherein execution of the instructions by one or more processors on one or more network computers performs actions, comprising:

sharing one or more portions of a hub file system that includes one or more nodes with one or more spoke file systems that includes one or more other nodes, wherein the hub file system shares a plurality of file system items with the one or more spoke file systems; and

employing a request from a client of the hub file system to access one or more file system items that are in the hub file system to perform further actions, including:

determining a portion of the one or more spoke file systems that have cached data associated with the one or more file system items;

updating the one or more file system items in the hub file system based on an execution of one or more commands associated with the request from the hub file system client;

generating one or more invalidation log entries on the hub file system based on an execution of the one or more commands;

communicating the one or more invalidation log entries to the portion of the one or more spoke file systems, wherein the portion of the one or more spoke file systems employs the one or more invalidation log entries to invalidate each cached portion of the data associated with the one or more file system items; and

employing another request from the client for access to one or more other file system items of the plurality of shared file system items to perform further actions, including:

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems, and wherein each log entry is associated with a logical timestamp that corresponds to a sequence that the one or more commands were executed in the other portion of the one or more spoke file systems;

executing the one or more commands in the sequence again in the hub file system for each logical timestamp of the one or more log entries to update the one or more other file system items in the hub file system; and

granting a lock on the one or more other file system items in the hub file system that enables the client of the one or more spoke file systems to access the one or more updated file system items from the other portion of one or more spoke file systems.

16. The media of claim 15 , further comprising:

employing another request from a client of a spoke file system to access one or more other file system items that are in the hub file system to perform further actions, including:

obtaining a lock on the one or more other file system items for the client of the spoke file system based on the other request;

generating one or more log entries based on an execution of one or more other commands associated with the other request, wherein the one or more log entries are stored in the spoke file system to complete the other request; and

communicating the one or more other log entries from the spoke file system to the hub file system based on one or more of a message from the hub file system or a policy of the hub file system.

17. The media of claim 15 , further comprising:

obtaining one or more log entries from a portion one or more spoke file systems, wherein the one or more log entries are associated with one or more commands that were previously executed on the portion of the one the one or more spoke file systems, and wherein the portion of the one or more spoke file system have one or more locks associated with a portion of the one or more file system items;

executing the one or more commands in the hub file system; and

directing the portion of the one or more spoke file systems to release each lock associated with the portion of one or more file system entries.

18. The media of claim 15 , further comprising:

employing an execution of one or more commands on the hub file system, to perform, further actions, including:

determining another portion of the one or more spoke file systems that have an existing lock on the one or more other file system items;

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems;

associating a logical timestamp of a log entry corresponding to a most recent executed command with the one or more file system items; and

disregarding one or more subsequent commands that are associated with another log entry that has a logical timestamp that is equal to or less than the logical timestamp associated with the one or more file system items.

19. The media of claim 15 , further comprising:

determining one or more meta-data updates associated with one or more other file system items stored in the hub file system;

determining another portion of the one or more spoke file systems that are shared a portion of the one or more other file system items;

generating one or more meta-data update log entries based on the one or more meta-data updates; and

communicating the one or more meta-data update log entries to the other portion of the one or more spoke file systems, wherein meta-data for each of the one or more other file system items is updated on the portion of one or more spoke file systems.

20. The media of claim 15 , further comprising:

generating a plurality of invalidation log entries on the hub file system based on an execution of a plurality of commands on the hub file system; and

communicating the plurality of invalidation log entries to the one or more spoke file systems, wherein each spoke file system is associated with a cursor that iterates independently through the plurality of invalidation log entries, and wherein each cursor enables its associated spoke file system to consume invalidation log entries at a different rate.

21. A network computer for managing data in a file system, comprising:

a memory that stores at least instructions; and

one or more processors that execute instructions that are configured to cause performance of actions, including:

sharing one or more portions of a hub file system that includes one or more nodes with one or more spoke file systems that includes one or more other nodes, wherein the hub file system shares a plurality of file system items with the one or more spoke file systems; and

employing a request from a client of the hub file system to access one or more file system items that are in the hub file system to perform further actions, including:

determining a portion of the one or more spoke file systems that have cached data associated with the one or more file system items;

updating the one or more file system items in the hub file system based on an execution of one or more commands associated with the request from the hub file system client;

generating one or more invalidation log entries on the hub file system based on an execution of the one or more commands;

communicating the one or more invalidation log entries to the portion of the one or more spoke file systems, wherein the portion of the one or more spoke file systems employs the one or more invalidation log entries to invalidate each cached portion of the data associated with the one or more file system items; and

employing another request from the client for access to one or more other file system items of the plurality of shared file system items to perform further actions, including:

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems, and wherein each log entry is associated with a logical timestamp that corresponds to a sequence that the one or more commands were executed in the other portion of the one or more spoke file systems;

executing the one or more commands in the sequence again in the hub file system for each logical timestamp of the one or more log entries to update the one or more other file system items in the hub file system; and

granting a lock on the one or more other file system items in the hub file system that enables the client of the one or more spoke file systems to access the one or more updated file system items from the other portion of one or more spoke file systems.

22. The network computer of claim 21 , wherein the one or more processors execute instructions that are configured to cause performance of actions, further comprising:

employing another request from a client of a spoke file system to access one or more other file system items that are in the hub file system to perform further actions, including:

obtaining a lock on the one or more other file system items for the client of the spoke file system based on the other request;

generating one or more log entries based on an execution of one or more other commands associated with the other request, wherein the one or more log entries are stored in the spoke file system to complete the other request; and

communicating the one or more other log entries from the spoke file system to the hub file system based on one or more of a message from the hub file system or a policy of the hub file system.

23. The network computer of claim 21 , wherein the one or more processors execute instructions that are configured to cause performance of actions, further comprising:

obtaining one or more log entries from a portion one or more spoke file systems, wherein the one or more log entries are associated with one or more commands that were previously executed on the portion of the one the one or more spoke file systems, and wherein the portion of the one or more spoke file system have one or more locks associated with a portion of the one or more file system items;

executing the one or more commands in the hub file system; and

directing the portion of the one or more spoke file systems to release each lock associated with the portion of one or more file system entries.

24. The network computer of claim 21 , wherein the one or more processors execute instructions that are configured to cause performance of actions, further comprising:

employing an execution of one or more commands on the hub file system, to perform, further actions, including:

determining another portion of the one or more spoke file systems that have an existing lock on the one or more other file system items;

obtaining one or more log entries from the other portion of the one or more spoke file systems, wherein the one or more log entries are associated with one or more other commands that were executed in the other portion of the one or more spoke file systems;

associating a logical timestamp of a log entry corresponding to a most recent executed command with the one or more file system items; and

disregarding one or more subsequent commands that are associated with another log entry that has a logical timestamp that is equal to or less than the logical timestamp associated with the one or more file system items.

25. The network computer of claim 21 , wherein the one or more processors execute instructions that are configured to cause performance of actions, further comprising:

determining one or more meta-data updates associated with one or more other file system items stored in the hub file system;

determining another portion of the one or more spoke file systems that are shared a portion of the one or more other file system items;

generating one or more meta-data update log entries based on the one or more meta-data updates; and

communicating the one or more meta-data update log entries to the other portion of the one or more spoke file systems, wherein meta-data for each of the one or more other file system items is updated on the portion of one or more spoke file systems.

26. The network computer of claim 21 , wherein the one or more processors execute instructions that are configured to cause performance of actions, further comprising:

generating a plurality of invalidation log entries on the hub file system based on an execution of a plurality of commands on the hub file system; and

communicating the plurality of invalidation log entries to the one or more spoke file systems, wherein each spoke file system is associated with a cursor that iterates independently through the plurality of invalidation log entries, and wherein each cursor enables its associated spoke file system to consume invalidation log entries at a different rate.

Assignments (2)

References Cited (400)

Cited By (8)

US 12,443,559 US 12,443,568 US 12,481,625 US 12,585,563 US 12,619,582 US 12,670,081 US 12,743,357 US 12,748,730