Distributed system with asynchronous execution systems and methods
View Patent ↗Systems and methods are provided for reducing latency in distributed systems by executing commands as sufficient information and system resources become available. In one embodiment, commands in a transaction include dependency information and an execution engine is configured to execute the commands as the dependencies become satisfied. In some embodiments, the commands also include priority information. If sufficient resources are not available to execute two or more commands with satisfied dependencies, the execution engine determines an order for executing the commands based at least in part on the priority information. In one embodiment, time-intensive commands are assigned a higher priority than commands that are expected to take less time to execute.
1. A method of performing a transaction in a distributed system, the method comprising:
obtaining, by a first computing node, a first command and a second command that define functions to be performed in the transaction, wherein the first command includes dependency information that comprises a first local dependency that must be satisfied before the first command is executed and a first remote dependency that must be satisfied before the first command is executed, and wherein the second command includes dependency information that comprises a second dependency that must be satisfied before the second command is executed;
holding the first command in a waiting state until the first local dependency is satisfied;
determining that the second dependency is satisfied;
causing the execution of the second command;
determining that the first local dependency is satisfied and that there is a first remote dependency that must be satisfied;
transmitting the first command to a second computing node for asynchronous execution, the second computing node to determine when to perform the functions defined by the first command based on the first remote dependency included in the first command; and
determining that the first command has been executed at the second computing node.
2. The method of claim 1 , wherein determining that the first local dependency is satisfied comprises determining that the second command has been executed.
3. The method of claim 1 , wherein the first command and the second command include prioritization information.
4. The method of claim 1 , wherein causing the execution of the second command comprises performing the function defined by the second first command on the first computing node.
5. The method of claim 1 , wherein causing the execution of the second command comprises:
transmitting the second command to the second computing node for execution, the second computing node to determine when to perform the functions defined by the second command.
6. The method of claim 5 , further comprising:
receiving data from the second computing node, wherein the data from the second computing node corresponds to data resulting from the execution of at least one of the first command and the second command.
7. The method of claim 1 , wherein the first computing node and the second computing node comprise smart storage units.
8. The method of claim 1 , wherein the transaction is selected from a group comprising one or more of:
a write transaction;
a mirror transaction;
a mirror recovery transaction;
a parity write transaction; and
a restripe transaction.
9. A distributed system comprising:
a plurality of commands, each command structured to include dependency information and priority information; and
a plurality of nodes comprising a first node and a second node, each node comprising at least one computer processor,
wherein the first node is configured to:
obtain a first command and a second command that define functions to be performed in a transaction, wherein the first command includes dependency information that comprises a first dependency that must be satisfied before the first command is executed, and wherein the second command includes dependency information that comprises a second local dependency that must be satisfied before the second command is executed and a second remote dependency that must be satisfied before the second command is executed;
process the dependency information included in the first command;
execute the first command only after the first dependency is satisfied;
process the dependency information included in the second command;
transmit the second command to a second node for asynchronous execution only after the second local dependency is satisfied; and
determine that the second node has executed the second command; and
wherein the second node is configured to:
receive the second command from the first node;
process the dependency information associated with the second command; and
execute the second command only after the second remote dependency is satisfied.
10. The distributed system of claim 9 , wherein the second local dependency is satisfied after the first command has been executed.
11. The distributed system of claim 9 , wherein the first dependency is satisfied after the second command has been executed.
12. The distributed system of claim 9 , wherein the first node is further configured to execute the second command subsequent to the second local dependency being satisfied.
13. The distributed system of claim 9 , wherein the second node is further configured to transmit a third command to the first computing device for asynchronous execution, the first computing device to determine when to perform the functions defined by the third command based on dependency information included in the third command.
14. The distributed system of claim 9 , wherein the second node is further configured to transmit result data to the first node, wherein the result data corresponds to data resulting from the execution of the second command.
15. The distributed system of claim 9 , wherein the plurality of nodes comprise smart storage units.
16. The distributed system of claim 9 , wherein the transaction is selected from a group comprising one or more of:
a write transaction;
a mirror transaction;
a mirror recovery transaction;
a parity write transaction; and
a restripe transaction.