Method and system for latency optimization in a distributed compute network
There is provided a computer-implemented method of provisioning resources in a distributed compute network comprising one or more routing nodes and one or more compute nodes configured to host one or more virtual application instances of an application thereon, the method being performed by at least one hardware processor and comprising: receiving, by a system manager, operational parameter data for each available routing node for a current state of the distributed compute network, wherein the operational parameter data comprises measured and/or predicted values of the local latency between a respective routing node and any available compute nodes accessible by the respective routing node; determining, by the system manager, an optimized global latency value for the current state of the distributed compute network based on the received operational parameter data; defining a latency threshold for a global latency for the application based on the optimized global latency value; generating, utilizing a trained machine learning model, a proposed new state of the distributed compute network having a global latency for the application which meets or exceeds the global latency threshold; and implementing the proposed new state on the distributed compute network by selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application.
1 . A computer-implemented method of provisioning resources in a distributed compute network comprising one or more routing nodes and one or more compute nodes configured to host one or more virtual application instances of an application thereon, the method being performed by at least one hardware processor and comprising:
a) receiving, by a system manager, operational parameter data for each available routing node for a current state of the distributed compute network, wherein the operational parameter data comprises measured and/or predicted values of a local latency between a respective routing node and any available compute nodes accessible by the respective routing node;
b) determining, by the system manager, an optimized global latency value for the current state of the distributed compute network based on the received operational parameter data;
c) defining a latency threshold for a global latency for the application based on the optimized global latency value;
d) generating, utilizing a trained machine learning model, a proposed new state of the distributed compute network having a global latency for the application which meets or exceeds the latency threshold; and
e) implementing the proposed new state on the distributed compute network by selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application.
2 . A computer-implemented method according to claim 1 , wherein the global latency of the application is a function of the local latencies of any provisioned virtual application instances.
3 . A computer-implemented method according to claim 1 , wherein the optimized global latency value is a function of the local latency values for each available routing node between a respective routing node and an available compute node accessible by the respective routing node having a lowest local latency value.
4 . A computer-implemented method according to claim 3 , wherein the optimized global latency value comprises an averaged sum of the local latency values for each available routing node between a respective routing node and the available compute node accessible by the respective routing node having the lowest latency.
5 . A computer-implemented method according to claim 1 , wherein step c) further comprises defining one or more further latency thresholds for the global latency for the application based on the optimized global latency value.
6 . A computer-implemented method according to claim 1 , wherein the trained machine learning model is trained using reinforcement-learning.
7 . A computer-implemented method according to claim 1 , wherein step c) further comprises:
f) proposing one or more actions to the current state to generate a proposed new state, the one or more actions comprising selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application;
g) determining whether the global latency of the proposed new state meets or exceeds the latency threshold and, if so determined, implementing the proposed new state at step e).
8 . A computer-implemented method according to claim 7 , wherein if, at step g) the global latency of the proposed new state does not meet or exceed the latency threshold, the method further comprises:
h) Iteratively repeating steps f) and g) until the latency threshold is met.
9 . A system for provisioning resources in a distributed compute network comprising one or more routing nodes and one or more compute nodes configured to host one or more virtual application instances of an application thereon, the system comprising:
at least one hardware processor operable to perform the steps of:
a) receiving, by a system manager, operational parameter data for each available routing node for a current state of the distributed compute network, wherein the operational parameter data comprises measured and/or predicted values of a local latency between a respective routing node and any available compute nodes accessible by the respective routing node;
b) determining, by the system manager, an optimized global latency value for the current state of the distributed compute network based on the received operational parameter data;
c) defining a latency threshold for a global latency for the application based on the optimized global latency value;
d) generating, utilizing a trained machine learning model, a proposed new state of the distributed compute network having a global latency for the application which meets or exceeds the latency threshold; and
e) implementing the proposed new state on the distributed compute network by selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application.
10 . A system according to claim 9 , wherein the global latency of the application is a function of the local latencies of any provisioned virtual application instances.
11 . A system according to claim 9 , wherein the optimized global latency value is a function of the local latency values for each available routing node between a respective routing node and an available compute node accessible by the respective routing node having a lowest local latency value.
12 . A system according to claim 11 , wherein the optimized global latency value comprises an averaged sum of the local latency values for each available routing node between a respective routing node and the available compute node accessible by the respective routing node having the lowest latency.
13 . A system according to claim 9 , wherein step c) further comprises defining one or more further latency thresholds for the global latency for the application based on the optimized global latency value.
14 . A system according to claim 9 , wherein the trained machine learning model is trained using reinforcement-learning.
15 . A system according to claim 9 , wherein step c) further comprises:
f) proposing one or more actions to the current state to generate a proposed new state, the one or more actions comprising selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application;
g) determining whether the global latency of the proposed new state meets or exceeds the latency threshold and, if so determined, implementing the proposed new state at step e).
16 . A computer-implemented method according to claim 7 , wherein if, at step g) the global latency of the proposed new state does not meet or exceed the latency threshold, the method further comprises:
h) Iteratively repeating steps f) and g) until the latency threshold is met.
17 . A non-transitory computer readable storage medium storing a program of instructions executable by a machine to perform a computer-implemented method of provisioning resources in a distributed compute network comprising one or more routing nodes and one or more compute nodes configured to host one or more virtual application instances of an application thereon, the method comprising:
a) receiving, by a system manager, operational parameter data for each available routing node for a current state of the distributed compute network, wherein the operational parameter data comprises measured and/or predicted values of a local latency between a respective routing node and any available compute nodes accessible by the respective routing node;
b) determining, by the system manager, an optimized global latency value for the current state of the distributed compute network based on the received operational parameter data;
c) defining a latency threshold for a global latency for the application based on the optimized global latency value;
d) generating, utilizing a trained machine learning model, a proposed new state of the distributed compute network having a global latency for the application which meets or exceeds the latency threshold; and
e) implementing the proposed new state on the distributed compute network by selecting and/or deselecting one or more compute nodes for provisioning of virtual application instances of the application.
18 . A non-transitory computer readable storage medium according to claim 17 , wherein the optimized global latency value is a function of the local latency values for each available routing node between a respective routing node and an available compute node accessible by the respective routing node having a lowest local latency value.
19 . A non-transitory computer readable storage medium according to claim 18 , wherein the optimized global latency value comprises an averaged sum of the local latency values for each available routing node between a respective routing node and the available compute node accessible by the respective routing node having the lowest latency.