Apparatus, articles of manufacture, and methods to partition neural networks for execution at distributed edge nodes
Methods, apparatus, systems, and articles of manufacture are disclosed to partition neural network models for executing at distributed Edge nodes. An example apparatus includes processor circuitry to perform at least one of first, second, or third operations to instantiate power consumption estimation circuitry to estimate a computation energy consumption for executing the neural network model on a first edge node, network bandwidth determination circuitry to determine a first transmission time for sending an intermediate result from the first edge node to a second or third edge node, power consumption estimation circuitry to estimate a transmission energy consumption for sending the intermediate result to the second or the third edge node, and neural network partitioning circuitry to partition the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second or third edge node.
1 . An apparatus to partition a neural network model comprising:
interface circuitry to communicate with an edge device; and
processor circuitry including one or more of:
at least one of a central processing unit, a graphic processing unit, or a digital signal processor, the at least one of the central processing unit, the graphic processing unit, or the digital signal processor having control circuitry to control data movement within the processor circuitry, arithmetic and logic circuitry to perform one or more first operations corresponding to instructions, and one or more registers to store a first result of the one or more first operations, the instructions in the apparatus;
a Field Programmable Gate Array (FPGA), the FPGA including first logic gate circuitry, a plurality of configurable interconnections, and storage circuitry, the first logic gate circuitry and interconnections to perform one or more second operations, the storage circuitry to store a second result of the one or more second operations; or
Application Specific Integrate Circuitry (ASIC) including second logic gate circuitry to perform one or more third operations;
the processor circuitry to perform at least one of the first operations, the second operations, or the third operations to instantiate:
power consumption estimation circuitry to estimate a computation energy consumption for executing the neural network model on a first edge node, the neural network model corresponding to a platform service with a service level agreement timeframe;
network bandwidth determination circuitry to determine a transmission time for sending an intermediate result from the first edge node to a second edge node or a third edge node;
the power consumption estimation circuitry to estimate a transmission energy consumption for sending the intermediate result of the neural network model to the second edge node or the third edge node; and
neural network partitioning circuitry to partition the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second edge node or the third edge node based on at least one of the service level agreement timeframe for the platform service, the computation energy consumption, the transmission energy consumption, or the transmission time.
2 . The apparatus of claim 1 , wherein the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate the network bandwidth determination circuitry to determine the transmission time based on available network bandwidth and a payload size, the payload size including the intermediate result.
3 . The apparatus of claim 1 , wherein the transmission time is a first transmission time, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate the network bandwidth determination circuitry to:
determine a second transmission time for sending a final result from the second edge node or the third edge node to the first edge node; and
calculate a total transmission time based on a sum of the first transmission time and the second transmission time, the partitioning of the neural network model into the first portion and the second portion based on the total transmission time satisfying the service level agreement timeframe.
4 . The apparatus of claim 1 , wherein the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate the network bandwidth determination circuitry to:
receive temperature data from a temperature sensor;
receive wind data from a wind sensor; and
receive humidity data from a humidity sensor, the estimation of at least one of the computation energy consumption or the transmission energy consumption based on at least one of the temperature data, the wind data, or the humidity data.
5 . The apparatus of claim 1 , wherein the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate the network bandwidth determination circuitry to:
estimate at least the computation energy consumption or the transmission energy consumption based on at least one of an artificial intelligence algorithm, a set of instructions, or a lookup table; and
calculate a total power consumption based on a sum of the computation energy consumption and the transmission energy consumption, the partitioning of the neural network model into the first portion and the second portion based on the total power consumption satisfying a power consumption threshold.
6 . The apparatus of claim 1 , wherein the processor circuitry is to:
assign a first identifier to input data that the neural network model is to process;
assign a second identifier to the intermediate result, the second identifier to identify (i) the neural network model that is to be executed at the second edge node or the third edge node (ii) an initial layer of the second portion that the second edge node or the third edge node is to execute;
send the intermediate result to the second edge node or the third edge node with the second identifier; and
determine a final result based on the intermediate result and the second identifier, the second identifier to identify the second portion of the neural network model that the second edge node or the third edge node is to execute.
7 . At least one non-transitory computer readable medium comprising instructions that, when executed, cause processor circuitry to at least:
infer a computation energy consumption for executing a neural network model on a first edge node, the neural network model corresponding to a platform service with a service level agreement timeframe;
compute a transmission time for transmitting an intermediate output from the first edge node to a second edge node or a third edge node;
infer a transmission energy consumption for transmitting the intermediate output of the neural network model to the second edge node or the third edge node; and
segment the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second edge node or the third edge node based on at least one of the service level agreement timeframe for the platform service, the computation energy consumption, the transmission energy consumption, or the transmission time.
8 . The at least one non-transitory computer readable medium of claim 7 , wherein the instructions, when executed, cause the processor circuitry to compute the transmission time based on available network bandwidth and a payload size, the payload size including the intermediate output.
9 . The at least one non-transitory computer readable medium of claim 7 , wherein the transmission time is a first transmission time, and the instructions, when executed, cause the processor circuitry to:
compute a second transmission time for sending a final result from the second edge node or the third edge node to the first edge node; and
determine a total transmission time based on a sum of the first transmission time and the second transmission time, the segmenting of the neural network model into the first portion and the second portion based on the total transmission time satisfying the service level agreement timeframe.
10 . The at least one non-transitory computer readable medium of claim 7 , wherein the instructions, when executed, cause the processor circuitry to
receive temperature data from a temperature sensor;
receive wind data from a wind sensor; and
receive humidity data from a humidity sensor, the inference of at least one of the computation energy consumption or the transmission energy consumption based on at least one of the temperature data, the wind data, or the humidity data.
11 . The at least one non-transitory computer readable medium of claim 7 , wherein the instructions, when executed, cause the processor circuitry to:
infer at least the computation energy consumption or the transmission energy consumption based on at least one of an artificial intelligence algorithm, a set of instructions, or a lookup table; and
compute a total power consumption based on a sum of the computation energy consumption and the transmission energy consumption, the segmentation of the neural network model into the first portion and the second portion based on the total power consumption satisfying a power consumption threshold.
12 . The at least one non-transitory computer readable medium of claim 7 , wherein the instructions, when executed, cause the processor circuitry to:
assign a first identifier to input data that the neural network model is to process;
assign a second identifier to the intermediate output, the second identifier to identify (i) the neural network model that is to be executed at the second edge node or the third edge node (ii) an initial layer of the second portion that the second edge node or the third edge node is to execute;
send the intermediate output to the second edge node or the third edge node with the second identifier; and
determine a final result based on the intermediate output and the second identifier, the second identifier to identify the second portion of the neural network model that the second edge node or the third edge node is to execute.
13 . An apparatus to partition a neural network model, the apparatus comprising:
means for determining a computation energy consumption for executing the neural network model on a first edge node, the neural network model corresponding to a platform service with a service level agreement timeframe, including means for determining a transmission energy consumption for sending an intermediate computation of the neural network model to a second edge node or a third edge node;
means for calculating a transmission time for sending the intermediate computation from the first edge node to the second edge node or the third edge node; and
means for dividing the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second edge node or the third edge node based on at least one of the service level agreement timeframe for the platform service, the computation energy consumption, the transmission energy consumption, and the transmission time.
14 . The apparatus of claim 13 , wherein the means for calculating is to calculate the transmission time based on available network bandwidth and a payload size, the payload size including the intermediate computation.
15 . The apparatus of claim 13 , wherein the transmission time is a first transmission time, and the means for calculating is to:
calculate a second transmission time for sending a final result from the second edge node or the third edge node to the first edge node; and
compute a total transmission time based on a sum of the first transmission time and the second transmission time the dividing of the neural network model into the first portion and the second portion based on the total transmission time satisfying the service level agreement timeframe.
16 . The apparatus of claim 13 , wherein the means for determining is to determine at least one of the computation energy consumption or the transmission energy consumption based on received ambient data including at least one of temperature data, wind data, or humidity data.
17 . The apparatus of claim 13 , wherein the means for determining is to:
determine at least the computation energy consumption or the transmission energy consumption based on at least one of an artificial intelligence algorithm, a set of instructions, or a lookup table; and
determine a total power consumption based on a sum of the computation energy consumption and the transmission energy consumption, the dividing of the neural network model into the first portion and the second portion based on the total power consumption satisfying a power consumption threshold.
18 . The apparatus of claim 13 , further including:
means for assigning a first identifier to input data that the neural network model is to process, the means for assigning is to assign a second identifier to the intermediate computation, the second identifier to identify (i) the neural network model that is to be executed at the second edge node or the third edge node (ii) an initial layer of the second portion that the second edge node or the third edge node is to execute;
means for sending the intermediate computation to the second edge node or the third edge node with the second identifier; and
means for determining a final computation based on the intermediate computation and the second identifier, the second identifier to identify the second portion of the neural network model that the second edge node or the third edge node is to execute.
19 . A method comprising:
estimating, by executing an instruction with processor circuitry, a computation energy consumption for executing a neural network model on a first edge node, the neural network model corresponding to a platform service with a service level agreement timeframe;
determining, by executing an instruction with the processor circuitry, a transmission time for sending an intermediate result from the first edge node to a second edge node or a third edge node;
estimating by executing an instruction with the processor circuitry, a transmission energy consumption for sending the intermediate result of the neural network model to the second edge node or the third edge node; and
partitioning, by executing an instruction with the processor circuitry, the neural network model into a first portion to be executed at the first edge node and a second portion to be executed at the second edge node or the third edge node based on at least one of the service level agreement timeframe for the platform service, the computation energy consumption, the transmission energy consumption, or the transmission time.
20 . The method of claim 19 , wherein determining the transmission time is based on available network bandwidth and a payload size, the payload size including the intermediate result.
21 . The method of claim 19 , wherein the transmission time is a first transmission time, and further including:
determining a second transmission time for sending a final result from the second edge node or the third edge node to the first edge node; and
calculating a total transmission time based on a sum of the first transmission time and the second transmission time, the partitioning of the neural network model into the first portion and the second portion based on the total transmission time satisfying the service level agreement timeframe.
22 . The method of claim 19 , wherein estimating the computation energy consumption includes:
receiving temperature data from a temperature sensor;
receiving wind data from a wind sensor; and
receiving humidity data from a humidity sensor, the estimation of at least one of the computation energy consumption or the transmission energy consumption based on at least one of the temperature data, the wind data, or the humidity data.
23 . The method of claim 19 , further including:
estimating at least the computation energy consumption or the transmission energy consumption based on at least one of an artificial intelligence algorithm, a set of instructions, or a lookup table; and
calculating a total power consumption based on a sum of the computation energy consumption and the transmission energy consumption, the partitioning of the neural network model into the first portion and the second portion based on the total power consumption satisfying a power consumption threshold.
24 . The method of claim 19 , further including:
assigning a first identifier to input data that the neural network model is to process;
assigning a second identifier to the intermediate result, the second identifier to identify (i) the neural network model that is to be executed at the second edge node or the third edge node (ii) an initial layer of the second portion that the second edge node or the third edge node is to execute;
sending the intermediate result to the second edge node or the third edge node with the second identifier; and
determining a final result based on the intermediate result and the second identifier, the second identifier to identify the second portion of the neural network model that the second edge node or the third edge node is to execute.