Artificial neural network module for performing artificial neural network operation on plurality of subgraphs and operating method thereof
A method for an artificial neural network operation on a plurality of subgraphs may include generating a resource determination trigger corresponding to a target subgraph among the plurality of subgraphs included in a target neural network model; generating a control signal for hardware allocated to the target subgraph and driving resource settings in response to the resource determination trigger; changing, based on the control signal, at least one of hardware allocated to the target subgraph and driving resource settings; and performing an operation on the target subgraph based on the changed hardware and driving resource settings.
1 . A method for an artificial neural network operation, the method comprising:
generating a resource determination trigger corresponding to a target subgraph among a plurality of subgraphs included in a target neural network model;
generating, in response to the resource determination trigger, a control signal, the control signal including information regarding hardware allocated to the target subgraph and driving resource settings;
changing, based on the control signal, at least one of an allocation of the hardware allocated to the target subgraph or the driving resource settings; and
performing an operation on the target subgraph based on at least one of the changed allocation of the hardware or the changed driving resource settings,
wherein generating the resource determination trigger comprises
calculating an expected latency for the at least one of the changed allocation of the hardware or the changed driving resource settings and
determining, based on the expected latency being greater than a threshold latency, a generation time of the resource determination trigger earlier by the expected latency from a time at which the resource determination trigger is planned to be generated,
wherein the time at which the resource determination trigger is planned to be generated is a time point immediately before an operation on the target subgraph is performed or a time point after an operation on a previous subgraph of the target subgraph is completed.
2 . The method of claim 1 , wherein the generating of the control signal comprises determining the allocation of the hardware and the driving resource settings based on bound state information regarding the hardware in which a data throughput is concentrated, wherein the bound state information comprises information about a subsequent subgraph to which the data is transmitted.
3 . The method of claim 2 , wherein the determining of the allocation of the hardware and the driving resource settings comprises
determining the allocation of the hardware and the driving resource settings corresponding to a computing-bound state, wherein the computing-bound state is a state in which data are oriented towards an operation subgraph, when the data throughput is concentrated in computing hardware; and
determining the allocation of the hardware and the driving resource settings corresponding to an input-bound state or an output-bound state when the data throughput is concentrated in input/output hardware, wherein the input-bound state is a state in which data is going to be received by an input/output subgraph, and wherein the output-bound state is a state in which data is going to be transmitted away from the input/output subgraph.
4 . The method of claim 1 , wherein the driving resource setting includes at least one of hardware dynamic voltage frequency scaling level information, signal transmission bandwidth information, last level cache allocation information, temperature control information, or power management integrated circuit control information.
5 . The method of claim 1 , wherein the generating of the control signal comprises determining the allocation of the hardware and the driving resource settings, based on metadata corresponding to each subgraph.
6 . The method of claim 5 , wherein the metadata includes at least one of preference driving information, oriented task information, tensor information, or expected latency for each subgraph.
7 . The method of claim 6 , wherein the metadata includes, for each of the plurality of subgraphs, information regarding a bound state in which a data throughput is concentrated.
8 . The method of claim 5 , wherein
the plurality of subgraphs includes an operation subgraph in which operation processing is concentrated and an input/output subgraph in which data is to be received or to which data is to be transmitted, and
the generating of the resource determination trigger comprises outputting the resource determination trigger corresponding to the operation subgraph.
9 . The method of claim 1 , wherein the generating of the control signal comprises determining the driving resource settings corresponding to the target subgraph based on situation-specific driving resource settings set for each hardware to be allocated.
10 . An artificial neural network module configured to perform an artificial neural network operation, the artificial neural network module comprising:
processing circuitry configured to
generate a resource determination trigger corresponding to a target subgraph among a plurality of subgraphs included in a target neural network model based on a clock signal;
generate, in response to the resource determination trigger, a control signal, the control signal including information regarding hardware allocated to the target subgraph and driving resource settings;
change, based on the control signal, at least one of an allocation of hardware allocated to the target subgraph or the driving resource settings; and
perform an operation on the target subgraph based on the at least one of the changed allocation of the hardware or the changed driving resource settings,
wherein the process circuitry is configured to
generate the resource determination trigger by calculating an expected latency for the at least one of the changed allocation of the hardware or the changed driving resource settings and
determine, based on the expected latency being greater than a threshold latency, a generation time of the resource determination trigger earlier by the expected latency from a time at which the resource determination trigger is planned to be generated,
wherein the time at which the resource determination trigger is planned to be generated is a time point immediately before an operation on the target subgraph is performed or a time point after an operation on a previous subgraph of the target subgraph is completed.
11 . The artificial neural network module of claim 10 , wherein the processing circuitry is configured to:
collect data processing information of a plurality of hardware;
generate computing data based on the data processing information; and
determine the at least one of hardware or the driving resource settings based on the computing data.
12 . The artificial neural network module of claim 10 , wherein the processing circuitry is configured to determine the hardware and the driving resource settings based on bound state information related to the hardware in which a data throughput is concentrated.
13 . The artificial neural network module of claim 10 , wherein the driving resource setting includes at least one of hardware dynamic voltage frequency scaling level information, signal transmission bandwidth information, last level cache allocation information, temperature control information, or power management integrated circuit control information.
14 . The artificial neural network module of claim 10 , wherein the processing circuitry is configured to determine the allocation of the hardware and the driving resource settings based on metadata corresponding to each subgraph.
15 . The artificial neural network module of claim 14 , wherein the metadata includes at least one of preference driving information, oriented task information, tensor information, or expected latency set for each of the plurality of subgraphs.
16 . The artificial neural network module of claim 15 , wherein
the plurality of subgraphs comprises an operation subgraph in which operation processing is concentrated and an input/output subgraph in which data is to be received or to which data is to be transmitted, and
the processing circuitry is configured to output the resource determination trigger corresponding to the operation subgraph.
17 . The artificial neural network module of claim 10 , wherein the processing circuitry is configured to determine the driving resource settings corresponding to the target subgraph based on situation-specific driving resource settings set for each hardware to be allocated.
18 . An electronic device comprising:
a computing device configured to perform an operation corresponding to a plurality of subgraphs constituting a target neural network model;
an input/output device configured to provide data to the computing device and receive data generated by the computing device; and
an artificial neural network module including processing circuitry configured to change at least one of driving resource settings or allocation of the computing device and the input/output device in response to a resource determination trigger corresponding to a target subgraph among the plurality of subgraphs,
wherein the resource determination trigger is generated by calculating an expected latency for the at least one of the changed driving resource settings or the changed allocation of the computing device and the input/output device and determining, based on the expected latency being greater than a threshold latency, a generation time of the resource determination trigger earlier by the expected latency from a time at which the resource determination trigger is planned to be generated, and
wherein the time at which the resource determination trigger is planned to be generated is a time point immediately before an operation on the target subgraph is performed or a time point after an operation on a previous subgraph of the target subgraph is completed.