Accelerator, method of operating the same, and electronic device including the same
View Patent ↗A processor-implemented accelerator method includes: reading, from a memory, an instruction to be executed in an accelerator; reading, from the memory, input data based on the instruction; and performing, on the input data and a parameter value included in the instruction, an inference task corresponding to the instruction.
1 . A processor-implemented method of operating an electronic device, the method comprising:
reading an instruction to be executed in an accelerator included in the electronic device, the instruction embedding a parameter value of at least some portion of layers in a neural network for an inference task, wherein the reading of the instruction comprises reading a plurality of instructions each including a partial value of the parameter value to be used in the inference task, wherein a maximum length of the partial value of the parameter value included in each of the plurality of instructions is less than a preset threshold value;
reading input data based on the instruction;
reading, from the instruction, the parameter value for the inference task; and
performing, on the input data and the parameter value embedded in the instruction, the inference task instructed by the instruction, and
wherein the parameter value is a value fixed for the inference task.
2 . The method of claim 1 , wherein the instruction is determined by substituting, with the parameter value, an indicator of the parameter value included in an initial code.
3 . The method of claim 2 , wherein
the indicator included in the initial code uses a loop variable index, and
the loop variable index is converted to an invariable index through loop unrolling.
4 . The method of claim 1 , wherein the instruction is determined such that a same parameter value is used in a thread block including a plurality of threads to be processed in the accelerator.
5 . The method of claim 4 , wherein an operation unit included in the accelerator and having an instruction cache is configured to process the thread block.
6 . The method of claim 1 ,
wherein the parameter value is determined based on each of the partial values respectively included in a corresponding one of the plurality of instructions.
7 . The method of claim 6 , wherein each of the plurality of instructions includes information indicating which part of the parameter value corresponds to the partial value of the parameter value included in the respective instruction.
8 . The method of claim 6 , wherein
the plurality of instructions includes a first instruction including a mantissa part of the parameter value and a second instruction including an exponent part of the parameter value, and
the performing of the inference task comprises performing a multiplication operation on the mantissa part in response to the first instruction being read, and performing an addition operation on the exponent part in response to the second instruction being read.
9 . The method of claim 1 , wherein
a maximum length of a parameter value portion of an instruction is less than a length of the parameter value,
the parameter value is stored in a memory as a plurality of parameter value portions, each of the parameter value portions being included in a respective instruction,
the reading of the instruction comprises reading the respective instructions, and
the parameter value is determined based on the parameter value portions.
10 . The method of claim 1 , wherein the parameter value is a parameter included in a neural network, and further comprising performing any one of speech recognition, machine translation, machine interpretation, object recognition, and pattern recognition based on a result of the performing of the inference task.
11 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of claim 1 .
12 . An accelerator comprising:
at least one processing element configured to:
read an instruction to be executed in an accelerator, the instruction embedding a parameter value of at least some portion of layers in a neural network for an inference task, wherein the reading of the instruction comprises reading a plurality of instructions each including a partial value of the parameter value to be used in the inference task, wherein a maximum length of the partial value of the parameter value included in each of the plurality of instructions is less than a preset threshold value;
read input data based on the instruction;
reading, from the instruction, the parameter value for the inference task; and
perform, on the input data and the parameter value embedded in the instruction, the inference task instructed by the instruction, and
wherein the parameter value is a value fixed for the inference task.
13 . The accelerator of claim 12 , wherein the instruction is determined by substituting, with the parameter value, an indicator of the parameter value included in an initial code.
14 . The accelerator of claim 13 , wherein
the indicator included in the initial code uses a loop variable index, and
the loop variable index is converted to an invariable index through loop unrolling.
15 . The accelerator of claim 12 , wherein the instruction is determined such that a same parameter value is used in a thread block including a plurality of threads to be processed in the accelerator.
16 . The accelerator of claim 15 , wherein an operation unit included in the accelerator and having an instruction cache is configured to process the thread block.
17 . The accelerator of claim 12 ,
wherein the parameter value is determined based on each of the partial values respectively included in a corresponding one of the plurality of instructions.
18 . The accelerator of claim 17 , wherein each of the plurality of instruction includes information indicating which part of the parameter value corresponds to the partial value included in the respective instruction.
19 . An electronic device comprising a memory storing the instruction and the accelerator of claim 13 .
20 . An electronic device comprising:
a memory configured to store an instruction to be executed in an accelerator; and
the accelerator configured to:
read the instruction from the memory, the instruction embedding a parameter value of at least some portion of layers in a neural network for an inference task,
wherein the reading of the instruction comprises reading a plurality of instructions each including a partial value of the parameter value to be used in the inference task, wherein a maximum length of the partial value of the parameter value included in each of the plurality of instructions is less than a preset threshold value;
read input data based on the instruction;
read, from the instruction, the parameter value for the inference task; and
perform, on the input data and the parameter value embedded in the instruction, the inference task instructed by the instruction, and
wherein the parameter value is a value fixed for the inference task.