Near-memory operator and method with accelerator performance improvement
A system configured to perform an operation includes: a hardware device comprising a plurality of computing modules and a plurality of memory modules arranged in a lattice form, each of the computing modules comprising a coarse-grained reconfigurable array and each of the memory modules comprising a static random-access memory and a plurality of functional units connected to the static random-access memory; and a compiler configured to divide a target operation and assign the divided target operation to the computing modules and the memory modules such that the computing modules and the memory modules of the hardware device perform the target operation.
1 . A system configured to perform an operation, the system comprising:
a hardware device comprising a plurality of computing modules and a plurality of memory modules arranged in a lattice form, each of the computing modules comprising a coarse-grained reconfigurable array and each of the memory modules comprising a static random-access memory and a plurality of functional units connected to the static random-access memory; and
a compiler configured to divide a target operation and assign the divided target operation to the computing modules and the memory modules such that the computing modules and the memory modules of the hardware device perform the target operation.
2 . The system of claim 1 , wherein, for each of the memory modules, a functional unit comprised in the memory module is individually connected to the static random-access memory of the memory module through a wire and configured to perform either one or both of an operation and a transmitting of a result of the operation to another functional unit comprised in the memory module.
3 . The system of claim 1 , wherein the compiler is configured to, in response to the target operation corresponding to a matrix multiplication operation between an input matrix and a weight matrix, tile each of the input matrix and the weight matrix to a length corresponding to the number of elements processible by one of the computing modules of the hardware device.
4 . The system of claim 3 , wherein the compiler is further configured to:
generate a plurality of big tiles by grouping a plurality of tiles comprised in the weight matrix by a total number of the computing modules comprised in the hardware device; and
input each of a plurality of tiles comprised in one of the big tiles to one of the computing modules, input a plurality of tiles comprised in a partial matrix of the input matrix corresponding to the big tile to the computing modules, and control the computing modules such that the computing modules generate a plurality of partial sums corresponding to the big tile by assigning an element-wise multiplication operation to the computing modules.
5 . The system of claim 4 , wherein the compiler is further configured to assign a reduction operation for summing two or more partial sums to the memory modules.
6 . The system of claim 1 , wherein, for the dividing the target operation and assigning the divided target operation, the compiler is further configured to divide the target operation into a plurality of sub-operations and assign each of the divided sub-operations to either one or both of the computing modules and the memory modules.
7 . The system of claim 1 , wherein, for the dividing the target operation and assigning the divided target operation, the compiler is further configured to:
divide the target operation into a plurality of sub-operations; and
for each of the divided sub-operations, assign the sub-operation to either one or both of the computing modules and the memory modules, based on determining whether the sub-operation corresponds to an input parallel.
8 . The system of claim 7 , wherein, for the assigning the sub-operation to either one or both of the computing modules and the memory modules, the compiler is further configured to:
in response to the sub-operation corresponding to an input parallel, assign the sub-operation to the computing modules; and
in response to the sub-operation not corresponding to an input parallel, assign the sub-operation either one or both of the computing modules and the memory modules, based on determining either one or both of whether the sub-operation is performable in a memory module and which module corresponds to the sub-operation by using a cost model.
9 . The system of claim 8 , wherein, for the assigning the sub-operation to either the computing modules or the memory modules, the compiler is further configured to:
in response to the sub-operation being performable in a memory module and a module corresponding to the sub-operation being determined to be a memory module by using the cost model, assign the sub-operation to the memory modules; and
in response to either one or both of the sub-operation not being performable in a memory module and a module corresponding to the sub-operation being determined to be a computing module by using the cost model, assign the sub-operation to the computing modules.
10 . The system of claim 6 , wherein, for the dividing the target operation and assigning the divided target operation, the compiler is further configured to determine computing modules of the computing modules to perform a second sub-operation of the sub-operations such that the second sub-operation that is dependent on a first sub-operation of sub-operations assigned to the memory modules and assigned to the computing modules is performed in computing modules that are adjacent to the memory modules configured to perform the first sub-operation.
11 . A processor-implemented method of performing an operation, the method comprising:
dividing a target operation and assigning the divided target operation to a plurality of computing modules and a plurality of memory modules of a hardware device, such that the computing modules and the memory modules of the hardware device perform the target operation,
wherein the computing modules and the memory modules are arranged in a lattice form, each of the computing modules comprises a coarse-grained reconfigurable array, and each of the memory modules comprises a static random-access memory and a plurality of functional units connected to the static random-access memory.
12 . The method of claim 11 , wherein, for each of the memory modules, a functional unit comprised in the memory module is individually connected to the static random-access memory of the memory module through a wire and configured to perform either one or both of an operation and a transmitting of a result of the operation to another functional unit comprised in the memory module.
13 . The method of claim 11 , further comprising, in response to the target operation corresponding to a matrix multiplication operation between an input matrix and a weight matrix, tiling each of the input matrix and the weight matrix to a length corresponding to the number of elements processible by one of the computing modules of the hardware device.
14 . The method of claim 13 , further comprising:
generating a plurality of big tiles by grouping a plurality of tiles comprised in the weight matrix by a total number of the computing modules comprised in the hardware device; and
inputting each of a plurality of tiles comprised in one of the big tiles to of the computing modules, inputting a plurality of tiles comprised in a partial matrix of the input matrix corresponding to the big tile to the computing modules, and controlling the computing modules such that the computing modules generate a plurality of partial sums corresponding to the big tile by assigning an element-wise multiplication operation to the computing modules.
15 . The method of claim 14 , further comprising assigning a reduction operation for summing two or more partial sums to the memory modules.
16 . The method of claim 11 , wherein the dividing the target operation and assigning the divided target operation comprises:
dividing the target operation into a plurality of sub-operations; and
assigning each of the divided sub-operations to either one or both of the computing modules and the memory modules.
17 . The method of claim 11 , wherein the dividing the target operation and assigning the divided target operation further comprises:
dividing the target operation into a plurality of sub-operations; and
for each of the divided sub-operations, assigning the sub-operation to either one or both of the computing modules and the memory modules, based on determining whether the sub-operation corresponds to an input parallel.
18 . The method of claim 17 , wherein the assigning the sub-operation to either one or both of the computing modules and the memory modules comprises:
in response to the sub-operation corresponding to an input parallel, assigning the sub-operation to the computing modules; and
in response to the sub-operation not corresponding to an input parallel, assigning the sub-operation either one or both of the computing modules and the memory modules, based on determining either one or both of whether the sub-operation is performable in a memory module and which module corresponds to the sub-operation by using a cost model.
19 . The method of claim 18 , wherein the assigning the sub-operation to either the computing modules or the memory modules further comprises:
in response to the sub-operation being performable in a memory module, and a module corresponding to the sub-operation is determined to be a memory module by using the cost model, assigning the sub-operation to the memory modules, and
in response to either one or both of the sub-operation not being performable in a memory module and a module corresponding to the sub-operation is determined to be a computing module by using the cost model, assigning the sub-operation to the computing modules.
20 . The method of claim 16 , wherein the dividing the target operation and assigning the divided target operation further comprises determining computing modules to perform a second sub-operation such that the second sub-operation that is dependent on a first sub-operation assigned to the memory modules and assigned to the computing modules are performed in computing modules that are adjacent to the memory modules configured to perform the first sub-operation.