Method and apparatus with data processing
A processor-implemented data processing method includes: normalizing input data of an activation function comprising a division operation; determining dividend data corresponding to a dividend of the division operation by reading, from a memory, a value of a first lookup table addressed by the normalized input data; determining divisor data corresponding to a divisor of the division operation by accumulating the dividend data; and determining output data of the activation function corresponding to an output of the division operation obtained by reading, from the memory, a value of a second lookup table addressed by the dividend data and the divisor data.
1 . A processor-implemented data processing method, the method comprising:
based on a precision of an activation function that is determined using a division estimation operation, selecting a first lookup table from a first set of first lookup tables and selecting a second lookup table from a second set of second lookup tables;
determining dividend data corresponding to a dividend of the division estimation operation by reading, from a memory, a value of the selected first lookup table addressed by input data of the activation function;
determining divisor data corresponding to a divisor of the division estimation operation by accumulating the dividend data using an accumulator; and
determining output data of the activation function, the output data comprising a quotient output of the division estimation operation obtained by reading, from the memory, a quotient value from the selected second lookup table.
2 . The method of claim 1 , wherein the input data is normalized data such that a maximum value of the input data is 0.
3 . The method of claim 1 , wherein the determining of the output data comprises:
determining a column-index value based on a value of the dividend data;
determining a row-index value based on a value of the divisor data; and
reading, from the memory, the quotient value from the second lookup table addressed by the column-index value and the row-index value.
4 . The method of claim 3 , wherein
the determining of the row-index value comprises determining the row-index value from a value indicated by a number of most significant bits associated with the division estimation operation.
5 . The method of claim 4 , wherein the determining of the row-index value comprises determining the number of the most significant bits based on a value of a log base 2 of a total number of rows of the second lookup table.
6 . The method of claim 5 , wherein the determining of the number of the most significant bits comprises adding 1 to the value of the log base 2 of the total number of rows of the second lookup table.
7 . The method of claim 3 , wherein the determining of the output data comprises reading the quotient value from the second lookup table by selecting a word line of the memory addressed by the row-index value.
8 . The method of claim 1 , further comprising:
selecting the first lookup table and the second lookup table based on previously generated associations between precisions and lookup tables.
9 . The method of claim 8 , wherein the selecting of the first lookup table and the second lookup table comprises:
selecting the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.
10 . The method of claim 1 , wherein the activation function comprises a Softmax function.
11 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of claim 1 .
12 . A data processing apparatus, the apparatus comprising:
a memory storing instructions; and
one or more processors configured by the instructions to:
based on a precision of an activation function that is determined using a division estimation operation, select a first lookup table from a first set of first lookup tables and select a second lookup table from a second set of second lookup tables;
determine dividend data corresponding to a dividend of the division estimation operation by reading, from the memory, a value of the selected first lookup table addressed by input data of the activation function;
determine divisor data corresponding to a divisor of the division estimation operation by accumulating the dividend data using an accumulator; and
determine output data of the activation function, the output data comprising a quotient output of the division estimation operation obtained by reading, from the memory, a quotient value from the selected second lookup table.
13 . The apparatus of claim 12 , wherein, the input data is normalized data such that a maximum value of the input data after being normalized is 0.
14 . The apparatus of claim 12 , wherein, for the determining of the output data, the one or more processors are configured to:
determine a column-index value based on a value of the dividend data;
determine a row-index value based on a value of the divisor data; and
read, from the memory, the quotient value from the second lookup table addressed by the column-index value and the row-index value.
15 . The apparatus of claim 14 , wherein, for the determining of the row-index value, the one or more processors are configured to determine the row-index value from a value indicated by a number of most significant bits associated with the division estimation operation.
16 . The apparatus of claim 15 , wherein, for the determining of the row-index value, the one or more processors are configured to determine the number of the most significant bits based on a value of a log base 2 of a total number of rows of the second lookup table.
17 . The apparatus of claim 14 , wherein
the memory comprises dynamic random access memory (DRAM) in which the second lookup table is stored, and
for the determining of the output data, the one or more processors are configured to read the quotient value from the second lookup table by selecting a word line of the DRAM addressed by the row-index value.
18 . The apparatus of claim 12 , wherein the one or more processors are configured to:
select the first lookup table and the second lookup table based on previously generated associations between precisions and lookup tables.
19 . The apparatus of claim 18 , wherein, for the selecting of the first lookup table and the second lookup table, the one or more processors are configured to select the first lookup table and the second lookup table such that the first lookup table and the second lookup table correspond to different precisions.
20 . The apparatus of claim 18 , wherein the precision is any one of 10-bit floating point, 32-bit floating point, 8-bit integer, 16-bit integer, and 8-bit unsigned integer.