Systems and methods for hardware acceleration of masking and normalizing data with a triangular input mask
A field programmable gate array including a configurable interconnect fabric connecting logic blocks implementing a circuit to: receive input data including data values organized into rows and columns, each row having N data values; select R[i] unmasked data values of a row of the input data in accordance with a mask and an index i of the row; select N−[i] unmasked data values of another row of the input data in accordance with the mask and an index of the another row; merge the R[i] unmasked data values of the row and the N−[i] data values of the another row into a combined data vector of N data values; and compute R[i] normalized values based on the R[i] unmasked data values of the combined data vector and N−[i] normalized values based on the N−[i] data values of the combined data vector to generate N normalized data values.
1 . A method for accelerating computations in a field programmable gate array (FPGA) comprising a configurable interconnect fabric connecting a plurality of logic blocks, the method comprising:
receiving, at a mask-normalize circuit implemented in the logic blocks and configurable interconnect fabric of the FPGA, input data including data values organized into rows and columns at a plurality of indices of the input data, each row having N data values;
selecting, by the mask-normalize circuit, R[i] unmasked data values of a row of the input data in accordance with a mask and an index i of the row;
selecting, by the mask-normalize circuit, N−R[i] unmasked data values of another row of the input data in accordance with the mask and an index of the another row;
merging, by the mask-normalize circuit, the R[i] unmasked data values of the row and the N−R[i] data values of the another row to form a combined data vector of N data values; and
computing, by a data normalization circuit of the mask-normalize circuit comprising a first accumulator and a second accumulator:
R[i] normalized values based on the R[i] unmasked data values in a first sub-vector of the combined data vector, wherein the data normalization circuit accumulates, in the first accumulator, a first denominator based on the R[i] unmasked data values of the first sub-vector and computes the R[i] normalized values based on the first denominator; and
N−R[i] normalized values based on the N−R[i] data values of a second sub-vector of the combined data vector to generate a combined normalized data vector of N normalized data values, wherein the data normalization circuit accumulates, in the second accumulator, a second denominator based on the N−R[i] unmasked data values of the second sub-vector and computes the N−R[i] normalized values based on the second denominator.
2 . The method of claim 1 , wherein the mask is a triangular mask.
3 . The method of claim 1 , further comprising:
padding the input data with a dummy row, wherein all of the data values of the dummy row are masked and wherein the dummy row has a negative index.
4 . The method of claim 1 , wherein the mask-normalize circuit is configured to apply a SoftMax function to the unmasked data values in the first sub-vector of the combined data vector and the second sub-vector of the combined data vector to compute the combined normalized data vector of N normalized values based on a maximum unmasked data value of the combined data vector.
5 . The method of claim 1 , further comprising:
padding the R[i] normalized values corresponding to the first sub-vector at masked-out locations in accordance with the mask and the index of the row and a row of normalized output data; and
padding the N−R[i] normalized values corresponding to the second sub-vector at masked out locations in accordance with the mask and the index of the another row as another row of normalized output data.
6 . The method of claim 1 , wherein the mask-normalize circuit reads the R[i] unmasked data values of the row of the input data via a first port of a device memory of the FPGA in parallel with reading the N−R[i] unmasked data values of the another row of the input data via a second port of the device memory of the FPGA.
7 . The method of claim 1 , wherein the merging the R[i] unmasked data values of the row and the N−R[i] data values of the another row is performed by a bitwise OR circuit of the mask-normalize circuit.
8 . The method of claim 1 , further comprising training a machine learning model, comprising:
receiving, by a machine learning model training application executed by a computing device comprising a processor, memory, and the FPGA, labeled training data;
supplying, by the machine learning model training application, the labeled training data to the machine learning model to generate input data;
supplying, by the machine learning model training application, the input data to the mask-normalize circuit implemented in the FPGA to generate masked and normalized data values as output scores;
updating the machine learning model based on the output scores; and
outputting the updated machine learning model as a trained machine learning model.
9 . Computer storage media storing a configuration file, the configuration file specifying a configuration of a field programmable gate array (FPGA) comprising a configurable interconnect fabric and a plurality of logic blocks, where an FPGA configured based on the configuration file comprises logic blocks, connected by the configurable interconnect fabric, implementing a mask-normalize circuit comprising a data normalization circuit, the mask-normalize circuit being configured to:
receive input data including data values organized into rows and columns at a plurality of indices of the input data, each row having N data values;
select, by a first local masking block, R[i] unmasked data values of a row of the input data in accordance with a mask and an index i of the row;
select, by a second local masking block, N−R[i] unmasked data values of another row of the input data in accordance with the mask and an index of the another row;
merge the R[i] unmasked data values of the row and the N−R[i] data values of the another row to form a combined data vector of N data values; and
compute, by the data normalization circuit comprising a first accumulator and a second accumulator:
R[i] normalized values based on the R[i] unmasked data values in a first sub-vector of the combined data vector, wherein the data normalization circuit accumulates, in the first accumulator, a first denominator based on the R[i] unmasked data values of the first sub-vector and computes the R[i] normalized values based on the first denominator; and
N−R[i] normalized values based on the N−R[i] data values of a second sub-vector of the combined data vector to generate a combined normalized data vector of N normalized data values, wherein the data normalization circuit accumulates, in the second accumulator, a second denominator based on the N−R[i] unmasked data values of the second sub-vector and computes the N−R[i] normalized values based on the second denominator.
10 . The computer storage media of claim 9 , wherein the mask is a triangular mask.
11 . The computer storage media of claim 9 , wherein the configuration file further specifies the configuration of the configurable interconnect fabric and the logic blocks of the FPGA to pad the input data with a dummy row, wherein all of the data values of the dummy row are masked and wherein the dummy row has a negative index.
12 . The computer storage media of claim 9 , wherein the configuration file further specifies the configuration of the configurable interconnect fabric and the logic blocks of the FPGA to implement an exponential circuit configured to apply a SoftMax function to the unmasked data values in the first sub-vector of the combined data vector and the second sub-vector of the combined data vector to compute the combined normalized data vector of N normalized values based on a maximum unmasked data value of the combined data vector.
13 . The computer storage media of claim 9 , wherein the configuration file further specifies the configuration of the configurable interconnect fabric and the logic blocks of the FPGA to implement a row splitting and padding circuit configured to:
pad the R[i] normalized values corresponding to the first sub-vector at masked-out locations in accordance with the mask and the index of the row and a row of normalized output data; and
pad the N−R[i] normalized values corresponding to the second sub-vector at masked out locations in accordance with the mask and the index of the another row as another row of normalized output data.
14 . The computer storage media of claim 9 , wherein the configuration file further specifies the configuration of the configurable interconnect fabric and the logic blocks of the FPGA to connect the first local masking block to a first port of a device memory of the FPGA and to connect the second local masking block to a second port of the device memory of the FPGA.
15 . A field programmable gate array (FPGA) comprising a configurable interconnect fabric connecting a plurality of logic blocks, the configurable interconnect fabric and the logic blocks being configured to implement a mask-normalize circuit comprising a data normalization circuit, the mask-normalize circuit being configured to:
receive input data including data values organized into rows and columns at a plurality of indices of the input data, each row having N data values;
select, by a first local masking block, R[i] unmasked data values of a row of the input data in accordance with a mask and an index i of the row;
select, by a second local masking block, N−R[i] unmasked data values of another row of the input data in accordance with the mask and an index of the another row;
merge the R[i] unmasked data values of the row and the N−R[i] data values of the another row to form a combined data vector of N data values; and
compute, by the data normalization circuit comprising a first accumulator and a second accumulator:
R[i] normalized values based on the R[i] unmasked data values in a first sub-vector of the combined data vector, wherein the data normalization circuit accumulates, in the first accumulator, a first denominator based on the R[i] unmasked data values of the first sub-vector and computes the R[i] normalized values based on the first denominator; and
N−R[i] normalized values based on the N−R[i] data values of a second sub-vector of the combined data vector to generate a combined normalized data vector of N normalized data values, wherein the data normalization circuit accumulates, in the second accumulator, a second denominator based on the N−R[i] unmasked data values of the second sub-vector and computes the N−R[i] normalized values based on the second denominator.
16 . The FPGA of claim 15 , wherein the mask is a triangular mask.
17 . The FPGA of claim 15 , wherein the mask-normalize circuit is further configured to pad the input data with a dummy row, wherein all of the data values of the dummy row are masked and wherein the dummy row has a negative index.
18 . The FPGA of claim 15 , wherein the mask-normalize circuit comprises an exponential circuit configured to apply a SoftMax function to the unmasked data values in the first sub-vector of the combined data vector and the second sub-vector of the combined data vector to compute the combined normalized data vector of N normalized values based on a maximum unmasked data value of the combined data vector.
19 . The FPGA of claim 15 , wherein the mask-normalize circuit comprises a row splitting and padding circuit configured to:
pad the R[i] normalized values corresponding to the first sub-vector at masked-out locations in accordance with the mask and the index of the row and a row of normalized output data; and
pad the N−R[i] normalized values corresponding to the second sub-vector at masked out locations in accordance with the mask and the index of the another row as another row of normalized output data.
20 . The FPGA of claim 15 , wherein the first local masking block is connected to a first port of a device memory of the FPGA and the second local masking block is connected to a second port of the device memory of the FPGA.