METHODS AND APPARATUS FOR SIMILAR DATA REUSE IN DATAFLOW PROCESSING SYSTEMS
A computerized method identifies an input and kernel similarity in binarized neural network (BNN) across different applications as they are being processed by processors such as a GPU. The input and kernel similarity in BNN across different applications are analyzed to reduce computation redundancy to accelerate BNN inference. A computer-executable instructions stored thereon an on-chip arrangement receives a first data value for a data source for processing by the BNN at an inference phase. The computer-executable instructions further receives a second data value for the data source for processing by the BNN at the inference phase. The first data value is processed bitwise operations. A difference between the first data value and the second data value is calculated. The difference is stored in the on-chip arrangement. The computer-executable instructions applies the bitwise operations to the stored difference.
1 . A computerized method for reducing a number of MAC operations at interference time comprising:
receiving a first input data for an input image for processing by a binarized neural network(BNN) at an inference phase;
receiving a second input data for the input image for processing by the BNN at the inference phase;
processing the first input data using bitwise operations;
calculating a difference between the first input data and the second input data;
storing the difference in an on-chip arrangement; and
applying the bitwise operations to the stored difference.
2 . The computerized method of claim 1 , wherein processing the first input data comprises processing the first input data using a graphical processing unit (GPU).
3 . The computerized method of claim 1 , further comprising:
detecting features in the input image using a set of kernels in a convolutional layer;
receiving a first kernel weight for one of the kernels;
receiving a second kernel weight for another of the kernels;
processing the first kernel weight using bitwise operations;
calculating a difference between the first kernel weight and the second kernel weight;
storing a kernel difference in an on-chip arrangement; and
applying the bitwise operations to the stored kernel difference.
4 . The computerized method of claim 3 , further comprising constructing a graph for the kernels, said graph being expressed as G(V, E, W), where each vertex v ∈ V corresponds to one of the kernels, two vertices being connected by link e ∈ E with a weight w ∈ W representing a degree of dissimilarity between two of the kernels.
5 . The computerized method of claim 4 , further comprising partitioning the graph.
6 . The computerized method of claim 5 , wherein partitioning the graph comprises partitioning the graph into subgraphs based a summed weight of links in between the subgraphs.
7 . A computerized method for reducing a number of MAC operations at interference time comprising:
receiving a first data value for a data source for processing by a binarized neural network(BNN) at an inference phase;
receiving a second data value for the data source for processing by the BNN at the inference phase;
processing the first data value using bitwise operations;
calculating a difference between the first data value and the second data value;
storing the difference in an on-chip arrangement; and
applying the bitwise operations to the stored difference.
8 . The computerized method of claim 7 , wherein processing the first input data comprises processing the first input data using a graphical processing unit (GPU).
9 . The computerized method of claim 7 , wherein the data source comprises an input image, wherein the first data value comprises a first input data of the input image and wherein the second data value comprises a second input data of the input image.
10 . The computerized method of claim 9 , wherein the data source comprises a set of kernels used in a convolutional layer for detecting features of the input image, wherein the first data value comprises a first kernel weight, and wherein the second data value comprises a second kernel weight.
11 . The computerized method of claim 10 , further comprising constructing a graph for the kernels, said graph being expressed as G(V, E, W), where each vertex v ∈ V corresponds to one of the kernels, two vertices being connected by link e ∈ E with a weight w ∈ W representing a degree of dissimilarity between two of the kernels.
12 . The computerized method of claim 11 , further comprising partitioning the graph.
13 . The computerized method of claim 11 , wherein partitioning the graph comprises partitioning the graph into subgraphs based a summed weight of links in between the subgraphs.
14 . A computer-executable instructions stored thereon an on-chip arrangement for reducing a number of MAC operations at interference time comprising:
receiving a first data value for a data source for processing by a binarized neural network(BNN) at an inference phase;
receiving a second data value for the data source for processing by the BNN at the inference phase;
processing the first data value using bitwise operations;
calculating a difference between the first data value and the second data value;
storing the difference in the on-chip arrangement; and
applying the bitwise operations to the stored difference.
15 . The computer-executable instructions of claim 14 , wherein processing the first input data comprises processing the first input data using a graphical processing unit (GPU).
16 . The computer-executable instructions of claim 7 , wherein the data source comprises an input image, wherein the first data value comprises a first input data of the input image and wherein the second data value comprises a second input data of the input image.
17 . The computer-executable instructions of claim 9 , wherein the data source comprises a set of kernels used in a convolutional layer for detecting features of the input image, wherein the first data value comprises a first kernel weight, and wherein the second data value comprises a second kernel weight.
18 . The computer-executable instructions of claim 10 , further comprising constructing a graph for the kernels, said graph being expressed as G(V, E, W), where each vertex v ∈ V corresponds to one of the kernels, two vertices being connected by link e ∈ E with a weight w ∈ W representing a degree of dissimilarity between two of the kernels.
19 . The computer-executable instructions claim 11 , further comprising partitioning the graph.
20 . The computer-executable instructions of claim 11 , wherein partitioning the graph comprises partitioning the graph into subgraphs based a summed weight of links in between the subgraphs.