Efficient convolutional engine
A hardware architecture for implementing a convolutional neural network. Certain ones of the convolver units may be controlled to be active and others may be controlled to be non-active by a controller in order to perform convolution with a striding of greater than or equal to two.
1. An apparatus, comprising:
a controller;
a two-dimensional synchronous shift register comprising a three by four array of data storage elements:
[
d
1
,
1
d
1
,
2
d
1
,
3
d
1
,
4
d
2
,
1
d
2
,
2
d
2
,
3
d
2
,
4
d
3
,
1
d
3
,
2
d
3
,
3
d
3
,
4
]
wherein:
data storage element d 1,1 is electrically coupled to data storage element d 2,1 ,
data storage element d 2,1 is electrically coupled to data storage element d 3,1 ,
data storage element d 1,2 is electrically coupled to data storage element d 2,2 ,
data storage element d 2,2 is electrically coupled to data storage element d 3,2 ,
data storage element d 1,3 is electrically coupled to data storage element d 2,3 ,
data storage element d 2,3 is electrically coupled to data storage element d 3,3 ,
data storage element d 1,4 is electrically coupled to data storage element d 2,4 , and
data storage element d 2,4 is electrically coupled to data storage element d 3,4 ,
a first convolver unit comprising a first plurality of multipliers, m 1,1 1 , m 1,2 1 , m 1,3 1 , m 2,1 1 , m 2,2 1 , m 2,3 1 , m 3,1 1 , m 3,2 1 , and m 3,3 1 , wherein:
multiplier m 1,1 1 is electrically coupled in a permanent manner to data storage element d 1,1 ,
multiplier m 1,2 1 is electrically coupled in a permanent manner to data storage element d 1,2 ,
multiplier m 1,3 1 is electrically coupled in a permanent manner to data storage element d 1,3 ,
multiplier m 2,1 1 is electrically coupled in a permanent manner to data storage element d 2,1 ,
multiplier m 2,2 1 is electrically coupled in a permanent manner to data storage element d 2,2 ,
multiplier m 2,3 1 is electrically coupled in a permanent manner to data storage element d 2,3 ,
multiplier m 3,1 1 is electrically coupled in a permanent manner to data storage element d 3,1 ,
multiplier m 3,2 1 is electrically coupled in a permanent manner to data storage element d 3,2 , and
multiplier m 3,3 1 is electrically coupled in a permanent manner to data storage element d 3,3 ; and
a second convolver unit comprising a second plurality of multipliers, m 1,1 2 , m 1,2 2 , m 1,3 2 , m 2,1 2 , m 2,2 2 , m 2,3 2 , m 3,1 2 , m 3,2 2 , and m 3,3 2 , wherein:
multiplier m 1,1 2 is electrically coupled in a permanent manner to data storage element d 1,2 ,
multiplier m 1,2 2 is electrically coupled in a permanent manner to data storage element d 1,3 ,
multiplier m 1,3 2 is electrically coupled in a permanent manner to data storage element d 1,4 ,
multiplier m 2,1 2 is electrically coupled in a permanent manner to data storage element d 2,2 ,
multiplier m 2,2 2 is electrically coupled in a permanent manner to data storage element d 2,3 ,
multiplier m 2,3 2 is electrically coupled in a permanent manner to data storage element d 2,4 ,
multiplier m 3,1 2 is electrically coupled in a permanent manner to data storage element d 3,2 ,
multiplier m 3,2 2 is electrically coupled in a permanent manner to data storage element d 3,3 , and
multiplier m 3,3 2 is electrically coupled in a permanent manner to data storage element d 3,4 ,
wherein, at a start of a first clock cycle:
data storage element d 1,1 is configured to store data value x 1,1 ,
data storage element d 1,2 is configured to store data value x 1,2 ,
data storage element d 1,3 is configured to store data value x 1,3 ,
data storage element d 1,4 is configured to store data value x 1,4 ,
data storage element d 2,1 is configured to store data value x 2,1 ,
data storage element d 2,2 is configured to store data value x 2,2 ,
data storage element d 2,3 is configured to store data value x 2,3 ,
data storage element d 2,4 is configured to store data value x 2,4 ,
data storage element d 3,1 is configured to store data value x 3,1 ,
data storage element d 3,2 is configured to store data value x 3,2 ,
data storage element d 3,3 is configured to store data value x 3,3 , and
data storage element d 3,4 is configured to store data value x 3,4 ,
wherein, during the first clock cycle, the first convolver unit is controlled by the controller to be active causing:
multiplier m 1,1 1 , to multiply data value x 1,1 with weight w 1 so as to generate a product w 1 x 1,1 ,
multiplier m 1,2 1 to multiply data value x 1,2 with weight w 2 so as to generate a product w 2 x 1,2 ,
multiplier m 1,3 1 to multiply data value x 1,3 with weight w 3 so as to generate a product w 3 x 1,3 ,
multiplier m 2,1 1 to multiply data value x 2,1 with weight w 4 so as to generate a product w 4 x 2,1 ,
multiplier m 2,2 1 to multiply data value x 2,2 with weight w 5 so as to generate a product w 5 x 2,2 ,
multiplier m 2,3 1 to multiply data value x 2,3 with weight w 6 so as to generate a product w 6 x 2,3 ,
multiplier m 3,1 1 to multiply data value x 3,1 with weight w 7 so as to generate a product w 7 x 3,1 ,
multiplier m 3,2 1 to multiply data value x 3,2 with weight w 8 so as to generate a product w 8 x 3,2 ,
multiplier m 3,3 1 to multiply data value x 3,3 with weight w 9 so as to generate a product w 9 x 3,3 , and
an adder to compute a first sum of terms including at least the product w 1 x 1,1 , the product w 2 x 1,2 , the product w 3 x 1,3 , the product w 4 x 2,1 , the product w 5 x 2,2 , the product w 6 x 2,3 , the product w 7 x 3,1 , the product w 8 x 3,2 , and the product w 9 x 3,3 , and
wherein, during the first clock cycle, the second convolver unit is controlled by the controller to be non-active.
2. The apparatus of claim 1 , wherein the first sum of terms additionally includes b 1 , wherein b 1 is a bias value.