Operation processing circuit and recognition system
Parallel processing suitable for convolution operation in a convolutional neural network is executed. Each of a plurality of selectors sequentially selects data, which is held in a two-dimensional shift register, in predetermined two-dimensional regions at least partially different from each other. Each of a plurality of two-dimensional convolution operation circuits multiplies the data selected by the corresponding selector by coefficient data stored in a coefficient memory and accumulates results of the multiplication to calculate, in parallel, two-dimensional convolution operation results in the two-dimensional regions. Each of a plurality of adder circuits adds operation results of the two-dimensional convolution operation circuits in a channel direction to output a three-dimensional convolution operation result.
1. An operation processing circuit comprising:
a two-dimensional shift register including a plurality of shift registers arranged in a first direction that are lined up and sequentially connected in a second direction perpendicularly intersecting the first direction;
a plurality of selectors that sequentially select data in predetermined two-dimensional regions at least partially different from each other from data held in the two-dimensional shift register;
a coefficient memory that stores coefficient data corresponding to the data held in the two-dimensional shift register;
a plurality of two-dimensional convolution operation circuits provided corresponding to the plurality of selectors and configured to multiply the data selected by the selectors by the coefficient data stored in the coefficient memory and accumulate results of the multiplication to calculate, in parallel, two-dimensional convolution operation results in the two-dimensional regions; and
a plurality of adder circuits provided corresponding to the plurality of two-dimensional convolution operation circuits and configured to add operation results of the plurality of two-dimensional convolution operation circuits in a channel direction to output a three-dimensional convolution operation result.
2. The operation processing circuit according to claim 1 , wherein
the coefficient memory stores a plurality of types of the coefficient data, and
each of the plurality of two-dimensional convolution operation circuits continuously performs the two-dimensional convolution operation of the data selected by the plurality of selectors and the plurality of types of the coefficient data without replacing the data held in the two-dimensional shift register.
3. The operation processing circuit according to claim 1 , wherein
the coefficient memory stores a plurality of types of the coefficient data, and
each of the plurality of two-dimensional convolution operation circuits performs, in parallel, the two-dimensional convolution operation for the plurality of types of the coefficient data.
4. The operation processing circuit according to claim 1 , wherein
each of the plurality of two-dimensional convolution operation circuits further performs the two-dimensional convolution operation in parallel for the data in the two-dimensional regions different from each other of the data held in the two-dimensional shift register.
5. The operation processing circuit according to claim 1 , further comprising:
a two-dimensional convolution operation result holding unit that holds operation results of the plurality of two-dimensional convolution operation circuits, wherein
the plurality of adder circuits add, in the channel direction, the operation results of the plurality of two-dimensional convolution operation circuits and the operation results held in the two-dimensional convolution operation result holding unit.
6. The operation processing circuit according to claim 1 , wherein
the plurality of adder circuits add the operation results of the plurality of two-dimensional convolution operation circuits in parallel in the channel direction to output a three-dimensional convolution operation result.
7. The operation processing circuit according to claim 1 , further comprising:
an activation process circuit that applies a predetermined activation process to the three-dimensional convolution operation result output from the plurality of adder circuits.
8. The operation processing circuit according to claim 1 , further comprising:
a pooling process circuit that applies a pooling process to the three-dimensional convolution operation result output from the plurality of adder circuits.
9. The operation processing circuit according to claim 8 , wherein
the pooling process includes a process of generating a pooling value of at least one of a sum total, an average, or a maximum value of a plurality of values in the three-dimensional convolution operation result.
10. The operation processing circuit according to claim 8 , wherein
the two-dimensional shift register holds equal to or more than (m×k w +p w −1)+1)×((k h +p h −1)+1) pieces of data,
the plurality of two-dimensional convolution operation circuits include (m×p w ×p h ) multipliers that perform the multiplication and (m×p w ×p h ) accumulators that perform the accumulation to perform, in parallel, the two-dimensional convolution operation of a (k w ×k h ) size for (p w ×p h ) two-dimensional regions, and
the pooling process circuit uses (m×p w ×p h ) three-dimensional convolution operation results as input data to generate m pooling values, m being an integer equal to or greater than 1, and k w , k h , p w , and p h are integers equal to or greater than 2.
11. The operation processing circuit according to claim 10 , wherein
the plurality of shift registers of the two-dimensional shift register include registers wired in the first direction so that there are (p w −1) registers between the registers, and the plurality of shift registers perform shift operation in the first direction on a basis of P w registers.
12. The operation processing circuit according to claim 8 , wherein
the data held in the two-dimensional shift register is used as an input to execute a feature amount extraction process of a convolutional layer and a pooling layer in a convolutional neural network.
13. A recognition system comprising:
a data acquisition unit that acquires data to be processed; and
an operation processing circuit that uses the data as an input to execute a feature amount extraction process of a convolutional layer and a pooling layer in a convolutional neural network,
the operation processing circuit including:
a two-dimensional shift register including a plurality of shift registers arranged in a first direction that are lined up and sequentially connected in a second direction perpendicularly intersecting the first direction;
a plurality of selectors that sequentially select data in predetermined two-dimensional regions at least partially different from each other from data held in the two-dimensional shift register;
a coefficient memory that stores coefficient data corresponding to the data held in the two-dimensional shift register;
a plurality of two-dimensional convolution operation circuits provided corresponding to the plurality of selectors and configured to multiply the data selected by the selectors by the coefficient data stored in the coefficient memory and accumulate results of the multiplication to calculate, in parallel, two-dimensional convolution operation results in the two-dimensional regions;
a plurality of adder circuits provided corresponding to the plurality of two-dimensional convolution operation circuits and configured to add operation results of the plurality of two-dimensional convolution operation circuits in a channel direction to output a three-dimensional convolution operation result; and
a pooling process circuit that applies a pooling process to the three-dimensional convolution operation result output from the plurality of adder circuits, wherein
the recognition system repeats operation of causing the two-dimensional shift register to hold the data to execute the process in the operation processing circuit to output the three-dimensional convolution operation result as a feature amount of the data and then causing the two-dimensional shift register to hold the output feature amount again to execute the process in the operation processing circuit to output the three-dimensional convolution operation result as a new feature amount.