Image processing device and image processing method
An image processing device includes: a reception circuit receiving image data spread in a height direction and in a horizontal direction; a line memory having a register group for each channel, the resister group being capable of holding data spread in the horizontal direction in units of row along the height direction; a shift data generation circuit generating a plurality of pieces of first intermediate data in which spatial positions including the height direction of the image data are shifted by different shift amounts, and storing them in a plurality of register groups mutually corresponding to a plurality of channels of the line memory; a filtering processing circuit extracting, in the plurality of pieces of first intermediate data, data indicating the maximum value among a plurality of pieces of data having the same spatial position and having different channels; and a pooling processing circuit extracting, in second intermediate data indicating the maximum value among the plurality of pieces of data for each predetermined spatial region, and generating output data.
1 . An image processing device comprising:
a reception circuit receiving image data spread in a height direction and a horizontal direction;
a line memory having a plurality of channels, each channel having a register group configured to hold pixel data spread in the horizontal direction in units of row along the height direction;
a shift data generation circuit generating, for an entirety of the image data, a plurality of pieces of first intermediate data in which a spatial position of the image data is shifted by respective shift amounts that are different from one another in one or both of the height direction and the horizontal direction, the shift data generation circuit storing the plurality of pieces of first intermediate data in a plurality of register groups respectively corresponding to the plurality of channels of the line memory;
a filtering processing circuit extracting data, which indicates a maximum value among a plurality of pieces of data having a same spatial position and having different channels, from the plurality of pieces of first intermediate data stored in the line memory, the filtering processing circuit generating second intermediate data; and
a pooling processing circuit extracting, from the second intermediate data, data indicating the maximum value among the plurality of pieces of data for each predetermined spatial region, and generating output data.
2 . The image processing device according to claim 1 , further comprising a convolution operation circuit, and
wherein the convolution operation circuit is used for the shift data generation circuit.
3 . The image processing device according to claim 2 ,
wherein the shift data generation circuit uses a plurality of kernels, in which the spatial position of the image data is shifted by the respective shift amounts that are different from one another in one or both of the height direction and the horizontal direction, to generate the plurality of pieces of intermediate data from the plurality of pieces of image data.
4 . The image processing device according to claim 3 ,
wherein the plurality of kernels are stored in a predetermined register in advance.
5 . The image processing device according to claim 1 , further comprising a Cross Channel Operation circuit, and
wherein the Cross Channel Operation circuit is used for the filtering processing circuit.
6 . The image processing device according to claim 1 ,
when a Max Pooling processing whose kernel size is K and whose stride is S is needed per direction with respect to the image data, wherein each of K and S is a positive integer, and wherein K is greater than S,
wherein the shift data generation circuit generates K−S+1 pieces of first intermediate data per direction, and
wherein the pooling processing circuit generates the output data by setting, as S, a size of the predetermined spatial region.
7 . An image processing device comprising:
a line memory having a plurality of channels, each channel having a register group configured to hold data spread in a horizontal direction in units of row along a height direction;
a reception circuit receiving a plurality of pieces of first intermediate data in which a spatial position including the height direction of image data is shifted by respective shift amounts that are different from one another in one or both of the height direction and the horizontal direction, the reception circuit storing the plurality of pieces of first intermediate data in a plurality of register groups respectively corresponding to the plurality of channels of the line memory;
a filtering processing circuit extracting data, which indicates a maximum value among a plurality of data having a same spatial position and having different channels, from the plurality of pieces of first intermediate data stored in the line memory, the filtering processing circuit generating second intermediate data; and
a pooling processing circuit extracting, from the second intermediate data, data indicating the maximum value among the plurality of pieces of data for each predetermined spatial region, and generating output data.
8 . The image processing device according to claim 7 ,
wherein the reception circuit is configured to further receive the image data spread in the height direction and in the horizontal direction, and
wherein the image processing device further comprises a shift data generation circuit generating the plurality of pieces of first intermediate data in which a spatial position including the height direction of the image data received by the reception circuit is shifted in the height direction by the respective shift amounts that are different from one another, the shift data generation circuit storing the plurality of pieces of first intermediate data in the plurality of register groups respectively corresponding to the plurality of channels of the line memory.
9 . An image processing method by an image processing device including a reception circuit receiving image data spread in a height direction and in a horizontal direction, a line memory having a plurality of channels, each channel having a register group configured to hold data spread in the horizontal direction in units of row along a height direction, a shift data generation circuit, a filtering processing circuit, and a pooling processing circuit, the image processing method comprising:
receiving the image data by the reception circuit;
generating a plurality of pieces of intermediate data in which a spatial position including the height direction of the image data is shifted by respective shift amounts that are different from one another in one or both of the height direction and the horizontal direction, by the shift data generation circuit;
storing the plurality of pieces of first intermediate data in a plurality of register groups mutually corresponding to a plurality of channels of the line memory, respectively;
extracting, by the filtering processing circuit, data indicating a maximum value among a plurality of pieces of data, which have a same spatial position and have a different channel, among the plurality of pieces of first intermediate data stored in the plurality of resister-register groups of the line memory, to generate second intermediate data; and
extracting, by the pooling processing circuit, data indicating the maximum value among the plurality of pieces data for each predetermined spatial region, to generate output data.
10 . The image processing method according to claim 9 , further comprising generating the plurality of pieces of first intermediate data from the image data by a convolution operation circuit used as the shift data generation circuit.
11 . The image processing method according to claim 10 , further comprising generating the plurality of pieces of first intermediate data, by the convolution operation circuit, from the image data by using a plurality of kernels in which the spatial position of the image data is shifted by the respective shift amounts that are different from one another in one or both of the height direction and the horizontal direction.
12 . The image processing method according to claim 11 , further comprising generating the plurality of pieces of first intermediate data, by the convolution operation circuit, from the image data by using the plurality of kernels stored in a predetermined register in advance.
13 . The image processing method according to claim 9 , further comprising generating the second intermediate data from the plurality of pieces of first intermediate data by a Cross Channel Operation circuit used as the filtering processing circuit.
14 . The image processing method according to claim 9 , further comprising, when a Max Pooling processing whose kernel size is K and whose stride is S is required for the image data per direction, generating the K−S+1 pieces of first intermediate data per direction and the output data in which a size of the predetermined spatial region per direction is set as S,
wherein each of K and S is a positive integer, and
wherein K is greater than S.