Methods and apparatus for image processing at pixel rate
View Patent ↗Embodiments of the present invention provide for improved timing control in 2-D image processing to maintain a constant rate of memory fetches and pixel outputs even when the processing operations transition to a new line or frame of pixels. A one-to-one relationship between incoming pixel rate and outgoing pixel rate is maintained without additional clock cycles or memory bandwidth as an improved timing control according to the present invention takes advantage of idle memory bandwidth by pre-fetching a new column of pixel data in a first pixel block of a next line or frame while a new column of an edge pixel block on a current line is duplicated or zeroed out. As the edge pixel block(s) on the current line are processed, the data in the first pixel block of the next line or frame become ready for computation without extra clock cycles or extra memory bandwidth.
1. A method for processing one or more images having a plurality of pixels, the method comprising:
processing successive pixel blocks in raster mode between a first edge and a second edge of an image;
during each single clock unit when a next pixel block to be processed does not extend beyond said second edge,
fetching, from a storage device, one new vector of the next pixel block; and
during each single clock unit when said next pixel block to be processed extends beyond said second edge,
filling a new vector in said next pixel block that is located beyond said second edge with predetermined pixel values, and
fetching, from said storage device, one new vector of a first pixel block on a next line or frame of pixels; and
outputting, in said each single clock unit, one output pixel value, thereby maintaining a constant rate of memory fetches and pixel outputs.
2. The method according to claim 1 , wherein said processing of successive pixel blocks comprises applying an n×m kernel matrix to said successive pixel blocks, n and m being odd integers.
3. The method according to claim 1 , wherein said processing of successive pixel blocks comprises two-dimensional convolution operations.
4. The method according to claim 1 , wherein said processing of successive pixel blocks comprises two-dimensional correlation operations.
5. The method according to claim 1 , wherein said predetermined pixel values are zeros.
6. The method according to claim 1 , wherein said predetermined pixel values are pixel values of a corresponding portion of said second edge.
7. An apparatus for processing one or more images having a plurality of pixels, the apparatus comprising:
means for processing successive pixel blocks in raster mode between a first edge and a second edge of an image;
means for, during each single clock unit when a next pixel block to be processed does not extend beyond said second edge,
fetching, from a storage device, one new vector of the next pixel block; and
means for, during each single clock unit when said next pixel block to be processed extends beyond said second edge,
filling a new vector in said next pixel block that is located beyond said second edge with predetermined pixel values, and
fetching, from said storage device, one new vector of a first pixel block on a next line or frame of pixels; and
means for outputting, in said each single clock unit, one output pixel value, thereby maintaining a constant rate of memory fetches and pixel outputs.
8. The apparatus according to claim 7 , wherein said means for processing successive pixel blocks further comprises means for applying an n×m kernel matrix to said successive pixel blocks, n and m being odd integers.
9. The apparatus according to claim 7 , wherein said means for processing successive pixel blocks further comprises means for two-dimensional convolution operations.
10. The apparatus according to claim 7 , wherein said means for processing successive pixel blocks further comprises means for two-dimensional correlation operations.
11. The apparatus according to claim 7 , wherein said predetermined pixel values are zeros.
12. The apparatus according to claim 7 , wherein said predetermined pixel values are pixel values of a corresponding portion of said second edge.
13. An apparatus for processing successive pixel blocks in raster mode between a first edge and a second edge of an image, the apparatus comprising:
an input data interface coupled with one or more storage devices to receive at least one input stream of pixel values of said image;
an array of pixel processing cells, each cell including:
a coefficient storage unit that stores a coefficient value of a kernel to be applied to said image during a two-dimensional image processing,
a pixel storage unit that stores a pixel value of said image, said pixel storage unit being coupled either to another pixel storage unit of an adjacent cell or to said input data interface, thereby allowing said pixel value to be initially received via said input data interface and subsequently propagated from one vector to an adjacent vector across said array,
a multiplexer unit, if said each cell is not in a center vector of said array, having inputs coupled to said pixel storage unit, a center-vector pixel storage unit on the same line of said array, and any pixel storage unit located between said center vector and said each cell, and
one or more intra-cell operation units;
inter-cell operation units that process outputs from said array of pixel processing cells to generate an output pixel value; and
a timing control module that coordinates operations of said input data interface, said array of pixel processing cells, and said inter-cell operation units, wherein
during each single clock unit when a next pixel block to be processed does not extend beyond said second edge, said timing control module causes one new vector of said next pixel block to be fetched from said one or more storage devices, and
during each single clock unit when said next pixel block to be processed extends beyond said second edge, said timing control module causes a new vector in said next pixel block that is located beyond said second edge to be filled with predetermined pixel values while causing one new vector of a first pixel block on a next line or frame of pixels to be fetched from said one or more storage devices, thereby maintaining a constant rate of memory fetches and pixel outputs.