Architecture for high performance, power efficient, programmable image processing
An apparatus is described. The apparatus includes an image processing unit. The image processing unit includes a plurality of stencil processor circuits each comprising an array of execution unit lanes coupled to a two-dimensional shift register array structure to simultaneously process multiple overlapping stencils through execution of program code. The image processing unit includes a plurality of sheet generators respectively coupled between the plurality of stencil processors and the network. The sheet generators are to parse input line groups of image data into input sheets of image data for processing by the stencil processors, and, to form output line groups of image data from output sheets of image data received from the stencil processors. The image processing unit includes a plurality of line buffer units coupled to the network to pass line groups in a direction from producing stencil processors to consuming stencil processors to implement an overall program flow.
1. A processor comprising:
a two-dimensional shift-register array; and
a two-dimensional array of processing elements,
wherein each shift register in the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements,
wherein each processing element is configured to execute instructions of a kernel program that causes the two-dimensional shift-register array to shift a sheet of input data stored in the two-dimensional shift-register array in a two-dimensional pattern and causes each processing element to read, from the respective shift register dedicated to the processing element, a plurality of input values in a sequence over the two-dimensional pattern, the plurality of input values corresponding to a two-dimensional region of the sheet of input data, and
wherein the instructions cause each processing element to compute a respective resultant output value using the plurality of respective input values read from the respective shift register dedicated to the processing element while shifting the sheet of input data in the two-dimensional pattern.
2. The processor of claim 1 , further comprising a sheet generator configured to receive a sheet of output values computed by the two-dimensional array of processing elements and to provide the sheet of output values to a line buffer.
3. The processor of claim 2 , wherein the line buffer is configured to receive the sheet of output values and to provide the sheet of output values to one or more consumer processors.
4. The processor of claim 2 , wherein the sheet generator is configured to receive a line group from a producer line buffer and to load a subsequent sheet of input values into the two-dimensional shift-register array of the processor.
5. The processor of claim 4 , wherein the processor, the sheet generator, and the line buffer are configured to implement a portion of an image processing pipeline.
6. The processor of claim 4 , wherein the sheet generator is configured to write sheets of output values to multiple line buffers.
7. The processor of claim 1 , wherein each shift register of the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements.
8. The processor of claim 1 , wherein each shift of data from a first shift register to a second shift register in the two-dimensional shift-register array replaces a value previously stored in the second shift register.
9. The processor of claim 1 , wherein the two-dimensional shift-register array maintains a single copy of the data while shifting the sheet of input data in the two-dimensional pattern.
10. The processor of claim 1 , wherein while shifting the sheet of input data in the two-dimensional pattern each processing element reads the plurality of respective input values from a same respective shift register dedicated to the processing element.
11. The processor of claim 1 , wherein the instructions cause each processing element to output a single respective resultant output value for the plurality of respective input values read while shifting the sheet of input data in the two-dimensional pattern.
12. A method performed by a processor comprising:
a two-dimensional shift-register array; and
a two-dimensional array of processing elements,
wherein each shift register in the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements, the method comprising:
executing, by each processing element in the two-dimensional array of processing elements, instructions of a kernel program that causes the two-dimensional shift-register array to shift a sheet of input data stored in the two-dimensional shift-register array in a two-dimensional pattern and causes each processing element to read, from the respective shift register dedicated to the processing element, a plurality of input values in a sequence over the two-dimensional pattern, the plurality of input values corresponding to a two-dimensional region of the sheet of input data; and
computing, by each processing element in the two-dimensional array of processing elements, a respective resultant output value using the plurality of respective input values read from the respective shift register dedicated to the processing element while shifting the sheet of input data in the two-dimensional pattern.
13. The method of claim 12 , further comprising:
receiving, by a sheet generator, a sheet of output values computed by the two-dimensional array of processing elements; and
providing the sheet of output values to a line buffer.
14. The method of claim 13 , further comprising:
receiving, by the line buffer, the sheet of output values; and
providing the sheet of output values to one or more consumer processors.
15. The method of claim 13 , further comprising:
receiving, by the sheet generator, a line group from a producer line buffer; and
loading, by the sheet generator, a subsequent sheet of input values into the two-dimensional shift-register array of the processor.
16. The method of claim 15 , wherein the processor, the sheet generator, and the line buffer implement a portion of an image processing pipeline.
17. The method of claim 15 , further comprising writing, by the sheet generator, sheets of output values to multiple line buffers.
18. The method of claim 12 , wherein each shift register of the two-dimensional shift-register array is dedicated to a respective processing element in the two-dimensional array of processing elements.
19. The method of claim 12 , wherein shifting data from a first shift register to a second shift register in the two-dimensional shift-register array comprises replacing a value previously stored in the second shift register.
20. The method of claim 12 , further comprising maintaining, by the two-dimensional shift-register array, a single copy of the data while shifting the sheet of input data in the two-dimensional pattern.
21. The method of claim 12 , further comprising:
while shifting the sheet of input data in the two-dimensional pattern, reading, by each processing element, the plurality of respective input values from a same respective shift register dedicated to the processing element.