Method for chaining image-processing functions on a SIMD processor
View Patent ↗In a single-instruction-multiple-data (SIMD) processor having multiple lanes, and local memory dedicated to each lane, a method of processing an image is disclosed. The method comprises mapping consecutive rasters of the image to consecutive lanes such that groups of consecutive rasters form image strips, and vertical stacks of strips comprise strip columns. Local memory allocates memory to the image strips. A sequence of functions is processed for execution on the SIMD processor in a pipeline implementation, such that the pipeline loops over portions of the image in multiple iterations, and intermediate data processed during the functions is stored in the local memory. Data associated with the image is traversed by first processing image strips from top to bottom in a left-most strip column, then progressing to each adjacent unprocessed strip column.
1. A method of processing image data, the method using a Single-Instruction-Multiple-Data (SIMD) processor having multiple lanes and local memory, the method comprising:
mapping a group of N consecutive rasters within the image data to N consecutive lanes of the SIMD processor;
processing an image strip formed of the group of N consecutive rasters within the image data, wherein a vertical stack of image strips forms a strip column, and the image data comprises one or more strip columns, wherein the processing of each of the N consecutive rasters advances within each of the respective N consecutive lanes from left to right across the image strip with different rasters of the image strip being processed in different lanes of the SIMD processor in parallel, wherein the step of processing an image strip further comprises processing the image strip in a sequence of functions configured to execute on the SIMD processor according to a pipeline implementation, the pipeline implementation looping over portions of the image data in multiple iterations, wherein intermediate data processed during the sequence of functions is stored in the local memory, wherein the sequence of functions comprises a sequence of kernels configured to execute on the SIMD processor according to the pipeline implementation, and wherein each kernel in the pipeline implementation is configured to implement an image processing function;
storing in the local memory the image strip;
repeating the processing step for different image strips forming a strip column, so as to process an entire strip column, and for different strip columns across the image, as necessary, so as to process the entire image data; and
defining overlapping regions between adjacent strip columns in the image data, wherein an amount of overlap corresponds to an amount of horizontal context requested by all of the kernels in the pipeline implementation.
2. The method according to claim 1 , wherein a raster represents a line segment in the image data.
3. The method according to claim 1 , wherein the consecutive rasters represent vertically consecutive line segments in the image data.
4. The method according to claim 1 , wherein a number of consecutive rasters for each image strip corresponds to a number of lanes of the SIMD processor.
5. The method according to claim 1 , wherein the sequence of kernels defines a kernel processing graph such that an output of one kernel directly feeds an input of another kernel.
6. The method according to claim 1 , wherein each iteration of the pipeline implementation comprises:
transferring data from an external memory to the local memory to provide local data;
performing each kernel in the pipeline implementation on the local data to produce processed data; and
transferring the processed data back from the local memory to the external memory.
7. The method according to claim 1 , wherein the image data is processed in sections, wherein each section defines an incremental buffer.
8. The method according to claim, further comprising:
maintaining vertical state for each kernel in the pipeline implementation from one image strip to a subsequent image strip within a strip column by selectively providing at least one private history strip to each kernel in the pipeline implementation.
9. The method according to claim, further comprising:
chaining image processing functions by passing image strips directly from one function to a subsequent function in the sequence of functions while maintaining an organization of the image data among the local memory.
10. The method according to claim 1 , further comprising:
chaining image processing functions by passing image strips directly from one function to a subsequent function in the sequence of functions using the local memory and without any intermediate transfers of the image strips to external memory.
11. The method according to claim 1 , further comprising:
generating, via a kernel function, pixels outside top and bottom image boundaries by providing to the kernel function a current line number and a total number of lines in the image data.
12. The method according to claim 1 , further comprising:
generating, via a kernel function, pixels outside left and right image boundaries by configuring the kernel to assign an input line as a full width of the image data and configuring the kernel to load extra horizontal context for internal vertical edges; and
selectively storing valid data back to a global memory.
13. The method according to claim 1 , further comprising:
traversing the image data by first processing image strips from top to bottom in a left-most strip column and then progressing to each adjacent unprocessed strip column.
14. A non-transitory computer-readable medium including instructions executable by an SIMD processor for processing image data, the instructions comprising:
instructions to map a group of N consecutive rasters within the image data to N consecutive lanes of the SIMD processor;
instructions to process an image strip formed of the group of N consecutive rasters within the image data, wherein a vertical stack of image strips forms a strip column, and the image data comprises one or more strip columns, wherein the processing of each of the N consecutive rasters advances within each of the respective N consecutive lanes from left to right across the image strip with different rasters of the image strip being processed in different lanes of the SIMD processor in parallel, wherein the instructions to process an image strip further comprises instructions to process the image strip in a sequence of functions configured to execute on the SIMD processor according to a pipeline implementation, the pipeline implementation looping over portions of the image data in multiple iterations, wherein intermediate data processed during the sequence of functions is stored in a local memory, wherein the sequence of functions comprises a sequence of kernels configured to execute on the SIMD processor according to the pipeline implementation, and wherein each kernel in the pipeline implementation is configured to implement an image processing function;
instructions to store in the local memory the image strip;
instructions to repeat processing for different image strips forming a strip column, so as to process an entire strip column, and for different strip columns across the image, as necessary, so as to process the entire image data; and
instructions to define overlapping regions between adjacent strip columns in the image data, wherein an amount of overlap corresponds to an amount of horizontal context requested by all of the kernels in the pipeline implementation.
15. An apparatus configured to process image data, the apparatus comprising:
a SIMD processor including multiple lanes and local memory; and
a computer-readable medium including instructions executable by the SIMD processor, which, when executed by SIMD processor, configures the SIMD processor to:
map a group of N consecutive rasters within the image data to N consecutive lanes of the SIMD processor;
process an image strip formed of the group of N consecutive rasters within the image data, wherein a vertical stack of image strips forms a strip column, and the image data comprises one or more strip columns, wherein the processing of each of the N consecutive rasters advances within each of the respective N consecutive lanes from left to right across the image strip with different rasters of the image strip being processed in different lanes of the SIMD processor in parallel;
process the image strip in a sequence of functions configured to execute on the SIMD processor according to a pipeline implementation, the pipeline implementation looping over portions of the image data in multiple iterations, wherein intermediate data processed during the sequence of functions is stored in the local memory, wherein the sequence of functions comprises a sequence of kernels configured to execute on the SIMD processor according to the pipeline implementation, and wherein each kernel in the pipeline implementation is configured to implement an image processing function;
store in the local memory the image strip;
repeat the processing step for different image strips forming a strip column, so as to process an entire strip column, and for different strip columns across the image, as necessary, so as to process the entire image data; and
define overlapping regions between adjacent strip columns in the image data, wherein an amount of overlap corresponds to an amount of horizontal context requested by all of the kernels in the pipeline implementation.
16. The apparatus according to claim 15 , wherein the instructions further configure the SIMD processor to:
traverse the image data by first processing image strips from top to bottom in a left-most strip column and then progressing to each adjacent unprocessed strip column.