Color filter array image data processing

1 . A method of processing color filter array, CFA, image data, the method comprising performing image processing in an image processing pipeline comprising a plurality of stages, the method comprising:

receiving CFA image data comprising a plurality of pixel values in a frame of image data representable as a main two-dimensional row-columnar pattern, each pixel value corresponding to a color filter in a CFA, wherein different pixel values of the plurality of pixel values are associated with respective different filter colors including first pixel values associated with a first filter color and second pixel values associated with a second filter color, wherein:

the first pixel values are arranged in at least a two-dimensional checkerboard sub-pattern in the main two-dimensional row-columnar pattern;

performing, in one stage of the image processing pipeline, a diagonally-aligned processing operation on the first pixel values, wherein the processing operation operates on the first pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array; and

generating output image data resulting from performing said diagonally-aligned image processing operation on the first pixel values, the output image data comprising first output pixel values,

performing, in a subsequent stage of the image processing pipeline, a further, different, diagonally-aligned processing operation on the first output pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array; and

generating second output pixel values resulting from performing said further diagonally-aligned image processing operation on the first pixel values.

2 . The method of processing CFA image data according to claim 1 , wherein the second pixel values are arranged in a two-dimensional row-columnar sub-pattern in the main two-dimensional row-columnar pattern, the method comprising:

performing a parallel-aligned processing operation on the second pixel values, wherein the parallel-aligned processing operation operates on the second pixel values in the alignment in which they are arranged in the two-dimensional row-columnar sub-pattern; and

generating output image data resulting from performing said parallel-aligned image processing operation on the second pixel values.

3 . The method of processing CFA image data according to claim 1 , wherein the diagonally-aligned processing operation comprises a filtering operation.

4 . The method of processing CFA image data according to claim 3 , wherein the filtering is performed using a separable kernel.

5 . The method of processing CFA image data according to claim 4 , wherein the separable kernel is configured to be decomposed into kernels operable to process the first pixel values in separate, mutually-orthogonal directions.

6 . The method of processing CFA image data according to claim 3 , wherein the filtering comprises a filtering operation of a denoising type, a sharpening type, a blurring type, an interpolation type or an edge detection type.

7 . The method of processing CFA image data according to claim 2 , wherein:

the diagonally-aligned processing operation comprises a first filtering operation and the parallel-aligned processing operation comprises a second filtering operation, and

wherein the first and second filtering operations are of a corresponding type.

8 . The method of processing CFA image data according to claim 7 , wherein the first and second filtering operations are one of a denoising type, a sharpening type, a blurring type, an interpolation type or an edge detection type.

9 . The method of processing CFA image data according to claim 1 , wherein the first pixel values are a square array of pixel values rotated by 45 degrees relative to the row-columnar directions of main two-dimensional row-columnar pattern.

10 . The method of processing CFA image data according to claim 2 , wherein:

the first pixel values are a square array of pixel values rotated by 45 degrees relative to the row-columnar directions of the main two-dimensional row-columnar pattern; and

the second pixel values are a square array of pixel values non-rotated relative to the row-columnar directions of the main two-dimensional row-columnar pattern.

11 . The method of processing CFA image data according to claim 1 , wherein said diagonally-aligned processing operation comprises image compression.

12 . The method of processing CFA image data according to claim 11 , wherein the image compression is performed in a blockwise manner wherein blocks of a first pixel value are processed sequentially in a zig-zag path across at least part of the frame.

13 . The method of processing CFA image data according to claim 2 , wherein said diagonally-aligned processing operation and said parallel-aligned processing operation each comprise image compression.

14 . The method of processing CFA image data according to claim 13 , wherein the diagonally-aligned processing operation is performed on first pixel value blocks comprising the first pixel values, wherein the parallel-aligned processing operation is performed on second pixel value blocks comprising the second pixel values, and wherein the image compression comprises:

for the diagonally-aligned blockwise-processing operation, processing of at least some of the first pixel value blocks sequentially in a zig-zag path across at least part of the frame; and

for the parallel-aligned processing operation, processing of at least some of the second pixel value blocks sequentially in a linear path across at least part of the frame.

15 . The method of processing CFA image data according to claim 11 , wherein the diagonally aligned processing operation is performed on first pixel value blocks comprising the first pixel values, and wherein:

the first pixel value blocks are parallelogram-shaped blocks with one pair of edges aligned 45 degrees relative to the row-columnar directions of the main two-dimensional row-columnar pattern and another pair of edges aligned with one of the row-columnar directions.

16 . The method of processing CFA image data according to claim 13 , wherein the diagonally-aligned processing operation is performed on first pixel value blocks comprising the first pixel values, wherein the parallel-aligned processing operation is performed on second pixel value blocks comprising the second pixel values, and wherein:

the first pixel value blocks are parallelogram-shaped blocks with one pair of edges aligned 45 degrees relative to the row-columnar directions of the main two-dimensional row-columnar pattern and another pair of edges aligned with one of the row-columnar directions; and

the second pixel value blocks are square blocks aligned with the row-columnar directions of the main two-dimensional row-columnar pattern.

17 . The method of processing CFA image data according to claim 11 , wherein the image compression comprises DCT compression, DWT compression, color quantization, chroma subsampling, fractal compression, run-length encoding and/or DEFLATE compression.

18 . An image signal processor comprising circuitry configured to perform a method comprising performing image processing in an image processing pipeline comprising a plurality of stages, the method comprising:

receiving color filter array, CFA, image data comprising a plurality of pixel values in a frame of image data representable as a main two-dimensional row-columnar pattern, each pixel value corresponding to a color filter in a CFA, wherein different pixel values of the plurality of pixel values are associated with respective different filter colors including first pixel values associated with a first filter color and second pixel values associated with a second filter color, wherein:

the first pixel values are arranged in at least a two-dimensional checkerboard sub-pattern in the main two-dimensional row-columnar pattern;

performing, in one stage of the image processing pipeline, a diagonally-aligned processing operation on the first pixel values, wherein the processing operation operates on the first pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array;

generating output image data resulting from performing said diagonally-aligned image processing operation on the first pixel values, the output image data comprising first output pixel values,

performing, in a subsequent stage of the image processing pipeline, a further, different, diagonally-aligned processing operation on the first output pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array; and

generating second output pixel values resulting from performing said further diagonally-aligned image processing operation on the first pixel values.

19 . An imaging system comprising:

an image sensor; and

an image signal processor comprising circuitry configured to perform a method comprising performing image processing in an image processing pipeline comprising a plurality of stages, the method comprising:

receiving color filter array, CFA, image data from the image sensor, the CFA image data comprising a plurality of pixel values in a frame of image data representable as a main two-dimensional row-columnar pattern, each pixel value corresponding to a color filter in a CFA, wherein different pixel values of the plurality of pixel values are associated with respective different filter colors including first pixel values associated with a first filter color and second pixel values associated with a second filter color, wherein:

the first pixel values are arranged in at least a two-dimensional checkerboard sub-pattern in the main two-dimensional row-columnar pattern;

performing, in one stage of the image processing pipeline, a diagonally-aligned processing operation on the first pixel values, wherein the processing operation operates on the first pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array;

generating output image data resulting from performing said diagonally-aligned image processing operation on the first pixel values, the output image data comprising first output pixel values,

performing, in a subsequent stage of the image processing pipeline, a further different diagonally-aligned processing operation on the first output pixel values in accordance with a reprojected array, the reprojected array corresponding to a reprojection of the two-dimensional checkerboard sub-pattern into a row-columnar array, and

generating second output pixel values resulting from performing said further diagonally-aligned image processing operation on the first pixel values.