Image processing method, camera assembly and mobile terminal

1. An image processing method applied to an image sensor; wherein the image sensor comprises a pixel array, the pixel array comprising a plurality of subunits, each subunit comprising at least one panchromatic light-sensitive pixel and at least one color light-sensitive pixel; each color light-sensitive pixel has a narrower spectral response than each panchromatic light-sensitive pixel; the method comprises:

obtaining a Bayer image and a panchromatic image; wherein the Bayer image comprises a plurality of first image pixels, a pixel value of each first image pixel being generated by the at least one light-sensitive pixel in a corresponding subunit, the plurality of first image pixels being arranged in a Bayer array; the panchromatic image comprises a plurality of second image pixels, a pixel value of each second image pixel being generated by the at least one panchromatic light-sensitive pixel in a corresponding subunit;

obtaining an RGB image after processing the Bayer image with a first processing algorithm and a demosaicing algorithm;

obtaining a luminance image after processing the panchromatic image with the first processing algorithm;

obtaining a fused RGB image by fusing the RGB image and the luminance image; and

obtaining a YUV image after processing the fused RGB image with a second processing algorithm.

2. The method according to claim 1 , wherein the image sensor is configured to output a merged image, the merged image comprising a plurality of merged pixels, a pixel value of each merged pixel being generated by at least one color light-sensitive pixel or at least one panchromatic light-sensitive pixel in a corresponding subunit; the method further comprises:

obtaining the Bayer image and the panchromatic image by processing the merged image.

3. The method according to claim 2 , wherein a sum or average value of multiple electrical signals generated by at least one light-sensitive pixel or at least one panchromatic light-sensitive pixel in a corresponding subunit after receiving light is taken as a merged image data.

4. The method according to claim 1 , wherein the method is implemented through an image pipeline, which is configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm.

5. The method according to claim 4 , configured to work in a preview mode or a video mode; wherein the image pipeline comprises an image front end (IFE) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm; the first processing algorithm comprises at least one of: Camera Interface (CAMIF), Pedestal Correction, linearization, phase pixel compensation (PDPC), channel gain, bad pixel correction/bad cluster correction/Adaptive Bayer Filter (BPC/BCC/ABF), Black Level Correction (BLC), lens roll off, and white balance.

6. The method according to claim 4 , configured to work in a preview mode or a video mode; wherein the image pipeline comprises an image front end (IFE) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm; the second processing algorithms comprises at least one of: color correction, global tone mapping, gamma correction, color space transform, and down scaler.

7. The method according to claim 4 , configured to work in a photo mode; wherein the image pipeline comprises a Bayer processing segment (BPS) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm; the first processing algorithm comprises at least one of: pedestal correction, linearization, bad pixel correction/bad cluster correction/phase pixel correction, channel gain, green imbalance correction, Bayer domain noise reduction module, Black Level Correction (BLC), lens roll off, and white balance; the second processing algorithm comprises at least one of: color correction, tone mapping, gamma correction, color space transform, hybrid noise reduction, and down scaler.

8. The method according to claim 1 , further comprising:

processing the YUV image with an image post-processing algorithm.

9. The method according to claim 8 , wherein the image processing method is implemented through an image pipeline; the image pipeline comprises an image processing engine (IPE) configured to implement the image post-processing algorithm.

10. The method according to claim 9 , wherein the image post-processing algorithm comprises at least one of: image correction and adjustment (ICA), advanced noise reduction, temporal filter, Chromatic Aberration Correction (CAC), color space transform, local tone mapping, color correction, gamma correction, 2D Look-Up Table (2D LUT), color adjustment and chroma enhancement (CV&Chroma enhancement), chroma suppression, Adaptive Spatial Filter (ASF), up scaler, grain adder, and down scaler.

11. The method according to claim 1 , further comprising:

storing the RGB image, the luminance image, and the fused RGB image.

12. A camera assembly, comprising an image sensor and a processor; wherein the image sensor comprises a pixel array, the pixel array comprising a plurality of subunits, each subunit comprising at least one panchromatic light-sensitive pixel and at least one color light-sensitive pixel; each color light-sensitive pixel has a narrower spectral response than each panchromatic light-sensitive pixel;

wherein the processor is configured to perform an image processing method comprising:

obtaining a Bayer image and a panchromatic image; wherein the Bayer image comprises a plurality of first image pixels, a pixel value of each first image pixel being generated by the at least one light-sensitive pixel in a corresponding subunit, the plurality of first image pixels being arranged in a Bayer array; the panchromatic image comprises a plurality of second image pixels, a pixel value of each second image pixel being generated by the at least one panchromatic light-sensitive pixel in a corresponding subunit;

obtaining an RGB image after processing the Bayer image with a first processing algorithm and a demosaicing algorithm;

obtaining a luminance image after processing the panchromatic image with the first processing algorithm;

obtaining a fused RGB image by fusing the RGB image and the luminance image; and

obtaining a YUV image after processing the fused RGB image with a second processing algorithm.

13. The camera assembly according to claim 12 , wherein the processor is configured to output a merged image, the merged image comprising a plurality of merged pixels, a pixel value of each merged pixel being generated by at least one color light-sensitive pixel or at least one panchromatic light-sensitive pixel in a corresponding subunit; the image processing method further comprises:

obtaining the Bayer image and the panchromatic image by processing the merged image.

14. The camera assembly according to claim 12 , wherein the image processing method is implemented through an image pipeline, which is configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm.

15. The camera assembly according to claim 14 , wherein the image processing method is configured to work in a preview mode or a video mode; the image pipeline comprises an image front end (IFE) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm; the first processing algorithm comprises at least one of: Camera Interface (CAMIF), Pedestal Correction, linearization, phase pixel compensation (PDPC), channel gain, bad pixel correction/bad cluster correction/Adaptive Bayer Filter (BPC/BCC/ABF), Black Level Correction (BLC), lens roll off, and white balance.

16. The camera assembly according to claim 14 , wherein the image processing method is configured to work in a preview mode or a video mode; the image pipeline comprises an image front end (IFE) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm; the second processing algorithms comprises at least one of: color correction, global tone mapping, gamma correction, color space transform, and down scaler.

17. The camera assembly according to claim 14 , wherein the image processing method is configured to work in a photo mode; the image pipeline comprises a Bayer processing segment (BPS) configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm;

the first processing algorithm comprises at least one of: pedestal correction, linearization, bad pixel correction/bad cluster correction/phase pixel correction, channel gain, green imbalance correction, Bayer domain noise reduction module, Black Level Correction (BLC), lens roll off, and white balance; the second processing algorithm comprises at least one of: color correction, tone mapping, gamma correction, color space transform, hybrid noise reduction, and down scaler.

18. The camera assembly according to claim 12 , wherein the image processing method further comprises:

processing the YUV image with an image post-processing algorithm.

19. The camera assembly according to claim 12 , wherein the image processing method further comprises:

storing the RGB image, the luminance image, and the fused RGB image.

20. A mobile terminal, comprising:

a camera assembly; and

a housing, wherein the camera assembly is arranged on the housing;

wherein the camera assembly comprises an image sensor and a processor; wherein the image sensor comprises a pixel array, the pixel array comprising a plurality of subunits, each subunit comprising at least one panchromatic light-sensitive pixel and at least one color light-sensitive pixel; each color light-sensitive pixel has a narrower spectral response than each panchromatic light-sensitive pixel;

wherein the processor is configured to perform an image processing method comprising:

obtaining a Bayer image and a panchromatic image; wherein the Bayer image comprises a plurality of first image pixels, a pixel value of each first image pixel being generated by the at least one light-sensitive pixel in a corresponding subunit, the plurality of first image pixels being arranged in a Bayer array; the panchromatic image comprises a plurality of second image pixels, a pixel value of each second image pixel being generated by the at least one panchromatic light-sensitive pixel in a corresponding subunit;

obtaining an RGB image after processing the Bayer image with a first processing algorithm and a demosaicing algorithm;

obtaining a luminance image after processing the panchromatic image with the first processing algorithm;

obtaining a fused RGB image by fusing the RGB image and the luminance image; and

obtaining a YUV image after processing the fused RGB image with a second processing algorithm.