Imaging apparatus, imaging system, and image processing method
To increase image quality of a moving image by suppressing a color afterimage while reducing color noise, provided is an imaging apparatus, including: an imaging device; and a signal processing unit, in which: the imaging device includes a first pixel group and a second pixel group each including a plurality of pixels each configured to output a pixel signal; and the signal processing unit is configured to perform weighted addition for a second pixel signal output from the second pixel group by inter-frame processing, and to change a weight on each frame in the weighted addition based on an inter-frame differential of a first pixel signal.
1. An imaging apparatus, comprising:
an imaging device; and
a signal processing unit, wherein:
the imaging device includes a first pixel and a second pixel each having a photoelectric converter;
the first pixel is a pixel configured to output a first pixel signal based on a light corresponding to white color on the photoelectric converter;
the second pixel is a pixel configured to output a second pixel signal based on a light corresponding to any one of colors of green, blue, and red on the photoelectric converter;
the first pixel is configured to output the first pixel signal in each of a first frame and a second frame;
the second pixel is configured to output the second pixel signal in each of the first frame and the second frame;
the signal processing unit is configured to perform a first inter-frame processing which is an addition of the first pixel signal of the first frame and the first pixel signal of the second frame, and a second inter-frame processing which is an addition of the second pixel signal of the first frame and the second pixel signal of the second frame; and
the signal processing unit is configured to perform the second inter-frame processing by changing weights of the second pixel signal of the first frame and the second pixel signal of the second frame, based on a difference between the first pixel signal of the first frame and the first pixel signal of the second frame.
2. An imaging apparatus according to claim 1 , further comprising a discrimination processing unit configured to output a discrimination signal indicating a result of comparing an inter-frame differential of the first pixel signal with a threshold value,
wherein the signal processing unit is further configured to change the weight on each frame based on a signal value of the discrimination signal.
3. An imaging apparatus according to claim 2 , wherein the discrimination processing unit is further configured to calculate an average value of signals output from a larger number of the first pixels than a number of the second pixels to be subjected to the inter-frame processing, and to generate the discrimination signal based on an inter-frame differential of the average value.
4. An imaging apparatus according to claim 2 , wherein the signal processing unit is further configured to reduce a weight on another frame of the second pixel signal with respect to a weight on a current frame of the second pixel signal as an inter-frame differential of the first pixel signal becomes larger.
5. An imaging apparatus according to claim 3 , wherein the first pixels to be subjected to the calculation of the average value comprise first pixels around the second pixel to be subjected to the inter-frame processing.
6. An imaging apparatus according to claim 2 , wherein:
the discrimination processing unit is further configured to generate the discrimination signal based on an inter-frame differential of information of a ratio between the first pixel signal interpolated in a position of the second pixel and the second pixel signal; and
the signal processing unit is further configured to reduce a weight on another frame of the second pixel signal with respect to a weight on a current frame of the second pixel signal as the inter-frame differential of the information of the ratio becomes larger.
7. An imaging apparatus according to claim 2 , wherein:
the discrimination processing unit is further configured to output the discrimination signal by calculating a motion vector through use of a plurality of frames of the first pixel signal; and
the signal processing unit is further configured to reduce a weight on another frame of the second pixel signal with respect to a weight on a current frame of the second pixel signal as an absolute value of the motion vector becomes larger.
8. An imaging apparatus according to claim 1 , further comprising a signal combining unit configured to combine the first pixel signal and the second pixel signal subjected to the inter-frame processing to generate an image signal obtained by expressing each pixel by respective values of R, G, and B.
9. An imaging apparatus according to claim 1 , wherein the signal processing unit performs the inter-frame processing comprising a moving average.
10. An imaging apparatus according to claim 1 , wherein the signal processing unit performs the inter-frame processing with a recursive filter.
11. An imaging apparatus according to claim 1 , wherein the signal processing unit performs the inter-frame processing with a non-recursive filter.
12. An imaging apparatus according to claim 1 , wherein the signal processing unit performs the inter-frame processing with a median filter.
13. An imaging apparatus according to claim 1 , further comprising a plurality of the first pixels, wherein the second pixel is surrounded by the plurality of first pixels.
14. An imaging system, comprising:
the imaging apparatus according to claim 1 ; and
an output signal processing unit configured to process a signal output from the imaging apparatus.
15. An image processing method for processing a pixel signal output from an imaging apparatus,
the imaging apparatus comprising:
an imaging device; and
a signal processing unit,
the imaging device comprising a first pixel and a second pixel each having a photoelectric converter,
the first pixel is a pixel configured to output a first pixel signal based on a light corresponding to white color on the photoelectric converter,
the second pixel is a pixel configured to output a second pixel signal based on a light corresponding to any one of colors of green, blue, and red on the photoelectric converter,
the first pixel is configured to output the first pixel signal in each of a first frame and a second frame, and
the second pixel is configured to output the second pixel signal in each of the first frame and the second frame,
the image processing method comprising:
performing a first inter-frame processing which is an addition of the first pixel signal of the first frame and the first pixel signal of the second frame, and a second inter-frame processing which is an addition of the second pixel signal of the first frame and the second pixel signal of the second frame; and
performing the second inter-frame processing by changing weights of the second pixel signal of the first frame and the second pixel signal of the second frame, based on a difference between the first pixel signal of the first frame and the first pixel signal of the second frame.
16. An imaging device, comprising:
a first pixel and a second pixel each having a photoelectric converter; and
a signal processing unit, wherein:
the first pixel is a pixel configured to output a first pixel signal based on a light corresponding to white color on the photoelectric converter;
the second pixel is a pixel configured to output a second pixel signal based on a light corresponding to any one of colors of green, blue, and red on the photoelectric converter;
the first pixel is configured to output the first pixel signal in each of a first frame and a second frame;
the second pixel is configured to output the second pixel signal in each of the first frame and the second frame;
the signal processing unit is configured to perform a first inter-frame processing which is an addition of the first pixel signal of the first frame and the first pixel signal of the second frame, and a second inter-frame processing which is an addition of the second pixel signal of the first frame and the second pixel signal of the second frame; and
the signal processing unit is configured to perform the second inter-frame processing by changing weights of the second pixel signal of the first frame and the second pixel signal of the second frame, based on a difference between the first pixel signal of the first frame and the first pixel signal of the second frame.
17. An imaging device according to claim 16 , wherein:
the first pixel and the second pixel are formed on a first semiconductor substrate;
the signal processing unit is formed on a second semiconductor substrate; and
the first semiconductor substrate and the second semiconductor substrate are laminated on one another.
18. An imaging apparatus, comprising:
an imaging device; and
a signal processing unit, wherein:
the imaging device includes a first pixel and a second pixel each having a photoelectric converter;
the first pixel is a pixel configured to output a first pixel signal based on a light corresponding to white color on the photoelectric converter;
the second pixel is a pixel configured to output a second pixel signal based on a light corresponding to any one of colors of magenta, cyan, and yellow on the photoelectric converter;
the first pixel is configured to output the first pixel signal in each of a first frame and a second frame;
the second pixel is configured to output the second pixel signal in each of the first frame and the second frame;
the signal processing unit is configured to perform a first inter-frame processing which is an addition of the first pixel signal of the first frame and the first pixel signal of the second frame, and a second inter-frame processing which is an addition of the second pixel signal of the first frame and the second pixel signal of the second frame; and
the signal processing unit is configured to perform the second inter-frame processing by changing weights of the second pixel signal of the first frame and the second pixel signal of the second frame, based on a difference between the first pixel signal of the first frame and the first pixel signal of the second frame.
19. An image processing method for processing a pixel signal output from an imaging apparatus,
the imaging apparatus comprising:
an imaging device; and
a signal processing unit,
the imaging device comprising a first pixel and a second pixel each having a photoelectric converter,
the first pixel is a pixel configured to output a first pixel signal based on a light corresponding to white color on the photoelectric converter,
the second pixel is a pixel configured to output a second pixel signal based on a light corresponding to any one of colors of magenta, cyan, and yellow on the photoelectric converter,
the first pixel is configured to output the first pixel signal in each of a first frame and a second frame, and
the second pixel is configured to output the second pixel signal in each of the first frame and the second frame,
the image processing method comprising:
performing a first inter-frame processing which is an addition of the first pixel signal of the first frame and the first pixel signal of the second frame, and a second inter-frame processing which is an addition of the second pixel signal of the first frame and the second pixel signal of the second frame; and
performing the second inter-frame processing by changing weights of the second pixel signal of the first frame and the second pixel signal of the second frame, based on a difference between the first pixel signal of the first frame and the first pixel signal of the second frame.