Image processing device and image processing method using three-dimensional and artificial intelligence noise reduction
An image processing device includes a three-dimensional noise reduction (3D NR) circuit, an artificial intelligence noise reduction (AI NR) circuit, a weight determination circuit and an image blending circuit. The 3D NR circuit performs a 3D NR operation on input image data to generate first image data. The AI NR circuit performs an AI NR operation on the input image data to generate second image data. The weight determination circuit outputs a blending weight according to a motion index. The image blending circuit blends the first image data and the second image data according to the blending weight to generate output image data.
1. An image processing device, comprising:
a three-dimensional noise reduction (3D NR) circuit, performing a 3D NR operation on input image data to generate first image data;
an artificial intelligence noise reduction (AI NR) circuit, performing an AI NR operation on the input image data to generate second image data;
a weight determination circuit, outputting a blending weight according to a motion index; and
a first image blending circuit, blending the first image data and the second image data according to the blending weight to generate output image data.
2. The image processing device of claim 1 , wherein when the motion index indicates that a current area in the input image data is a still area, a weight corresponding to the first image data in the blending weight is a first weight; when the motion index indicates that the current area is a motion area, the weight corresponding to the first image data in the blending weight is a second weight, wherein the first weight is greater than the second weight.
3. The image processing device of claim 1 , wherein when the motion index indicates that a number of motion areas in a current frame of the input image data is a first value, a weight corresponding to the first image data in the blending weight is a first weight; when the motion index indicates that the number of motion areas in the current frame is a second value, the weight corresponding to the first image data in the blending weight is a second weight, wherein the first value is smaller than the second value and the first weight is greater than the second weight.
4. The image processing device of claim 1 , wherein the 3D NR circuit comprises:
a temporal filter, performing temporal filtering according to a reference frame and a current frame in the input image data to generate first data;
a spatial filter, performing spatial filtering according to the reference frame and the current frame to generate second data; and
a second image blending circuit, blending the first data and the second data according to the motion index to generate the first image data.
5. The image processing device of claim 4 , wherein the 3D NR circuit further comprises:
a motion detector, detecting motion area information of the current frame according to a difference between the current frame and the reference frame to generate the motion index.
6. The image processing device of claim 5 , wherein the motion detector comprises:
a sum of absolute differences (SAD) calculation circuit, calculating sums of absolute differences between a plurality of corresponding blocks in the current frame and the reference frame to generate a plurality of differences;
a comparison circuit, comparing each of the differences with a threshold value to generate a plurality of detection values; and
a determination circuit, determining the motion index according to the detection values.
7. The image processing device of claim 1 , wherein the AI NR circuit determines the motion index further according to the input image data.
8. The image processing device of claim 7 , wherein the AI NR circuit determines the motion index according to a current frame in the input image data and data generated by a temporal filter in the 3D NR circuit.
9. The image processing device of claim 1 , wherein the motion detector is generated by an AI circuit.
10. The image processing device of claim 1 , wherein the weight determination circuit comprises:
a memory circuit, storing a look-up table; and
a look-up table (LUT) circuit, reading the look-up table according to the motion index to output the blending weight.
11. The image processing device of claim 1 , wherein the weight determination circuit is implemented by an AI circuit.
12. The image processing device of claim 11 , wherein the AI NR circuit performs training based on the input image data and the motion index to generate the blending weight.
13. The image processing device of claim 11 , wherein the weight determination circuit further adjusts the blending weight according to an external command.
14. The image processing device of claim 11 , wherein the AI circuit is included in the AI NR circuit.
15. The image processing device of claim 1 , wherein the AI NR circuit comprises:
a process control module; and
a multiply-accumulate array, reading weight data based on control of the process control module, and performing the AI NR operation on the input image data according to the weight data to generate the second image data.
16. An image processing method, comprising:
performing a 3D NR operation on input image data by a three-dimensional noise reduction (3D NR) circuit to generate first image data;
performing an AI NR operation on the input image data by an artificial intelligence noise reduction (AI NR) circuit to generate second image data;
generating a blending weight according to a motion index; and
blending the first image data and the second image data according to the blending weight to generate output image data.
17. The image processing method of claim 16 , wherein when the motion index indicates that a current area in the input image data is a still area, a weight corresponding to the first image data in the blending weight is a first weight; when the motion index indicates that the current area is a motion area, the weight corresponding to the first image data in the blending weight is a second weight, wherein the first weight is greater than the second weight.
18. The image processing method of claim 16 , wherein when the motion index indicates that a number of motion areas in a current frame of the input image data is a first value, a weight corresponding to the first image data in the blending weight is a first weight; when the motion index indicates that the number of motion areas in the current frame is a second value, the weight corresponding to the first image data in the blending weight is a second weight, wherein the first value is smaller than the second value and the first weight is greater than the second weight.
19. An image processing device, comprising:
a temporal filter, performing temporal filtering according to a reference frame and a current frame in input image data to generate first data;
a spatial filter, performing spatial filtering according to the reference frame and the current frame to generate second data;
an artificial intelligence noise reduction (AI NR) circuit, performing an AI NR operation on the input image data to generate third data; and
an image blending circuit, blending the first data, the second data and the third data to generate output image data.