Image processing device including neural network processor and operating method thereof
An image processing device includes: an image sensor configured to generate first image data by using a color filter array; and processing circuitry configured to select a processing mode from a plurality of processing modes for the first image data, the selecting being based on information about the first image data; generate second image data by reconstructing the first image data using a neural network processor based on the processing mode; and generate third image data by post-processing the second image data apart from the neural network processor based on the processing mode.
1. An image processing device comprising:
an image sensor configured to generate first image data by using a color filter array;
a pre-processor configured to select any one of first and second processing modes as a processing mode for the first image data based on information about the first image data;
a neural network processor configured to generate second image data by reconstructing the first image data when the first processing mode is selected from among the first and second processing modes; and
a main processor configured to generate third image data by post-processing the second image data in the first processing mode,
wherein the main processor is configured to generate the second image data by reconstructing the first image data apart from the neural network processor when the second processing mode is selected by the pre-processor, and
wherein the information includes at least one of,
quality information of the first image data, and
noise information of the first image data.
2. The image processing device of claim 1 , wherein,
the color filter array has a first pattern, and
the neural network processor is further configured to,
generate the second image data corresponding to a second pattern by performing a remosaic operation on the first image data.
3. The image processing device of claim 2 , wherein the main processor is further configured to,
generate full-color image data by performing a demosaic operation on the second image data.
4. The image processing device of claim 3 , wherein,
the first image data corresponds to tetra data,
the second image data corresponds to Bayer data, and
the full-color image data corresponds to red, green, and blue (RGB) data.
5. The image processing device of claim 2 , wherein the neural network processor is further configured to perform at least one of,
a cross-talk correction pre-processing operation on the first image data, and
a despeckle pre-processing operation on the first image data.
6. The image processing device of claim 1 , wherein the neural network processor is further configured to,
generate full-color image data by performing a demosaic operation on the first image data.
7. The image processing device of claim 6 , wherein
the first image data corresponds to tetra data, and
the full-color image data corresponds to red, green, and blue (RGB) data.
8. The image processing device of claim 7 , wherein the neural network processor is further configured to,
perform a denoising operation on the full-color image data.
9. The image processing device of claim 1 , wherein, based on the selection of the second processing mode,
the neural network processor is deactivated.
10. An operating method of an image processing device comprising a color filter array, a pre-processor, a neural network processor and a main processor, the operating method comprising:
generating, by the color filter array, first image data;
selecting, by the pre-processor, any one of first and second processing modes as a processing modes for the first image data based on information about the first image data;
generating, by the neural network processor, second image data by reconstructing the first image data when the first processing mode is selected from among the processing modes; and
generating, by the neural network processor, third image data by post-processing the second image data in the first processing mode,
wherein when the second processing mode is selected by the pre-processor, the generating the second image data is performed by the main processor and apart from the neural network processor, and
wherein the information includes at least one of,
quality information of the first image data, and
noise information of the first image data.
11. The operating method of claim 10 , wherein the generating of the second image data further comprises:
generating the second image data corresponding to a Bayer pattern by performing a remosaic operation on the first image data.
12. The image processing device of claim 1 , wherein the pre-processor is configured to select the first processing mode when an artifact degree of the first image data in the quality information or a noise level of the first image data in the noise information is greater than a threshold.
13. The operating method of claim 10 , wherein the selecting any one of the first and second processing modes as the processing mode further comprises:
selecting the first processing mode when artifact degree of the first image data in the quality information or a noise level of the first image data in the noise information is greater than a threshold.