Image sensor and operating method of image sensor

1 . An image sensor comprising:

a pixel array configured to convert optical signals into electrical signals;

a readout circuit configured to convert the electrical signals into image data and output the image data; and

a bad-pixel correction circuit configured to correct a cluster bad pixel included in the image data based on a neural network without performing a convolution operation, the neural network comprising a plurality of fully connected layers connected to each other and receiving a flattened one dimensional N1×1 matrix as an input,

wherein a first fully connected layer, among the plurality of fully connected layers, is configured to:

receive the flattened one dimensional N1×1 matrix,

generate an one dimensional N2×1 matrix based on the flattened one dimensional N1×1 matrix and a first kernel, and

output the one dimensional N2×1 matrix to a second fully connected layer, among the plurality of fully connected layers,

wherein N1 and N2 are integers.

2 . The image sensor of claim 1 , wherein the plurality of fully connected layers of the neural network comprise a fully connected layer.

3 . The image sensor of claim 1 , wherein the bad-pixel correction circuit is further configured to:

generate corrected pixel data corresponding to the cluster bad pixel based on input data including first bad pixels corresponding to the cluster bad pixel and pixels around the first bad pixels, and

correct the cluster bad pixel based on the corrected pixel data,

wherein the input data comprises:

pixels arranged in an N×N matrix of a same color and

a color pattern in which four pixel groups arranged in a matrix are repeatedly arranged, and

wherein N is an integer of 2 or more.

4 . The image sensor of claim 3 , wherein the neural network is trained based on training data including bad pixels arranged in an M×K matrix, and

wherein M is an integer of 2 or more and K is an integer of 2 or more.

5 . The image sensor of claim 4 , wherein, based on the first bad pixels corresponding to the cluster bad pixel being identical to the bad pixels in the M×K matrix included in the training data, the bad-pixel correction circuit is further configured to:

generate the corrected pixel data by inputting the input data comprising the cluster bad pixel to the neural network, and

replace pixel data of the cluster bad pixel with the corrected pixel data.

6 . The image sensor of claim 4 , wherein, based on the first bad pixels corresponding to the cluster bad pixel being different from the bad pixels in the M×K matrix included in the training data, the bad-pixel correction circuit is further configured to:

perform pre-processing on the input data comprising the cluster bad pixel and output pre-processed input data,

generate the corrected pixel data by inputting the pre-processed input data to the neural network,

generate post-processed pixel data by performing post-processing on the corrected pixel data, and

replace pixel data of the cluster bad pixel with the post-processed pixel data.

7 . The image sensor of claim 6 , wherein, based on the cluster bad pixel including the first bad pixels arranged in the M×K matrix, the bad-pixel correction circuit is further configured to:

transpose the input data comprising the cluster bad pixel and output transposed input data, and

input the transposed input data to the neural network.

8 . The image sensor of claim 6 , wherein, based on the cluster bad pixel including the first bad pixels arranged in an M×K matrix, and based on a color pattern of the input data being symmetrical to a color pattern of the training data in one of a first direction and a second direction, the bad-pixel correction circuit is further configured to:

flip the input data in one of the first direction and the second direction and output flipped input data; and

input the flipped input data to the neural network.

9 . The image sensor of claim 6 , wherein, based on pixels, which are arranged in the M×K matrix including the cluster bad pixel, including at least one valid pixel, the bad-pixel correction circuit is further configured to:

pre-process the at least one valid pixel in the input data as bad pixels and output pre-processed input data, and

input the pre-processed input data to the neural network.

10 . The image sensor of claim 9 , wherein the bad-pixel correction circuit is further configured to apply the corrected pixel data generated based on the pre-processed input data, to the image data, and recover a pixel value of at least one pixel corresponding to the at least one valid pixel to a pixel value before correction.

11 . The image sensor of claim 3 , wherein, based on the pixels around the cluster bad pixel including at least one second bad pixel, the bad-pixel correction circuit is further configured to:

perform pre-processing on the input data to replace a pixel value of the at least one second bad pixel with an average value of pixel values of pixels around the at least one second bad pixel and output pre-processed input data, and

input the pre-processed input data to the neural network.

12 . The image sensor of claim 3 , wherein the bad-pixel correction circuit is further configured to:

divide the input data into N pieces of sub-input data including pixels at a same position in each pixel group, and

provide the N pieces of sub-input data to each of N sub-networks of the neural network.

13 . An image sensor comprising:

a pixel array configured to convert optical signals into electrical signals;

a readout circuit configured to convert the electrical signals into image data and output the image data, the image data comprising a plurality of pixel groups each comprising pixels of a same color arranged in an N×N matrix, and N being an integer of 2 or more; and

an image signal processor configured to correct a cluster bad pixel occurring in at least one of the plurality of pixel groups based on a neural network without including a convolutional layer,

wherein the neural network comprises a plurality of fully connected layers connected to each other and receiving a flattened one dimensional N1×1 matrix as an input,

wherein a first fully connected layer, among the plurality of fully connected layers, is configured to:

receive the flattened one dimensional N1×1 matrix,

generate an one dimensional N2×1 matrix based on the flattened one dimensional N1×1 matrix and a first kernel, and

output the one dimensional N2×1 matrix to a second fully connected layer, among the plurality of fully connected layers,

wherein N1 and N2 are integers.

14 . The image sensor of claim 13 , wherein the neural network includes a plurality of fully connected layers.

15 . The image sensor of claim 13 , wherein the image signal processor is further configured to:

pre-process input data including pixels corresponding to the cluster bad pixel and peripheral pixels around the pixels corresponding to the cluster bad pixel such that the input data is processable by the neural network and output pre-processed input data,

generate corrected pixel data by applying the pre-processed input data to the neural network, and

correct the cluster bad pixel based on the corrected pixel data.

16 . The image sensor of claim 15 , wherein, based on at least one valid pixel being included in pixels arranged in an M×K matrix comprising the cluster bad pixel, the image signal processor is further configured to pre-process the at least one valid pixel in the input data as a bad pixel,

wherein M is an integer of 2 or more and K is an integer of 2 or more.

17 . The image sensor of claim 15 , wherein, based on the peripheral pixels including at least one bad pixel, the image signal processor is further configured to generate the pre-processed input data by correcting the at least one bad pixel based on pixel values of pixels around the at least one bad pixel.

18 . The image sensor of claim 15 , wherein the image signal processor is further configured to:

divide the input data into N pieces of sub-input data comprising pixels at a same position in each pixel group, and

provide the N pieces of sub-input data to each of N sub-networks of the neural network.

19 . An operating method of an image sensor, the operating method comprising:

generating image data based on an optical signal;

detecting a cluster bad pixel in the image data;

generating corrected pixel data, by operating a neural network without including a convolutional layer, based on input data comprising the cluster bad pixel and pixels around the cluster bad pixel; and

correcting the cluster bad pixel based on the corrected pixel data,

wherein the neural network comprises a plurality of fully connected layers connected to each other and receiving a flattened one dimensional N1×1 matrix as an input, and

wherein the method further comprises:

by a first fully connected layer, among the plurality of fully connected layers:

receiving the flattened one dimensional N1×1 matrix,

generating an one dimensional N2×1 matrix based on the flattened one dimensional N1×1 matrix and a first kernel, and

outputting the one dimensional N2×1 matrix to a second fully connected layer, among the plurality of fully connected layers,

wherein N1 and N2 are integers.

20 . The operating method of claim 19 , wherein the generating of the corrected pixel data comprises pre-processing the input data such that the input data matches training data used in training of the neural network.