Image pixel crosstalk correction
Systems and methods for image correction are provided. One aspect of the systems and methods includes obtaining first image data at a first time and second image data at a second time. Another aspect of the systems and methods includes computing calibration parameters for the second image data in real time based on the first image data. Another aspect of the systems and methods includes correcting the second image data based on the calibration parameters to obtain corrected second image data. Another aspect of the systems and methods includes generating an output image based on the corrected second image data.
1 . A method comprising:
obtaining first image data at a first time and second image data at a second time;
computing, using a machine learning model, calibration parameters for the second image data in real time based on the first image data;
correcting for crosstalk the second image data based on the calibration parameters to obtain a corrected second image data; and
generating an output image based on the corrected second image data,
wherein crosstalk is indicative of unwanted transfer of image information from a first image channel to a second image channel, and
wherein the machine learning model is trained based on a crosstalk metric that is computed based on predicted image data and ground-truth image data.
2 . The method of claim 1 , wherein:
the first image data and the second image data are obtained in real time from a same scene.
3 . The method of claim 1 , wherein:
the first image data comprises preview image data.
4 . The method of claim 3 , further comprising:
obtaining predetermined calibration data, wherein the calibration parameters are computed based on the predetermined calibration data.
5 . The method of claim 1 , wherein:
the calibration parameters comprise parameters of a function of a pixel array position.
6 . The method of claim 1 , wherein:
the calibration parameters comprise illumination parameters.
7 . The method of claim 1 , wherein:
the calibration parameters comprise crosstalk correction parameters.
8 . The method of claim 1 , further comprising:
encoding the first image data to obtain an image embedding having fewer dimensions than the first image data, wherein the calibration parameters are computed based on the image embedding.
9 . The method of claim 1 , wherein:
the first image data has a lower resolution than the second image data.
10 . A method comprising:
obtaining training data including input image data and ground-truth image data;
computing, using a machine learning model, calibration parameters based on the input image data;
performing crosstalk correction based on the calibration parameters to obtain predicted image data;
training the machine learning model based on the predicted image data and the ground-truth image data; and
computing a crosstalk metric based on the predicted image data and the ground-truth image data, wherein the machine learning model is trained based on the crosstalk metric.
11 . The method of claim 10 , further comprising:
computing an image quality metric based on the predicted image data, wherein the machine learning model is trained based on the image quality metric.
12 . The method of claim 10 , further comprising:
encoding the input image data to obtain an image embedding having fewer dimensions than the input image data, wherein the calibration parameters are computed based on the image embedding.
13 . The method of claim 10 , further comprising:
obtaining additional training data including preview input image data;
computing, using an additional machine learning model, additional calibration parameters based on the preview input image data;
performing crosstalk correction based on the additional calibration parameters to obtain additional predicted image data; and
training the additional machine learning model based on the additional predicted image data.
14 . The method of claim 13 , further comprising:
obtaining predetermined calibration data, wherein the additional calibration parameters are computed based on the predetermined calibration data.
15 . An apparatus comprising:
at least one memory component;
at least one processing device coupled to the at least one memory component, wherein the processing device is configured to execute instructions stored in the at least one memory component;
an image sensor configured to obtain first image data at a first time and second image data at a second time;
a dynamic calibration component comprising parameters stored in the at least one memory component and configured to compute, using a machine learning model, calibration parameters for the second image data in real time based on the first image data; and
a correction component comprising parameters stored in the at least one memory component and configured to correct for crosstalk the second image data based on the calibration parameters to obtain corrected second image data,
wherein crosstalk is indicative of unwanted transfer of image information from a first image channel to a second image channel, and
wherein the machine learning model is trained based on a crosstalk metric that is computed based on predicted image data and ground-truth image data.
16 . The apparatus of claim 15 , wherein the apparatus further comprises an image processing component configured to generate an output image based on the corrected second image data.
17 . The apparatus of claim 15 , wherein the dynamic calibration component comprises a machine learning model trained to compute the calibration parameters.
18 . The apparatus of claim 17 , wherein the machine learning model comprises an encoder configured to encode the first image data to obtain an image embedding having fewer dimensions than the first image data, wherein the calibration parameters are computed based on the image embedding.
19 . The apparatus of claim 15 , wherein the dynamic calibration component comprises an additional machine learning model trained to compute additional calibration parameters based on preview input image data.