Systems and methods of adaptive phase detection autofocus offset correction
Imaging systems and techniques are described. An imaging system receives image data captured by an image sensor according to one or more image capture settings, for instance according to an aperture size, temperature, lux, lens position, and/or region of interest. The image data includes image pixel data and focus pixel data. The imaging system determines a first focus setting based on phase detection using the focus pixel data. The imaging system determines a focus offset based on use of the one or more image capture settings as inputs to a trained model (e.g., decision tree, random forest, neural network). The imaging system determines a second focus setting at least in part by adjusting the first focus setting according to the focus offset, and causes a focus control mechanism to set a focus parameter to the second focus setting.
1 . An apparatus for imaging, the apparatus comprising:
at least one memory; and
one or more processors coupled to the at least one memory, the one or more processors configured to:
receive image data captured by an image sensor according to one or more image capture settings, wherein the image data includes focus pixel data;
determine a first focus setting based on phase detection using the focus pixel data;
predict a focus offset between the first focus setting and a target focus setting based on the first focus setting, the one or more image capture settings, and a regression model, the regression model generated based on samples of previously-determined phase-based focus settings and previously-determined contrast-based focus settings corresponding to different image capture settings for the image sensor; and
cause a focus control mechanism to set a focus parameter to a second focus setting, wherein the second focus setting is based on application of the focus offset to the first focus setting.
2 . The apparatus of claim 1 , wherein, to predict the focus offset, the one or more processors are configured to:
process the first focus setting and the one or more image capture settings using the regression model to generate the focus offset.
3 . The apparatus of claim 1 , wherein, to predict the focus offset, the one or more processors are configured to:
process the first focus setting using at least one focus offset map and the regression model to generate the focus offset, wherein the at least one focus offset map includes respective focus offsets corresponding to different regions of interest in the image data.
4 . The apparatus of claim 1 , wherein the regression model includes a linear regression model.
5 . The apparatus of claim 1 , wherein, to predict the focus offset, the one or more processors are configured to:
process the first focus setting using a decision tree and the regression model to generate the focus offset.
6 . The apparatus of claim 1 , wherein the one or more processors are configured to:
update the regression model based on the focus offset.
7 . The apparatus of claim 1 , wherein the previously-determined phase-based focus settings are generated using phase detection autofocus (PDAF), and wherein the previously-determined contrast-based focus settings are generated using contrast detection autofocus (CDAF).
8 . The apparatus of claim 1 , wherein, to predict the focus offset, the one or more processors are configured to:
process the first focus setting and the one or more image capture settings using the regression model and a trained machine learning model to generate the focus offset, the trained machine learning model trained using training data that is generated based on prior focus settings for the focus control mechanism that are determined by the one or more processors based on prior image capture settings for image capture using the image sensor.
9 . The apparatus of claim 1 , wherein, to determine the first focus setting based on phase detection using the focus pixel data, the one or more processors are configured to identify a phase difference between a first focus dataset of the focus pixel data and a second focus dataset of the focus pixel data, wherein the first focus dataset is associated with a first focus pixel of the image sensor, wherein the second focus dataset is associated with a second focus pixel of the image sensor.
10 . The apparatus of claim 1 , wherein the one or more image capture settings include a third focus setting that is distinct from the first focus setting and the second focus setting, and wherein, to cause the focus control mechanism to set the focus parameter to the second focus setting, the one or more processors are configured to cause the focus control mechanism to adjust the focus parameter from the third focus setting to the second focus setting.
11 . The apparatus of claim 1 , wherein, to cause the focus control mechanism to set the focus parameter to the second focus setting, the one or more processors are configured to cause actuation of a linear actuator of the focus control mechanism to move a lens from a first lens position to a second lens position that corresponds to the second focus setting, wherein the image data is captured based on light passing through the lens and reaching the image sensor.
12 . The apparatus of claim 1 , wherein, to cause the focus control mechanism to set the focus parameter to the second focus setting, the one or more processors are configured to cause actuation of a linear actuator of the focus control mechanism to move a lens in a direction that is perpendicular to an image plane of the image sensor.
13 . The apparatus of claim 1 , wherein the one or more image capture settings include at least one of aperture size, temperature, lux, lens position, or region of interest.
14 . The apparatus of claim 1 , wherein the one or more processors are configured to:
receive secondary image data captured by the image sensor according to the second focus setting; and
output the secondary image data.
15 . The apparatus of claim 14 , further comprising:
a display interface, wherein, to output the secondary image data, the one or more processors are configured to cause the secondary image data to be displayed using a display at least in part by sending the secondary image data to the display through the display interface.
16 . The apparatus of claim 14 , further comprising:
a communication interface, wherein, to output the secondary image data, the one or more processors are configured to send the secondary image data to a recipient device using the communication interface.
17 . A method for imaging, the method comprising:
receiving image data captured by an image sensor according to one or more image capture settings, wherein the image data includes focus pixel data;
determining a first focus setting based on phase detection using the focus pixel data;
predicting a focus offset between the first focus setting and a target focus setting based on the first focus setting, the one or more image capture settings, and a regression model, the regression model generated based on samples of previously-determined phase-based focus settings and previously-determined contrast-based focus settings corresponding to different image capture settings for the image sensor; and
causing a focus control mechanism to set a focus parameter to a second focus setting, wherein the second focus setting is based on application of the focus offset to the first focus setting.
18 . The method of claim 17 , wherein predicting the focus offset includes processing the first focus setting and the one or more image capture settings using the regression model to generate the focus offset.
19 . The method of claim 17 , wherein predicting the focus offset includes processing the first focus setting using at least one focus offset map and the regression model to generate the focus offset, wherein the at least one focus offset map includes respective focus offsets corresponding to different regions of interest in the image data.
20 . The method of claim 17 , wherein the regression model includes a linear regression model.
21 . The method of claim 17 , wherein predicting the focus offset includes processing the first focus setting using a decision tree and the regression model to generate the focus offset.
22 . The method of claim 17 , further comprising:
updating the regression model based on the focus offset.
23 . The method of claim 17 , wherein the previously-determined phase-based focus settings are generated using phase detection autofocus (PDAF), and wherein the previously-determined contrast-based focus settings are generated using contrast detection autofocus (CDAF).
24 . The method of claim 17 , wherein predicting the focus offset includes processing the first focus setting using the regression model and a trained machine learning model to generate the focus offset, the trained machine learning model trained using training data that is generated based on prior focus settings for the focus control mechanism that are determined based on prior image capture settings for image capture using the image sensor.
25 . The method of claim 17 , wherein determining the first focus setting based on phase detection using the focus pixel data includes identifying a phase difference between a first focus dataset of the focus pixel data and a second focus dataset of the focus pixel data, wherein the first focus dataset is associated with a first focus pixel of the image sensor, wherein the second focus dataset is associated with a second focus pixel of the image sensor.
26 . The method of claim 17 , wherein the one or more image capture settings include a third focus setting that is distinct from the first focus setting and the second focus setting, and wherein causing the focus control mechanism to set the focus parameter to the second focus setting includes causing the focus control mechanism to adjust the focus parameter from the third focus setting to the second focus setting.
27 . The method of claim 17 , wherein causing the focus control mechanism to set the focus parameter to the second focus setting includes causing actuation of a linear actuator of the focus control mechanism to move a lens from a first lens position to a second lens position that corresponds to the second focus setting, wherein the image data is captured based on light passing through the lens and reaching the image sensor.
28 . The method of claim 17 , wherein causing the focus control mechanism to set the focus parameter to the second focus setting includes causing actuation of a linear actuator of the focus control mechanism to move a lens in a direction that is perpendicular to an image plane of the image sensor.
29 . The method of claim 17 , wherein the one or more image capture settings include at least one of aperture size, temperature, lux, lens position, or region of interest.
30 . The method of claim 17 , further comprising:
receiving secondary image data captured by the image sensor according to the second focus setting; and
outputting the secondary image data.