Image-capturing device, solid-state image-capturing element, camera module, electronic device, and image-capturing method for shake correction using shake angle
There is provided an image-capturing device including an image-capturing unit that captures a subject to output the captured image, a detection unit that physically detects a shake angle when the image-capturing unit has shaken, and a correction processing unit that calculates a correction amount based on an arrangement position of a pixel constituting the image outputted by the image-capturing unit and the shake angle detected by the detection unit, to correct the image according to the correction amount.
1. A solid-state image-capturing element, comprising:
a pixel circuit; and
a logic circuit, the logic circuit comprising:
a gyro sensor that physically detects a shake angle; and
a camera shake correction processing unit that calculates a correction amount based on an arrangement position of a pixel within a set of pixels constituting an image output from an output unit and the shake angle detected by the gyro sensor, to correct the image according to the correction amount to obtain a corrected image,
wherein the logic circuit is provided on a first circuit board which is different from a second circuit board on which the pixel circuit is provided, and
wherein the camera shake correction processing unit calculates the correction amount for each pixel in the set of pixels by using a pupil distance as an optical distance from a pupil position of an optical system for forming the image to a surface of the solid-state image-capturing element, the arrangement position, an elevation angle from the pupil position with respect to a point on the surface corresponding to the arrangement position, and the shake angle.
2. The solid-state image-capturing element according to claim 1 ,
wherein the camera shake correction processing unit uses different correction amounts depending on distances in a vertical direction and in a horizontal direction from a substantial center of the image.
3. The solid-state image-capturing element according to claim 1 ,
wherein the correction amount for each pixel in the set of pixels to be corrected is expressed by Formula (1) and Formula (2) as follows:
[
Formula
(
1
)
]
y
=
d
·
(
TAN
α
-
TAN
(
α
+
θ
)
)
(
1
)
[
Formula
(
2
)
]
x
=
X
·
(
1
-
COS
α
COS
(
α
+
θ
)
)
(
2
)
wherein Δx represents the correction amount in a horizontal direction of the image, Δy represents the correction amount in a vertical direction of the image, d represents the pupil distance, α represents the elevation angle, θ represents the shake angle, and X represents a distance in the horizontal direction from a center of the image.
4. The solid-state image-capturing element according to claim 1 ,
wherein the camera shake correction processing unit performs interpolation from a predetermined number of pixels around the set of pixels to be corrected to obtain a pixel value of each of the set of pixels to be corrected, so that a pixel position of each pixel constituting the corrected image corresponds to a pixel position of each pixel constituting the image before the correction.
5. The solid-state image-capturing element according to claim 1 , wherein the logic circuit further comprises:
a signal synchronization unit that synchronizes the image output from the output unit with the shake angle output from the gyro sensor to transmit the image and the shake angle to the camera shake correction processing unit.
6. The solid-state image-capturing element according to claim 5 ,
wherein the image comprises horizontal lines, and the signal synchronization unit synchronizes the image output from the output unit with the shake angle output from the gyro sensor for each of the horizontal lines.
7. The solid-state image-capturing element according to claim 1 ,
wherein the correction amount comprises correcting an effect on a vertical direction of the image by the shake angle when the image is vertically moved, and further comprises correcting an effect on a horizontal direction of the image by the shake angle when the image is vertically moved; and
wherein the correction amount further comprises correcting an effect on the horizontal direction of the image by the shake angle when the image is horizontally moved, and further comprises correcting an effect on the vertical direction of the image by the shake angle when the image is horizontally moved.
8. The solid-state image-capturing element according to claim 1 ,
wherein the solid-state image-capturing element captures the image by using a global shutter system in which a whole image is recorded substantially at a same time.
9. The solid-state image-capturing element according to claim 1 ,
wherein the solid-state image-capturing element captures the image by using a rolling shutter system in which capturing timing is different for each horizontal line within the image.
10. The solid-state image-capturing element according to claim 1 ,
wherein a drive unit drives a lens of an optical system for forming the image on a surface of the solid-state image-capturing element in a direction in which the image on the solid-state image-capturing element does not move, on the basis of the shake angle detected by the gyro sensor; and
wherein a part not depending on the arrangement position of the pixel is corrected by a parallel drive by using the drive unit.
11. The solid-state image-capturing element according to claim 1 ,
wherein the camera shake correction processing unit uses a subject distance as an optical distance from the pupil position to a subject to calculate an additional correction amount for additionally correcting the correction amount to perform camera shake correction processing according to the subject distance.
12. The solid-state image-capturing element according to claim 11 ,
wherein the correction amount, when performing the camera shake correction processing according to the subject distance, is expressed by Formula (3) and Formula (4) as follows:
[
Formula
(
3
)
]
y
=
d
·
(
TAN
α
-
TAN
(
α
+
θ
)
)
+
[
{
r
·
SIN
θ
-
d
·
(
1
-
COS
θ
)
}
·
TAN
α
+
d
·
SIN
θ
+
r
·
(
1
-
COS
θ
)
]
×
Ny
2
·
D
·
TAN
β
(
3
)
[
Formula
(
4
)
]
x
=
X
·
(
1
-
COS
α
COS
(
α
+
θ
)
)
+
[
{
r
·
SIN
θ
-
d
·
(
1
-
COS
θ
)
}
·
TAN
α
+
d
·
SIN
θ
+
r
·
(
1
-
COS
θ
)
]
×
Nx
2
·
D
·
TAN
γ
(
4
)
wherein Δx represents the correction amount in a horizontal direction of the image, Δy represents the correction amount in a vertical direction of the image, d represents the pupil distance, α represents the elevation angle, θ represents the shake angle, X represents a distance in the horizontal direction from a center of the image, of a pixel to be corrected, r represents a rotation radius as a distance from a rotational center to a center point of the surface when the solid-state image-capturing element has shaken, D represents the subject distance, Ny represents a pixel count of a side in the vertical direction of the solid-state image-capturing element, β represents a maximum view angle in the vertical direction of the solid-state image-capturing element, Nx represents a pixel count of a side in the horizontal direction of the solid-state image-capturing element, and γ represents a maximum view angle in the horizontal direction of the solid-state image-capturing element.
13. The solid-state image-capturing element according to claim 11 ,
wherein the camera shake correction processing unit, when the subject distance is equal to or less than a predetermined threshold value previously set, performs the camera shake correction processing according to the subject distance.
14. The solid-state image-capturing element according to claim 11 ,
wherein a focus drive unit drives a lens for focus adjustment of the optical system to focus on a subject captured in the image; and
wherein a subject distance estimating unit estimates the subject distance on the basis of a control signal for controlling the drive of the lens for focus adjustment by the focus drive unit.
15. The solid-state image-capturing element according to claim 14 ,
wherein the gyro sensor detects a direction of gravitational acceleration to supply the direction of the gravitational acceleration to the subject distance estimating unit; and
wherein the subject distance estimating unit corrects the estimated subject distance according to the direction of the gravitational acceleration.
16. An image-capturing camera module, comprising:
a solid-state image-capturing element, the solid-state image-capturing element comprising:
a pixel circuit; and
a logic circuit, the logic circuit comprising:
a gyro sensor that physically detects a shake angle when the image-capturing camera module has shaken; and
a camera shake correction processing unit that calculates a correction amount based on an arrangement position of a pixel within a set of pixels constituting an image output from an output unit, and the shake angle detected by the gyro sensor, to correct the image according to the correction amount to obtain a corrected image,
wherein the logic circuit is provided on a first circuit board which is different from a second circuit board on which the pixel circuit is provided, and
wherein the camera shake correction processing unit calculates the correction amount for each pixel in the set of pixels by using a pupil distance as an optical distance from a pupil position of an optical system for forming the image to a surface of the solid-state image-capturing element, the arrangement position, an elevation angle from the pupil position with respect to a point on the surface corresponding to the arrangement position, and the shake angle.
17. A method for a solid-state image-capturing element that captures a subject to output an image, the solid-state image-capturing element comprising a pixel circuit and a logic circuit, the logic circuit comprising a gyro sensor that physically detects a shake angle and a camera shake correction processing unit, the method comprising the steps of:
outputting the image from an output unit;
outputting the shake angle by the gyro sensor; and
calculating a correction amount based on an arrangement position of a pixel within a set of pixels constituting the image output from the output unit, and the shake angle detected by the gyro sensor, to correct the image according to the correction amount to obtain a corrected image,
wherein the correction amount for each pixel in the set of pixels by using a pupil distance as an optical distance from a pupil position of an optical system for forming the image to a surface of the solid-state image-capturing element, the arrangement position, an elevation angle from the pupil position with respect to a point on the surface corresponding to the arrangement position, and the shake angle.