Detecting aspect ratios of document pages on smartphone photographs by learning camera view angles
Detecting an aspect ratio of an image captured with a smartphone includes detecting at least one convex quadrangle of arbitrary shape on the image and generating a plurality of additional convex quadrangles having vertices in a pre-determined vicinity of vertices of the quadrangle on the image. A linear projective mapping matrix is generated for mapping each of the quadrangle and the plurality of additional quadrangles onto a unit square. A plurality of estimated focal lengths of the camera of the smartphone is determined according to matrixes corresponding to the linear projective mappings onto a unit square of the quadrangle and each of the plurality of additional quadrangles. The quadrangle is used to determine the aspect ratio of the image in response to a range of the plurality of estimated focal lengths including a true value of the focal length of the camera of the smartphone.
1. A method of correcting image distortion, the method comprising:
storing an image of a physical object captured by a device that includes a camera;
analyzing the image to detect at least one convex quadrangle of arbitrary shape within the image;
generating a plurality of additional convex quadrangles, wherein a respective vertex of a respective additional convex quadrangle is in a pre-determined vicinity of a corresponding vertex of the at least one convex quadrangle within the image;
generating a plurality of linear projective mapping matrixes, including, for each of the at least one convex quadrangle and the plurality of additional convex quadrangles, generating a respective linear projective mapping matrix;
using the plurality of linear projective mapping matrixes to map each of the at least one convex quadrangle and the plurality of additional convex quadrangles onto a unit square;
determining a plurality of estimated focal lengths of the camera using the mappings of the plurality of linear projective mapping matrixes onto the unit square;
using the at least one convex quadrangle to determine the aspect ratio of the image in response to a range of the plurality of estimated focal lengths including a true value of the focal length of the camera; and
correcting distortion of the image using the determined aspect ratio.
2. A method, according to claim 1 , wherein the image is one of: a still photo captured with the camera of the device or a frame in a video flow captured with the camera of the device.
3. A method, according to claim 1 , wherein each of the plurality of the estimated focal lengths of the camera is determined using the formula:
C
(
Q
)
=
H
11
·
H
12
+
H
21
·
H
22
H
31
·
H
32
where H 11 , H 12 , H 21 , H 22 , H 31 , H 32 , are all components of the linear projective mapping matrix.
4. A method, according to claim 1 , wherein two opposite vertices of the at least one convex quadrangle are mapped onto two opposite corners of a new quadrangle having a predefined distance between the two opposite vertices and a predefined orientation of an interval connecting the two opposite vertices and wherein four coordinates of two unmapped vertices of the new quadrangle are used to determine the aspect ratio.
5. A method, according to claim 4 , wherein the four coordinates of two unmapped vertices of the new quadrangle are entered into a pre-built Support Vector Regression model which derives the aspect ratio.
6. A method, according to claim 5 , wherein the pre-built Support Vector Regression model is built by:
artificially generating a plurality of document pages that represent a range of aspect ratios; and
capturing images of each document page of the plurality of document pages from a plurality of view angles.
7. A method, according to claim 1 , further comprising:
lowering resolution of the image prior to detecting the at least one convex quadrangle.
8. A method, according to claim 7 , wherein the pre-determined vicinity depends upon the lowered resolution of the image.
9. A method, according to claim 1 , wherein the pre-determined vicinity varies according to a tolerance level of acceptance of the plurality of estimated focal lengths of the camera.
10. A method, according to claim 9 , wherein the tolerance level is set by a user.
11. A non-transitory computer-readable medium containing software that corrects image distortion, the software comprising:
executable code that stores an image of a physical object captured by a device that includes a camera;
executable code that analyzes the image to detect at least one convex quadrangle of arbitrary shape within the image;
executable code that generates a plurality of additional convex quadrangles, wherein a respective vertex of a respective additional convex quadrangle is in a pre-determined vicinity of a corresponding vertex of the at least one convex quadrangle within the image;
executable code that generates a plurality of linear projective mapping matrixes, including, for each of the at least one convex quadrangle and the plurality of additional convex quadrangles, generating a respective linear projective mapping matrix;
executable code that uses the plurality of linear projective mapping matrixes to map each of the at least one convex quadrangle and the plurality of additional convex quadrangles onto a unit square;
executable code that determines a plurality of estimated focal lengths of the camera using the mappings of the plurality of linear projective mapping matrixes onto the unit square;
executable code that uses the at least one convex quadrangle to determine the aspect ratio of the image in response to a range of the plurality of estimated focal lengths including a true value of the focal length of the camera; and
correcting distortion of the image using the determined aspect ratio.
12. A non-transitory computer-readable medium, according to claim 11 , wherein the image is one of: a still photo captured with the camera of the device or a frame in a video flow captured with the camera of the device.
13. A non-transitory computer-readable medium, according to claim 11 , wherein each of the plurality of the estimated focal lengths of the camera is determined using the formula:
C
(
Q
)
=
H
11
·
H
12
+
H
21
·
H
22
H
31
·
H
32
where H 11 , H 12 , H 21 , H 22 , H 31 , H 32 , are all components of the linear projective mapping matrix.
14. A non-transitory computer-readable medium, according to claim 11 , wherein two opposite vertices of the at least one convex quadrangle are mapped onto two opposite corners of a new quadrangle having a predefined distance between the two opposite vertices and a predefined orientation of an interval connecting the two opposite vertices and wherein four coordinates of two unmapped vertices of the new quadrangle are used to determine the aspect ratio.
15. A non-transitory computer-readable medium, according to claim 14 , wherein the four coordinates of two unmapped vertices of the new quadrangle are entered into a pre-built Support Vector Regression model which derives the aspect ratio.
16. A non-transitory computer-readable medium, according to claim 15 , wherein the pre-built Support Vector Regression model is built by:
artificially generating a plurality of document pages that represent a range of aspect ratios; and
capturing images of each document page of the plurality of document pages from a plurality of view angles.
17. A non-transitory computer-readable medium, according to claim 11 , further comprising: executable code that lowers resolution of the image prior to detecting the at least one convex quadrangle.
18. A non-transitory computer-readable medium, according to claim 17 , wherein the predetermined vicinity depends upon the lowered resolution of the image.
19. A non-transitory computer-readable medium, according to claim 11 , wherein the predetermined vicinity varies according to a tolerance level of acceptance of the plurality of estimated focal lengths of the camera.
20. A non-transitory computer-readable medium, according to claim 19 , wherein the tolerance level is set by a user.