IP Library Granted Patent US 10,652,472
Granted Patent B2
US 10,652,472 · App. 15/902,899 · Granted May 12, 2020

Enhanced automatic perspective and horizon correction

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,652,472
App. No.
15/902,899
Granted
May 12, 2020
Kind
B2
Abstract

Embodiments relate to automatic perspective and horizon correction. Generally, a camera captures an image as an image file. Capture-time orientation data from one or more sensors is used to determine the camera's attitude with respect to a defined reference frame. The orientation data and/or attitude can be registered into metadata of the image file and used to generate axis lines representative of the camera's reference frame. A reference line such as a horizon can be automatically identified from detected line segments in the image that align with one of the axis lines within a predetermined angular threshold. The reference line can be used to generate a camera transformation from a starting orientation reflected by the camera attitude to a transformed orientation that aligns the reference line with the reference frame. The transformation can be applied to the image to automatically correct perspective distortion and/or horizon tilt in the image.

Claims (50)

1. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:

capturing an image as an image file with the one or more computing devices;

accessing capture-time orientation data from one or more sensors associated with the one or more computing devices;

determining, based on the capture-time orientation data, a capture-time attitude of the one or more computing devices with respect to a defined reference frame;

automatically identifying a reference line from the image using the capture-time attitude of the one or more computing devices; and

generating target parameters for a camera transformation from a starting orientation reflected by the capture-time attitude to a transformed orientation that aligns the reference line with the reference frame.

2. The media of claim 1 , the operations further comprising registering the capture-time attitude into metadata of the image file.

3. The media of claim 1 , the operations further comprising:

automatically detecting line segments from the image;

automatically generating, using the capture-time attitude, an axis line parallel to a horizontal or a vertical axis of the reference frame; and

automatically identifying the reference line from ones of the detected line segments that align with the axis line within a predetermined angular threshold.

4. The media of claim 3 , wherein automatically identifying the reference line comprises giving priority to detected line segments of longer length.

5. The media of claim 1 , the operations further comprising registering the target parameters into metadata of the image.

6. The media of claim 1 , the operations further comprising automatically applying the camera transformation to the image to correct at least one of perspective distortion or horizon tilt in the image.

7. The media of claim 1 , wherein the reference frame is aligned with at least one of gravity or magnetic north.

8. A computerized method for facilitating the automatic correction of horizon tilt in an image, the method comprising:

accessing the image captured as an image file by a camera;

accessing a capture-time attitude of the camera with respect to a defined reference frame;

automatically detecting a horizon of the image from detected line segments in the image, giving priority to ones of the detected line segments of longer length, using the capture-time attitude of the camera; and

automatically generating target parameters for a 3D camera transformation from a starting orientation reflected by the capture-time attitude to a transformed orientation that aligns the detected horizon with a horizontal axis of the reference frame.

9. The media of claim 8 , wherein giving priority to detected line segments of longer length comprises:

sorting the detected line segments by length to generate a sorted list;

iterating through the sorted list to identify a longest detected line segment that aligns with the horizontal axis; and

based on identifying the longest detected line segment, avoiding subsequent iterations of shorter line segments in the sorted list.

10. The method of claim 8 , additionally comprising accessing the capture-time attitude from metadata of the image file.

11. The method of claim 8 , additionally comprising:

automatically generating, using the capture-time attitude, a horizontal axis line parallel to the horizontal axis of the reference frame; and

automatically identifying the horizon from ones of the detected line segments that align with the horizontal axis line within a predetermined angular threshold.

12. The method of claim 8 , additionally comprising registering the target parameters into metadata of the image.

13. The method of claim 8 , additionally comprising automatically applying the 3D camera transformation to the image to correct the horizon tilt.

14. The method of claim 8 , wherein the reference frame is aligned with gravity.

15. A computer system comprising:

one or more hardware processors and memory configured to provide computer program instructions to the one or more hardware processors;

a camera module comprising a photo sensor, wherein the camera module is configured to capture an image as an image file;

an attitude component configured to:

access capture-time orientation data from one or more sensors associated with the camera module; and

determine, based on the capture-time orientation data, a capture-time attitude of the one or more computing devices with respect to a defined reference frame; and

a means for automatically generating target parameters for a camera transformation based on the capture-time orientation data.

16. The computer system of claim 15 :

wherein the camera transformation is from a starting orientation reflected by the capture-time attitude to a transformed orientation that aligns a horizontal reference line of the image with a horizontal axis of the reference frame and that aligns a vertical reference line of the image with a vertical axis line of the reference frame; and

wherein the means for automatically generating target parameters for a camera transformation is additionally configured to automatically identify the reference line from the image using the capture-time attitude of the camera module.

17. The computer system of claim 16 , wherein the means for automatically generating target parameters for a camera transformation is additionally configured to:

automatically detect line segments from the image;

automatically generate, using the capture-time attitude, a horizontal axis line parallel to the horizontal axis of the reference frame;

automatically generate, using the capture-time attitude, a vertical axis line parallel to the vertical axis of the reference frame;

automatically identify the horizontal reference line from ones of the detected line segments that align with the horizontal axis line within a predetermined angular threshold; and

automatically identify the vertical reference line from ones of the detected line segments that align with the vertical axis line within the predetermined angular threshold.

18. The computer system of claim 17 , wherein the means for automatically generating target parameters for a camera transformation is additionally configured to automatically identify at least one of the horizontal reference line or the vertical reference line by giving priority to detected line segments of longer length.

19. The computer system of claim 15 , wherein the means for automatically generating target parameters for a camera transformation is additionally configured to register the target parameters into metadata of the image.

20. The computer system of claim 15 , additionally comprising an image correction component configured to automatically apply the camera transformation to the image to correct at least one of perspective distortion or horizon tilt in the image.

Assignments (2)
CHANGE OF NAME Recorded Nov 29, 2018
From: ADOBE SYSTEMS INCORPORATED
To: ADOBE INC.
Reel/Frame 047687/0115 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2018
From: GUPTA, SUBHAM; BHALLA, POONAM; KARKI, KRISHNA SINGH; BEDI, AJAY
To: ADOBE SYSTEMS INCORPORATED
Reel/Frame 045824/0074 →