IP Library Granted Patent US 9,986,150
Granted Patent B2
US 9,986,150 · App. 15/010,620 · Granted May 29, 2018

Algorithm to estimate yaw errors in camera pose

Inventors: Daniel G Van Olst (Menlo Park, CA); Kathrin Berkner (Los Altos, CA); Ramya Narasimha (Palo Alto, CA)
Assignee: Ricoh Co., Ltd.
H04N5/23219G06T3/0006G06T3/4038G06T7/73H04N5/23222H04N5/23238H04N5/23293G06T2200/24G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,986,150
App. No.
15/010,620
Granted
May 29, 2018
Kind
B2
Abstract

A system and method that calculates a yaw error in an image and provides a user interface to a user for correcting the yaw error. The method includes receiving an image, performing line detection in the image, computing a line parameterization for lines in the image, computing a yaw angle for the image and providing the yaw data calculated in the image.

Claims (67)

1. A method comprising:

receiving, using one or more computing devices, a preview image of an object;

performing, using the one or more computing devices, line detection on the preview image to detect a plurality of lines, the plurality of lines including a first line and a second line;

determining, using the one or more computing devices, a first height between the first line and the second line at a first point in the preview image;

determining using the one or more computing devices a second height between the first line and the second line at a second point in the preview image;

computing, using the one or more computing devices, a line parameterization for the plurality of lines by calculating a first distance from a capture device point to the first point in the preview image, calculating a second distance from the capture device point to the second point in the preview image and determining one or more of start points end points and point and slope of the first line and the second line;

calculating, using the one or more computing devices, a yaw angle using the line parameterization; and

correcting, using the one or more computing devices, the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

2. The method of claim 1 , wherein the first line and the second line are non-parallel.

3. The method of claim 1 , further comprising:

stitching, using the one or more computing devices, the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.

4. The method of claim 1 , wherein correcting the preview image of the object using the yaw angle includes presenting, using the one or more computing devices, an indicator representing the yaw angle in a user interface on a display device.

5. The method of claim 4 , wherein the indicator representing the yaw angle is a vertical eyelid indicator, the method further comprising:

computing, using the one or more computing devices, a pitch angle and a roll angle for the preview image;

determining, using the one or more computing devices, the vertical eyelid indicator based on the yaw angle;

determining, using the one or more computing devices, a horizontal eyelid indicator based on the pitch angle;

determining, using the one or more computing devices, a ball indicator based on the roll angle;

presenting, using the one or more computing devices, the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

presenting, using the one or more computing devices, the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

presenting, using the one or more computing devices, the ball indicator along an outside edge of the larger concentric circle centered on the display device.

6. The method of claim 1 , further comprising:

determining, using the one or more computing devices, a vanishing point using the line parameterization of the first line and the second line.

7. A system comprising:

a processor; and

a memory storing instructions that, when executed, cause the system to:

receive a preview image of an object;

perform line detection on the preview image to detect a plurality of lines the plurality of lines including a first line and a second line;

determine a first height between the first line and the second line at a first point in the preview image;

determine a second height between the first line and the second line at a second point in the preview image;

compute a line parameterization for the plurality of lines by calculating a first distance from a capture device to the first point in the preview image, calculating a second distance from the capture device to the second point in the preview image, and determining one or more of start points, end points and point and slope of the first line and the second line;

calculate a yaw angle using the line parameterization; and

correct the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

8. The system of claim 7 , wherein the first line and the second line are non-parallel.

9. The system of claim 7 , wherein the instructions that, when executed, further cause the system to:

stitch the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.

10. The system of claim 7 , wherein the instructions that, when executed, cause the system to correct the preview image of the object using the yaw angle includes instructions that, when executed, cause the system to present an indicator representing the yaw angle in a user interface on a display device.

11. The system of claim 10 , wherein the indicator representing the yaw angle is a vertical eyelid indicator, and wherein the instructions that, when executed, further cause the system to:

compute a pitch angle and a roll angle for the preview image;

determine the vertical eyelid indicator based on the yaw angle;

determine a horizontal eyelid indicator based on the pitch angle;

determine a ball indicator based on the roll angle;

present the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

present the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

present the ball indicator along an outside edge of the larger concentric circle centered on the display device.

12. A computer program product comprising a non-transitory computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer cause the computer to:

receive a preview image of an object;

perform line detection on the preview image to detect a plurality of lines, the plurality of lines including a first line and a second line;

determine a first height between the first line and the second line at a first point in the preview image;

determine a second height between the first line and the second line at a second point in the preview image;

compute a line parameterization for the plurality of lines by calculating a first distance from a capture device point to the first point in the preview image, calculating a second distance from the capture device point to the second point in the preview image, and determining one or more of start points, end points, and point and slope of the first line and the second line;

calculate a yaw angle using the line parameterization; and

correct the preview image of the object using the yaw angle to adjust the preview image and remove a portion of an error introduced by the yaw angle.

13. The computer program product of claim 12 , wherein the first line and the second line are non-parallel.

14. The computer program product of claim 12 , wherein the computer readable program when executed on a computer causes the computer to correct the preview image of the object using the yaw angle and further causes the computer to present an indicator representing the yaw angle in a user interface on a display device.

15. The computer program product of claim 14 , wherein the computer readable program when executed on a computer to further cause the computer to present the indicator representing the yaw angle is a vertical eyelid indicator further cause the computer to:

compute a pitch angle and a roll angle for the preview image;

determine the vertical eyelid indicator based on the yaw angle;

determine a horizontal eyelid indicator based on the pitch angle;

determine a ball indicator based on the roll angle;

present the vertical eyelid indicator over a portion of an inner area of a smaller concentric circle centered on the display device;

present the horizontal eyelid indicator over a portion of an inner area of a larger concentric circle centered on the display device; and

present the ball indicator along an outside edge of the larger concentric circle centered on the display device.

16. The method of claim 1 , further comprising:

determining a vanishing point using the line parameterization of the first line and the second line.

17. The computer program product of claim 12 , wherein the computer readable program when executed on a computer causes the computer to determine a vanishing point using the line parameterization of the first line and the second line.

18. The computer program product of claim 12 , wherein the computer readable program when executed on a computer causes the computer to stitch the adjusted preview image into a plurality of previously captured images to create a linear panoramic image of the object.

19. The system of claim 7 , wherein the instructions that, when executed, further cause the system to determine a vanishing point using the line parameterization of the first line and the second line.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2016
From: VAN OLST, DANIEL G.; BERKNER, KATHRIN; NARASIMHA, RAMYA
To: RICOH COMPANY, LTD.
Reel/Frame 040119/0874 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2016
From: VAN OLST, DANIEL G.
To: RICOH COMPANY, LTD.
Reel/Frame 037624/0171 →
Continuity (2)
Provisional Application 62234804 · Sep 30, 2015
Related Publication 20170094158A1 · Mar 30, 2017