IP Library › Granted Patent US 10,235,774
Granted Patent B1
US 10,235,774 · App. 15/812,428 · Granted Mar 19, 2019

Method and system for calibration of an image capturing device mounted on a machine

Inventors: David Paul Gitz (Peoria, IL); Paul Russell Friend (Morton, IL); Maikel Orlando Torres Pineiro (Peoria, IL); Qi Chen (Canton, MI)
Assignee: Caterpillar Inc.
G06T7/80G01S19/51G06F3/04883G06T11/60H04N5/23296H04N7/18H04N17/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,235,774
App. No.
15/812,428
Granted
Mar 19, 2019
Kind
B1
Abstract

A method for calibrating an image capturing device mounted on a machine is provided. The image capturing device is configured to capture one or more images of a calibration target. The one or more images include a set of reference points corresponding to the calibration target. The method includes receiving a first positional data and a second positional data associated with the image capturing device and the calibration target respectively. The method includes generating, through the processor, a set of augmented reality points based on the first positional data and the second positional data. The method further includes overlaying, through the processor, the set of augmented reality points on the set of reference points displayed on the display. The method includes iteratively adjusting, through the processor, at least one of a roll, a yaw, or a pitch of the image capturing device until the set of augmented reality points align with the corresponding set of reference points.

Claims (48)

1. A method for calibrating an image capturing device, the method comprising:

receiving, by one or more processors, first positional data associated with the image capturing device,

the image capturing device being mounted on a machine that includes ground engaging elements or wheels;

receiving, by the one or more processors, second positional data associated with a calibration target that is in a position relative to the machine;

generating, by the one or more processors, a set of augmented reality points, corresponding to the calibration target, based on the first positional data and the second positional data;

overlaying, by the one or more processors, the set of augmented reality points on a set of reference points of one or more images, of the calibration target, captured by the image capturing device; and

iteratively adjusting, by the one or more processors, at least one of a roll, a yaw, or a pitch of the image capturing device until the set of augmented reality points align with the set of reference points.

2. The method of claim 1 , wherein the calibration target comprises at least three non-collinear points on a ground surface.

3. The method of claim 1 , wherein the calibration target is an object of polygon shape of predetermined dimensions being placed on a ground surface at a predetermined distance from the image capturing device.

4. The method of claim 1 , wherein the second positional data is obtained by a survey of the calibration target using satellite-based positioning systems.

5. The method of claim 1 , wherein the set of augmented reality points are vertices of an augmented reality polygon.

6. The method of claim 1 , wherein iteratively adjusting comprises adjusting the roll, the yaw, and the pitch in a predefined order.

7. The method of claim 1 , wherein iteratively adjusting comprises adjusting at least one of the roll, the yaw, or the pitch based on one or more user inputs.

8. A calibration system comprising:

an image capturing device configured to capture one or more images of a calibration target,

wherein the image capturing device is mounted on a machine that includes ground engaging elements or wheels,

wherein the calibration target is in a position relative to the machine, and

wherein the one or more images include a set of reference points corresponding to the calibration target;

and

a processor configured to:

receive first positional data associated with the image capturing device;

receive second positional data associated with the calibration target;

generate a set of augmented reality points, corresponding to the calibration target, based on the first positional data and the second positional data;

overlay the set of augmented reality points on the set of reference points; and

iteratively adjust at least one of a roll, a yaw, or a pitch of the image capturing device until the set of augmented reality points align with the set of reference points.

9. The calibration system of claim 8 , wherein the calibration target comprises at least three non-collinear points on a ground surface.

10. The calibration system of claim 8 , wherein the calibration target is an object of polygon shape of predetermined dimensions being placed on a ground surface at a predetermined distance from the image capturing device.

11. The calibration system of claim 8 , wherein the second positional data is obtained by a survey of the calibration target using satellite-based positioning systems.

12. The calibration system of claim 8 , wherein the set of augmented reality points are vertices of an augmented reality polygon.

13. The calibration system of claim 8 , wherein the roll, the yaw, and the pitch are adjusted in a predefined order.

14. The calibration system of claim 8 , wherein the roll, the yaw, and the pitch are adjusted based on one or more user inputs.

15. The calibration system of claim 8 , wherein the processor is further configured to iteratively adjust a x-coordinate, a y-coordinate, and a z-coordinate of the image capturing device until the set of augmented reality points align with the set of reference points.

16. A transitory computer-readable storage medium comprising:

one or more instructions that, when executed by a processor, cause the processor to:

receive first positional data associated with an image capturing device,

the image capturing device being mounted on a machine that includes ground engaging elements or wheels;

receive second positional data associated with a calibration target that is in a position relative to the machine;

generate a set of augmented reality points, corresponding to the calibration target, based on the first positional data and the second positional data;

overlay the set of augmented reality points on a set of reference points of one or more images, of the calibration target, captured by the image capturing device; and

iteratively adjust at least one of a roll, a yaw, or a pitch of the image capturing device until the set of augmented reality points align with the set of reference points.

17. The non-transitory computer-readable storage medium of claim 16 ,

wherein the image capturing device is mounted on a rear side of the machine, and

wherein the image capturing device is configured for viewing areas behind the machine.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the first positional data is received from a positioning system of the machine.

19. The non-transitory computer-readable storage medium of claim 16 , wherein the second positional data includes at least one of:

angular dimensions of the calibration target, or

information identifying a distance of the calibration target to the machine.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the calibration target is a calibration triangle formed using ropes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2017
From: GITZ, DAVID PAUL; FRIEND, PAUL RUSSELL; PINEIRO, MAIKEL ORLANDO TORRES; CHEN, QI
To: CATERPILLAR INC.
Reel/Frame 044122/0612 →
Cited By (2)
US 12,335,655 US 12,657,772