IP Library Granted Patent US 10,445,898
Granted Patent B2
US 10,445,898 · App. 15/016,936 · Granted Oct 15, 2019

System and method for camera calibration by use of rotatable three-dimensional calibration object

Inventors: Cheng-Yi Liu (San Jose, CA); Alexander Berestov (San Jose, CA)
Assignee: SONY CORPORATION
G06T7/80H04N13/246H04N17/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,445,898
App. No.
15/016,936
Granted
Oct 15, 2019
Kind
B2
Abstract

Various aspects of a system and a method for camera calibration by use of a rotatable three-dimensional (3-D) calibration object are disclosed herein. In accordance with an embodiment, the system includes a first electronic device, which determines a rotation pattern of the 3-D calibration object, based on a set of pre-selected images. The set of pre-selected images includes the 3-D calibration object captured at pre-defined viewing angles. Control information is communicated by the first electronic device to a second electronic device associated with the 3-D calibration object to rotate the 3-D calibration object in accordance with the determined rotation pattern. A plurality of image frames of the 3-D calibration object are captured to calibrate intrinsic and/or extrinsic camera parameters of the first electronic device.

Claims (93)

1. A camera calibration system, comprising:

a second electronic device comprising first circuitry configured to rotate a three dimensional (3-D) calibration object based on first control information; and

a first electronic device comprising second circuitry configured to:

acquire a first set of images, wherein

the first set of images is a set of pre-selected images,

each of the first set of images comprises a two dimensional (2-D) texture pattern that corresponds to a first instance of capture of the 3-D calibration object,

a viewing angle for capture of a first image of the first set of images is different from a viewing angle for capture of a second image of the first set of images, and

the 2-D texture pattern indicates a plurality of feature points;

detect the plurality of feature points of the 2-D texture pattern in the first set of images;

select a second set of images from the first set of images, based on a quality threshold of the plurality of feature points, wherein the quality threshold is associated with the detection of the plurality of feature points in the first set of images;

determine a first rotation pattern of the 3-D calibration object based on the second set of images;

generate the first control information based on the first rotation pattern;

communicate the first control information to the second electronic device to rotate the 3-D calibration object;

obtain a plurality of image frames of the rotated 3-D calibration object, wherein each of the plurality of image frames corresponds to a second instance of capture of the rotated 3-D calibration object; and

calibrate one of intrinsic camera parameters or extrinsic camera parameters of the first electronic device based on the plurality of image frames.

2. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to capture the plurality of image frames of the rotated 3-D calibration object.

3. The camera calibration system according to claim 1 , wherein the first circuitry is further configured to:

store the first control information in the second electronic device; and

rotate the 3-D calibration object on one of a horizontal axis or a vertical axis, based on the first control information.

4. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to:

determine an angle of capture of the plurality of image frames of the rotated 3-D calibration object; and

capture the plurality of image frames of the rotated 3-D calibration object based on the angle of capture.

5. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to:

receive position information of each of a plurality of imaging devices;

determine an angle of capture for each of the plurality of imaging devices, based on the position information; and

control each of the plurality of imaging devices to capture the plurality of image frames of the 3-D calibration object, based on the angle of capture.

6. The camera calibration system according to claim 1 ,

wherein the 3-D calibration object comprises at least one facade, and

wherein the at least one facade includes at least one of a unique identifier at a first position on the at least one facade or a sub-pattern that corresponds to the 2-D texture pattern.

7. The camera calibration system according to claim 6 , wherein the second circuitry is further configured to detect at least one of the unique identifier or the sub-pattern in each of the plurality of image frames.

8. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to:

receive the plurality of image frames of the 3-D calibration object from a plurality of imaging devices,

wherein the plurality of image frames are captured by the plurality of imaging devices around the 3-D calibration object.

9. The camera calibration system according to claim 1 ,

wherein the second circuitry is further configured to capture the plurality of image frames of the 3-D calibration object at a plurality of rotation states of the 3-D calibration object, and

wherein a first rotation state of the plurality of rotation states is different from a second rotation state of the plurality of rotation states.

10. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to:

compare a first distribution of the plurality of feature points in the 2-D texture pattern with a second distribution of the plurality of feature points of one of the second set of images; and

determine, based on the comparison, a similarity associated with the first distribution of the plurality of feature points between each of the plurality of image frames and the one of the second set of images.

11. The camera calibration system according to claim 10 , wherein the second circuitry is further configured to select one of the plurality of image frames based on the similarity.

12. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to:

determine a second rotation pattern of the 3-D calibration object;

generate second control information based on the second rotation pattern; and

communicate the second control information to the second electronic device.

13. The camera calibration system according to claim 12 ,

wherein the first circuitry is further configured to:

rotate the 3-D calibration object based on the second control information; and

generate a timestamp for each rotation state of the 3-D calibration object,

wherein the timestamp for each rotation state corresponds to a time of capture of each of the plurality of image frames, and

wherein the second circuitry is further configured to synchronize, at each step angle of rotation of the 3-D calibration object, the time of capture of each of the plurality of image frames to each rotation state of the 3-D calibration object, wherein the synchronization of the time of capture of each of the plurality of image frames to each rotation state of the 3-D calibration object is based on one of the first rotation pattern or the second rotation pattern.

14. The camera calibration system according to claim 13 , wherein the second circuitry is further configured to:

calibrate the intrinsic camera parameters based on the plurality of image frames;

determine a plurality of 3-D world coordinates of the plurality of feature points on the 3-D calibration object; and

determine a 2-D to 3-D map of each image frame of the plurality of image frames, based on:

the synchronization of the time of capture to each rotation state,

the calibrated intrinsic camera parameters, and

the plurality of 3-D world coordinates of the plurality of feature points on the 3-D calibration object.

15. The camera calibration system according to claim 1 , wherein the second circuitry is further configured to obtain the plurality of image frames of the rotated 3-D calibration object from a plurality of imaging devices that are communicatively coupled to the first electronic device.

16. A camera calibration method, comprising:

acquiring, by a first electronic device, a first set of images, wherein

the first set of images is a set of pre-selected images,

each of the first set of images comprises a two dimensional (2-D) texture pattern that corresponds to a first instance of capture of a three-dimensional (3-D) calibration object,

a viewing angle for capture of a first image of the first set of images is different from a viewing angle for capture of a second image of the first set of images, and

the 2-D texture pattern indicates a plurality of feature points;

detecting, by the first electronic device, the plurality of feature points in the 2-D texture pattern in the first set of images;

selecting, by the first electronic device, a second set of images from the first set of images, based on a quality threshold of the plurality of feature points, wherein the quality threshold is associated with the detection of the plurality of feature points in the first set of images;

determining, by the first electronic device, a first rotation pattern of the 3-D calibration object based on the second set of images;

generating, by the first electronic device, first control information based on the first rotation pattern;

communicating, by the first electronic device, the first control information to a second electronic device;

rotating, by the second electronic device, the 3-D calibration object based on the first control information;

obtaining, by the first electronic device, a plurality of image frames of the rotated 3-D calibration object, wherein each of the plurality of image frames corresponds to a second instance of capture of the rotated 3-D calibration object; and

calibrating, by the first electronic device, one of intrinsic camera parameters or extrinsic camera parameters of the first electronic device based on the plurality of image frames.

17. The camera calibration method according to claim 16 , further comprising detecting, by the first electronic device, at least one of a unique identifier or a sub-pattern in each of the plurality of image frames.

18. The camera calibration method according to claim 16 , further comprising:

comparing, by the first electronic device, a first distribution of the plurality of feature points in the 2-D texture pattern with a second distribution of the plurality of feature points of one of the second set of images; and

determining, by the first electronic device, a similarity based on the comparison, wherein the similarity is associated with the first distribution of the plurality of feature points between each of the plurality of image frames and the one of the second set of images.

19. The camera calibration method according to claim 18 , further comprising selecting, by the first electronic device, one of the plurality of image frames based on the similarity.

20. The camera calibration method according to claim 16 , further comprising:

determining, by the first electronic device, a second rotation pattern of the 3-D calibration object;

generating, by the first electronic device, second control information based on the second rotation pattern; and

communicating, by the first electronic device, the second control information to the second electronic device.

21. The camera calibration method according to claim 20 , further comprising:

rotating, by the second electronic device, the 3-D calibration object based on the second control information;

generating, by the second electronic device, a timestamp for each rotation state of the 3-D calibration object,

wherein the timestamp for each rotation state corresponds to a time of capture of each of the plurality of image frames; and

synchronizing, by the first electronic device, at each step angle of rotation of the 3-D calibration object, the time of capture of each of the plurality of image frames to each rotation state of the 3-D calibration object, wherein the synchronization of the time of capture of each of the plurality of image frames to each rotation state of the 3-D calibration object is based on one of the first rotation pattern or the second rotation pattern.

22. The camera calibration method according to claim 21 , further comprising:

calibrating, by the first electronic device, the intrinsic camera parameters based on the plurality of image frames;

determining, by the first electronic device, a plurality of 3-D world coordinates of the plurality of feature points on the 3-D calibration object; and

determining, by the first electronic device, a 2-D to 3-D map of each image frame of the plurality of image frames, based on:

the synchronization of the time of capture to each rotation state,

the calibrated intrinsic camera parameters, and

the plurality of 3-D world coordinates of the plurality of feature points on the 3-D calibration object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2016
From: LIU, CHENG-YI; BERESTOV, ALEXANDER
To: SONY CORPORATION
Reel/Frame 037717/0255 →
Continuity (1)
Related Publication 20170228864A1 · Aug 10, 2017
Cited By (3)
US 12,307,690 US 12,657,772 US 12,688,610