Granted Patent B2

US 12,519,925 · App. 18/322,695 · Granted Jan 6, 2026

Inventor: Shun Sato (Matsumoto, JP)

Assignee: SEIKO EPSON CORPORATION

H04N17/002G06T7/80G06T11/60G06T2207/10028G06T2207/30204G06T2207/30244

Abstract

Claims (48)

1 . A method of visualizing a calibration state of a first camera, the method comprising:

(a) obtaining a calibration parameter of the first camera;

(b) using a second camera to obtain a relative position between the first camera and the second camera, the second camera being attached to VR glasses configured to be worn by a user, the step (b) further including:

(b1) determining first 3D coordinate values in a reference coordinate system of the first camera with respect to a specific position located within an imaging range of the first camera, the step (b1) further including:

obtaining depth information at the specific position in an image taken by the first camera, using a third camera; and

calculating the first 3D coordinate values at the specific position using a pixel coordinate value of the first camera related to the specific position, the depth information, and an internal parameter included in the calibration parameter;

(b2) imaging, by the second camera, a marker with which the relative position to the first camera is determined, to thereby obtain a marker image; and

(b3) determining a coordinate conversion matrix representing the relative position between the first camera and the second camera, using the marker image; and

(c) displaying a specific image representing a calibration state of the first camera on a display device with a positional posture changing with a value of the calibration parameter so as to be superimposed on an image taken by the second camera.

2 . The method according to claim 1 , wherein

the step (c) includes

(c1) converting the first 3D coordinate values in the reference coordinate system of the first camera into second 3D coordinate values in a reference coordinate system of the second camera, using the coordinate conversion matrix, and

(c2) setting a position represented by the second 3D coordinate values on a position of the specific image in an image taken by the second camera.

3 . The method according to claim 1 , wherein

the step (b1) includes;

obtaining the depth information at the specific position in the image taken by the first camera, using a pattern with which a 3D position is recognized.

4 . The method according to claim 1 , wherein

the specific position is a horizontal X-Y plane, and

the specific image is an image representing the X-Y plane.

5 . The method according to claim 1 , wherein

the calibration parameter includes a plurality of sets of the calibration parameters, and

the step (c) includes displaying a plurality of the specific images corresponding respectively to the plurality of sets of the calibration parameters on the display device.

6 . The method according to claim 1 , further comprising:

(d) displaying an adjustment tool configured to adjust at least one of a positional posture of the specific image and a value of the calibration parameter on the display device together with the specific image; and

(e) adjusting the value of the calibration parameter in accordance with the adjustment by the user using the adjustment tool.

7 . A system configured to visualize a calibration state of a first camera, the system comprising:

the first camera configured to take an image to be used by a camera-applied device;

a second camera to be used for a confirmation of the calibration state of the first camera, the second camera being attached to VR glasses configured to be worn by a user;

a display device configured to display an image taken by the second camera; and

a control device to be coupled to the first camera, the second camera, and the display device, wherein

the control device is configured to execute:

(a) obtaining a calibration parameter of the first camera;

(b) using the second camera to obtain a relative position between the first camera and the second camera, the (b) further including;

(b1) determining first 3D coordinate values in a reference coordinate system of the first camera with respect to a specific position located within an imaging range of the first camera, the (b1) further including:

obtaining depth information at the specific position in an image taken by the first camera, using a third camera; and

calculating the first 3D coordinate values at the specific position using a pixel coordinate value of the first camera related to the specific position, the depth information, and an internal parameter included in the calibration parameter;

(b2) imaging, by the second camera, a marker with which the relative position to the first camera is determined, to hereby obtain a marker image; and

(b3) determining a coordinate conversion matrix representing the relative position between the first camera and the second camera, using the marker image; and

(c) displaying a specific image representing the calibration state of the first camera on the display device with a positional posture changing with a value of the calibration parameter so as to be superimposed on an image taken by the second camera.

8 . A non-transitory computer-readable storage medium storing a computer program of making a processor execute processing of visualizing a calibration state of a first camera, the processing comprising:

(a) obtaining a calibration parameter of the first camera;

(b) using a second camera to obtain a relative position between the first camera and the second camera, the second camera being attached to VR glasses configured to be worn by a user, the step (b) further including:

(b1) determining first 3D coordinate values in a reference coordinate system of the first camera with respect to a specific position located within an imaging range of the first camera, the step (b1) further including:

obtaining depth information at the specific position in an image taken by the first camera, using a third camera; and

calculating the first 3D coordinate values at the specific position using a pixel coordinate value of the first camera related to the specific position, the depth information, and an internal parameter included in the calibration parameter;

(b2) imaging, by the second camera, a marker with which the relative position to the first camera is determined, to thereby obtain a marker image; and

(b3) determining a coordinate conversion matrix representing the relative position between the first camera and the second camera, using the marker image; and

(c) displaying a specific image representing a calibration state of the first camera on a display device with a positional posture changing with a value of the calibration parameter so as to be superimposed on an image taken by the second camera.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (26)