IP Library Granted Patent US 11,587,261
Granted Patent B2
US 11,587,261 · App. 15/905,618 · Granted Feb 21, 2023

Image processing apparatus and ranging apparatus

Inventors: Jun Yamaguchi (Kawasaki Kanagawa, JP); Yusuke Moriuchi (Yokohama Kanagawa, JP); Nao Mishima (Inagi Tokyo, JP)
Assignee: Kabushiki Kaisha Toshiba
G06T7/97G06T3/40G06T5/003G06T5/006G06T7/55G06T2200/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,587,261
App. No.
15/905,618
Granted
Feb 21, 2023
Kind
B2
Abstract

According to one embodiment, an image processing apparatus includes a buffer and processing circuitry. The buffer stores first and second images capturing an object. The circuitry calculates at least one of a first distance to the object in the first image and a second distance to the object in the second image by using a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object, calculates three-dimensional coordinates of the object on a relative scale by using the first and second images, and calculates three-dimensional coordinates of the object on a real scale based on at least one of the first and second distances, and the three-dimensional coordinates of the object on the relative scale.

Claims (41)

1. An image processing apparatus comprising:

a buffer configured to store a first image capturing an object and a second image capturing the object, the first image being different from the second image, the first image and the second image being captured by an image capturing device; and

processing circuitry configured to:

calculate at least one of a first distance to the object in the first image and a second distance to the object in the second image, the first and second distances each including a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object;

calculate three-dimensional coordinates of the object on a relative scale by using only the first image and the second image;

calculate the correction parameter and a scale parameter for converting the three-dimensional coordinates on the relative scale into three-dimensional coordinates on a real scale so that at least one of the first distance and the second distance, and the three-dimensional coordinates of the object on the relative scale indicate a same position in a real space; and

calculate the three-dimensional coordinates of the object on the real scale by using the calculated correction parameter and the calculated scale parameter.

2. The image processing apparatus of claim 1 , wherein the processing circuitry is configured to calculate the three-dimensional coordinates of the object on the real scale based on the calculated scale parameter and the three-dimensional coordinates on the relative scale.

3. The image processing apparatus of claim 1 ,

wherein the first image comprises a first wavelength component image and a second wavelength component image,

the second image comprises a third wavelength component image and a fourth wavelength component image,

the processing circuitry is further configured to:

calculate a first blur correction amount used for correcting a blur included in the first wavelength component image to a blur included in the second wavelength component image;

calculate a second blur correction amount used for correcting a blur included in the third wavelength component image to a blur included in the fourth wavelength component image;

calculate the first distance based on the first blur correction amount; and

calculate the second distance based on the second blur correction amount.

4. The image processing apparatus of claim 3 , wherein:

at least one of the blur included in the first wavelength component image and the blur included in the second wavelength component image has a point-asymmetric shape, and

at least one of the blur included in the third wavelength component image and the blur included in the fourth wavelength component image has a point-asymmetric shape.

5. The image processing apparatus of claim 3 , wherein the processing circuitry is configured to:

calculate the correction parameter and a scale parameter that is used for converting the three-dimensional coordinates on the relative scale into the three-dimensional coordinates on the real scale by using at least one of the first distance and the second distance, and the three-dimensional coordinates on the relative scale; and

calculate the three-dimensional coordinates of the object on the real scale based on the calculated correction parameter and the first blur correction amount or based on the calculated correction parameter and the second blur correction amount.

6. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter for each corresponding point between the first image and the second image.

7. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter for each area having a similar color on the first image or the second image.

8. The image processing apparatus of claim 5 , wherein the processing circuitry is configured to calculate the correction parameter common in pixels of the first image and pixels of the second image.

9. The image processing apparatus of claim 1 , wherein the processing circuitry is further configured to calculate one or more parameters indicative of a relative positional relationship between a position and posture at which the first image is captured and a position and posture at which the second image is captured.

10. The image processing apparatus of claim 1 , wherein distance information to the object at a time of capturing is encoded in each of the first image and the second image.

11. The image processing apparatus of claim 1 , wherein the first image and the second image are both obtained by an optical system.

12. A ranging apparatus comprising:

an image processing apparatus comprising:

a buffer configured to store a first image capturing an object and a second image capturing the object, the first image different from the second image; and

a processing circuitry configured to:

calculate at least one of a first distance to the object in the first image and a second distance to the object in the second image, the first and second distances each including a correction parameter for correcting at least one of influences caused by ambient light, a reflection characteristic of the object, or a color of the object;

calculate three-dimensional coordinates of the object on a relative scale by using only the first image and the second image;

calculate the correction parameter and a scale parameter for converting the three-dimensional coordinates on the relative scale into three-dimensional coordinates on a real scale so that at least one of the first distance and the second distance, and the three-dimensional coordinates of the object on the relative scale indicate a same position in a real space; and

calculate the three-dimensional coordinates of the object on the real scale by using the calculated correction parameter and the calculated scale parameter;

an image capturing device configured to capture the first image and the second image.

13. The ranging apparatus of claim 12 , wherein the image capturing device comprises:

a lens;

an image sensor comprising at least two types of imaging elements configured to receive light rays having different wavelength bands; and

a filter provided on an optical path of light rays penetrating the lens and entering the image sensor, comprising at least two filter areas which transmit light rays having different wavelength bands.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 26, 2018
From: YAMAGUCHI, JUN; MORIUCHI, YUSUKE; MISHIMA, NAO
To: KABUSHIKI KAISHA TOSHIBA
Reel/Frame 045447/0012 →
Priority Claims (1)
JP JP2017173346 · Sep 8, 2017 · national
Continuity (1)
Related Publication 20190080481A1 · Mar 14, 2019