IP Library Granted Patent US 11,973,921
Granted Patent B2
US 11,973,921 · App. 17/322,093 · Granted Apr 30, 2024

Image processing apparatus

Inventor: Akitaka Oko (Tokyo, JP)
Assignee: SUBARU CORPORATION
H04N13/106G06T5/006G06T5/50G06T7/246G06T7/285G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,973,921
App. No.
17/322,093
Granted
Apr 30, 2024
Kind
B2
Abstract

An image processing apparatus includes a corresponding point calculator and a determination unit. The corresponding point calculator identifies corresponding points relating to an object based on left and right images in each stereo image. The corresponding points include a right image point in the right image and a left image point in the left image. The determination unit predicts a first position of the left image point in a second stereo image, and a second position of the right image point in a second stereo image, based on a position of the left image point in a first stereo image and a position of the right image point in the first stereo image, and determines whether the object is a mobile body based on a difference between the first position predicted and the first position identified, and a difference between the second position predicted and the second position identified.

Claims (39)

1. An image processing apparatus comprising:

a corresponding point calculator configured to identify corresponding points relating to an object on a basis of a left image and a right image included in each of a plurality of stereo images, the corresponding points including a left image point in the left image and a right image point in the right image, the left image point and the right image point being correlated with each other, the plurality of stereo images including a first stereo image and a second stereo image subsequent to the first stereo image; and

a determination unit configured to

predict a first position and a second position relating to the object in a future second stereo image on a basis of a position of the left image point in the left image of the first stereo image and a position of the right image point in the right image of the first stereo image under an assumption that the object corresponding to the right and left image points is a static object, the first position being a position of the left image point in the left image of the future second stereo image, the second position being a position of the right image point in the right image of the future second stereo image, and

determine whether the object is a mobile body on a basis of a first difference between the first position predicted in the future second stereo image and the first position in the second stereo image identified by the corresponding point calculator, and a second difference between the second position predicted in the future second stereo image and the second position in the second stereo image identified by the corresponding point calculator, wherein the object is determined as a mobile body in a case where at least one of the first difference and the second difference is greater than a predetermined threshold.

2. The image processing apparatus according to claim 1 , further comprising:

an image corrector configured to

perform image correction that shifts positions of pixel values of the left image and the right image on a basis of correction map data, and

supply the left image and the right image to the corresponding point calculator after the image correction; and

a generator configured to

generate a parallax correction value on a basis of information on the corresponding points identified by the corresponding point calculator and a result of detection by a sensor that detects a position and an orientation of a stereo camera that generates the plurality of stereo images,

accumulate the parallax correction value to obtain a plurality of the parallax correction values, and

generate the correction map data on a basis of one or more parallax correction values out of the plurality of the parallax correction values, the one or more parallax correction values excluding parallax correction values correlated with the object determined as a mobile body by the determination unit.

3. The image processing apparatus according to claim 2 , wherein

the plurality of stereo images further includes a third stereo image and a fourth stereo image, and

the determination unit is further configured to determine whether the object is a mobile body on a basis of a position of the left image point in the left image of the third stereo image, a position of the right image point in the right image of the third stereo image, a position of the left image point in the left image of the fourth stereo image, a position of the right image point in the right image of the fourth stereo image, and the result of detection by the sensor.

4. The image processing apparatus according to claim 2 , further comprising a distance image generator configured to generate a distance image, the distance image generator including the corresponding point calculator.

5. The image processing apparatus according to claim 3 , further comprising a distance image generator configured to generate a distance image, the distance image generator including the corresponding point calculator.

6. The image processing apparatus according to claim 1 , further comprising:

an image corrector configured to correct the left image or the right image on a basis of a correction map data, wherein the correction map data is generated on a basis of accumulated correction values after deleting a correction value correlated with the object determined as a mobile body by the determination unit from the accumulated correction values.

7. An image processing apparatus comprising:

circuitry configured to

identify corresponding points relating to an object on a basis of a left image and a right image included in each of a plurality of stereo images, the corresponding points including a left image point in the left image and a right image point in the right image, the left image point and the right image point being correlated with each other, the plurality of stereo images including a first stereo image and a second stereo image subsequent to the first stereo image,

predict a first position and a second position relating to the object in a future the second stereo image on a basis of a position of the left image point in the left image of the first stereo image and a position of the right image point in the right image of the first stereo image under an assumption that the object corresponding to the right and left image points is a static object, the first position being a position of the left image point in the left image of the future second stereo image, the second position being a position of the right image point in the right image of the future second stereo image, and

determine whether the object is a mobile body on a basis of a first difference between the first position predicted in the future second stereo image and the first position identified in the second stereo image, and a second difference between the second position predicted in the future second stereo image and the second position identified in the second stereo image, wherein the object is determined as a mobile body in a case where at least one of the first difference and the second difference is greater than a predetermined threshold.

8. The image processing apparatus according to claim 2 , further comprising

a recognizer configured to make recognition of the object on a basis of the left image and the right image, wherein

the determination unit is further configured to determine whether the object is a mobile body on a basis of a result of the recognition by the recognizer.

9. The image processing apparatus according to claim 3 , further comprising:

a recognizer configured to make recognition of the object on a basis of the left image and the right image, wherein

the determination unit is further configured to determine whether the object is a mobile body on a basis of a result of the recognition by the recognizer.

10. The image processing apparatus according to claim 8 , further comprising

a distance image generator configured to generate a distance image, the distance image generator including the corresponding point calculator.

11. The image processing apparatus according to claim 9 , further comprising

a distance image generator configured to generate a distance image, the distance image generator including the corresponding point calculator.

12. The image processing apparatus according to claim 7 , wherein the circuitry is further configured to:

delete a correction value correlated with the object determined as a mobile body from accumulated correction values;

generate a correction map data on a basis of the accumulated correction values; and

correct the left image or the right image on a basis of the correction map data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2021
From: OKO, AKITAKA
To: SUBARU CORPORATION
Reel/Frame 056261/0112 →
Priority Claims (1)
JP 2020-089002 · May 21, 2020 · national
Continuity (1)
Related Publication 20210368151A1 · Nov 25, 2021
Cited By (1)
US 12,354,293