Granted Patent B2

US 12,327,365 · App. 17/587,121 · Granted Jun 10, 2025

Inventor: Qingqing Xie (Guangdong, CN)

Assignee: AUTEL ROBOTICS CO., LTD.

G06T7/40G06T7/337G06T7/593

Abstract

Claims (67)

1. A stereo matching method, comprising:

calculating aggregate cost values between all reference pixels and a target pixel in a preset search region, the reference pixel being a pixel in a reference image, and the target pixel being a pixel in a target image;

determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values; and

obtaining a disparity value between the reference image and the target image at a position of the target pixel by using a method for calculating a disparity value corresponding to the texture property, wherein the texture property comprises: a normal region, a repeated texture region and a texture-less region,

wherein the determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values comprises:

counting low-cost pixels in a low aggregation cost interval in which the aggregate cost values are located, the low aggregation cost interval being defined by a lowest value among the aggregate cost values;

calculating a proportion of the low-cost pixels in the reference pixels in the search region; and

determining that the texture property is the texture-less region, when the proportion is greater than a preset proportion threshold.

2. The stereo matching method according to claim 1 , wherein when the texture property is the texture-less region, the using a method for calculating a disparity value corresponding to the texture property, to obtain a disparity value between the reference image and the target image at a position of the target pixel comprises:

setting the disparity value at the position of the target pixel to an invalid value.

3. The stereo matching method according to claim 1 , wherein a lower limit of the low aggregation cost interval is the lowest value among the aggregate cost values, and an upper limit of the low aggregation cost interval is 1.5 times of the lowest value among the aggregate cost values.

4. The stereo matching method according to claim 1 , wherein the determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values comprises:

determining whether low-cost pixels in a low aggregation cost interval in which the aggregate cost values are located are unique, the low aggregation cost interval being defined by a lowest value among the aggregate cost values; and

determining the texture property is the normal region if yes.

5. The stereo matching method according to claim 4 , wherein when the texture property is the normal region, the using a method for calculating a disparity value corresponding to the texture property, to obtain a disparity value between the reference image and the target image at a position of the target pixel comprises:

determining that a reference pixel with a lowest aggregate cost value is a corresponding pixel of the target pixel in the reference image; and

using a distance between the corresponding pixel and the target pixel as the disparity value.

6. The stereo matching method according to claim 1 , wherein the determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values of the pixels comprises:

counting low-cost pixels in a low aggregation cost interval in which the aggregate cost values are located, the low aggregation cost interval being defined by a lowest value among the aggregate cost values;

calculating a proportion of the low-cost pixels in the reference pixels in the search region; and

determining that the texture property is a repeated texture region, when the proportion is less than a preset proportion threshold and a quantity of the low-cost pixels is greater than a preset quantity threshold.

7. The stereo matching method according to claim 6 , wherein when the texture property is the repeated texture region, the using a method for calculating a disparity value corresponding to the texture property, to obtain a disparity value between the reference image and the target image at a position of the target pixel comprises:

updating a size of the search region according to a distribution width of the low-cost pixels; and

recalculating aggregate cost values between all reference pixels and a target pixel in the updated search region.

8. The stereo matching method according to claim 7 , wherein the updating a size of the search region according to a distribution width of the low-cost pixels comprises:

in the reference image, calculating a distance between a low-cost pixel at a maximum horizontal coordinate and a low-cost pixel at a minimum horizontal coordinate; and

using the distance as a length of the updated search region.

9. The stereo matching method according to claim 1 , wherein when the texture property is a normal region, the method further comprises:

performing, by quadratic curve fitting, subpixel refinement on the disparity value at the position of the target pixel.

10. The stereo matching method according to claim 9 , wherein the performing, by quadratic curve fitting, subpixel refinement on the disparity value at the position of the target pixel comprises:

determining a reference pixel with a lowest aggregate cost value as an optimal pixel;

using an aggregate cost value of the optimal pixel and aggregate cost values of two reference pixels adjacent to the optimal pixel in the reference image to fit a quadratic curve; and

using a horizontal coordinate corresponding to a minimum point of the quadratic curve obtained through fitting as a refined disparity value.

11. The stereo matching method according to claim 1 , wherein the calculating aggregate cost values between all reference pixels and a target pixel in a preset search region comprises:

performing grayscale value sampling in the target image by using a first sampling window with a preset side length, to obtain a first sampling result, a center of the first sampling window being the target pixel;

performing grayscale value sampling in the reference image by using a second sampling window with the same size as the first sampling window, to obtain a second sampling result, a center of the second sampling window being the reference pixel;

calculating a difference between grayscale values of the first sampling result and the second sampling result at corresponding positions; and

superposing differences between all the grayscale values, to obtain an aggregate cost value of the reference pixels.

12. The stereo matching method according to claim 11 , wherein the target image is in epipolar alignment with the reference image, and the preset search region is a region between a search lower limit pixel and a search upper limit pixel;

two-dimensional coordinates of the search lower limit pixel in the reference image are (x0+k1, y0), and two-dimensional coordinates of the search upper limit pixel in the reference image are (x0+k2, y0); and two-dimensional coordinates of the target pixel in the target image are (x0, y0), and k1 and k2 are constant values greater than 0.

13. An image processing chip, comprising:

at least one processor;

and a memory, communicatively connected to the at least one processor,

the memory storing computer-executable program instructions, the computer-executable program instructions, when invoked, causing the at least one processor to perform:

calculating aggregate cost values between all reference pixels and a target pixel in a preset search region, the reference pixel being a pixel in a reference image, and the target pixel being a pixel in a target image;

determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values;

obtaining a disparity value between the reference image and the target image at a position of the target pixel by using a method for calculating a disparity value corresponding to the texture property; and

obtaining a disparity space image according to the disparity value, wherein the texture property comprises: a normal region, a repeated texture region and a texture-less region,

wherein the determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values comprises:

counting low-cost pixels in a low aggregation cost interval in which the aggregate cost values are located, the low aggregation cost interval being defined by a lowest value among the aggregate cost values;

calculating a proportion of the low-cost pixels in the reference pixels in the search region; and

determining that the texture property is the texture-less region, when the proportion is greater than a preset proportion threshold.

14. A mobile vehicle, comprising:

at least a pair of cameras, configured to capture first image information and second image information respectively; and

an image processing chip, configured to: receive the first image information and the second image information, the image processing chip comprising

at least one processor;

and a memory, communicatively connected to the at least one processor, the memory storing computer-executable program instructions, the computer-executable program instructions, when invoked, causing the at least one processor to perform:

calculating aggregate cost values between all reference pixels and a target pixel in a preset search region, the reference pixel being a pixel in a reference image, and the target pixel being a pixel in a target image;

determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values;

obtaining a disparity value between the reference image and the target image at a position of the target pixel by using a method for calculating a disparity value corresponding to the texture property; and

obtaining a disparity space image according to the disparity value; and

converting the disparity space image into corresponding depth information according to internal parameters and external parameters of the cameras, wherein the texture property comprises: a normal region, a repeated texture region and a texture-less region,

wherein the determining a texture property of the reference image in the search region according to a distribution of the aggregate cost values comprises:

counting low-cost pixels in a low aggregation cost interval in which the aggregate cost values are located, the low aggregation cost interval being defined by a lowest value among the aggregate cost values;

calculating a proportion of the low-cost pixels in the reference pixels in the search region; and

determining that the texture property is the texture-less region, when the proportion is greater than a preset proportion threshold.

15. The mobile vehicle according to claim 14 , wherein optical axes of the pair of cameras are parallel, and the first image information and the second image information that are captured are image information with epipolar alignment.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (23)