Image processing device and image processing method

1. An image processing device, comprising:

a memory that stores a range image from a captured image; and

a processor that performs operations including:

calculating a three-dimensional coordinate in each pixel of the range image;

detecting a reference surface having a predetermined height in the range image based on the three-dimensional coordinate of each pixel;

detecting an object region existing on the reference surface in the range image based on the three-dimensional coordinate of each pixel;

deciding, for determining a depth of the object region, a depth of each vertical line in the object region based on the three-dimensional coordinate of a pixel on the vertical line;

calculating a change amount of depth between adjacent vertical lines in the object region;

grouping a set of vertical lines in which the change amount of depth is less than or equal to a predetermined threshold;

describing each group in a predetermined data format; and

determining an entry region as a place where a difference of depths between a right end of a certain group and a left end of an adjacent group is within a predetermined range.

2. The image processing device according to claim 1 , wherein the processor describes each group by three-dimensional coordinates for four points including upper ends and lower ends of the vertical lines at both ends of the group.

3. The image processing device according to claim 2 , wherein the processor calculates an average value of a depth of each pixel included in the object region as the depth of the object region.

4. The image processing device according to claim 1 , wherein the processor describes each group by two-dimensional coordinates on the range image for four points including upper ends and lower ends of the vertical lines at both ends of the group and depths of regions at both of the ends of the group.

5. The image processing device according to claim 4 , wherein the processor calculates an average value of a depth of each pixel included in the object region as the depth of the object region.

6. The image processing device according to claim 1 , wherein the processor describes each group by two-dimensional coordinates on the range image for four points including upper ends and lower ends of the vertical lines of each group and the depth of each vertical line obtained by linear interpolation of depths of regions at both of the ends of the group.

7. The image processing device according to claim 6 , wherein the processor calculates an average value of a depth of each pixel included in the object region as the depth of the object region.

8. The image processing device according to claim 1 , wherein the processor increases weighting of a pixel closer to an upper end or a lower end of regions at both ends of a group, and calculates a weighting average of a depth of each pixel as the depth of the object region.

9. The image processing device according to claim 1 , wherein the predetermined threshold is determined according to the depth of each vertical line.

10. The image processing device according to claim 1 , wherein the predetermined threshold is determined according to the depth of each vertical line and a detection error amount of a depth of the range image.

11. The image processing device according to claim 1 , wherein, different values are used as the predetermined threshold in a case where the depth of the object region changes from a front toward a depth between the adjacent vertical lines when viewed from a left to a right and a case where the depth of the object region changes from the depth toward the front.

12. The image processing device according to claim 1 , further comprising:

a camera that captures the image,

wherein the camera captures a color image or a grayscale image in which the range image is correlated with a positional relationship between pixels, and

the processor weights the predetermined threshold according to a difference in color or luminance of the object region between the adjacent vertical lines.

13. The image processing device according to claim 1 , wherein the processor calculates a correction amount, for correcting an error of a self-position, by comparing a position of the entry region and cartographic information.

14. An image processing method, comprising:

obtaining a range image from a captured image;

calculating a three-dimensional coordinate in each pixel of the range image;

detecting a reference surface having a predetermined height in the range image based on the three-dimensional coordinate of each pixel;

detecting an object region existing on the reference surface in the range image based on the three-dimensional coordinate of each pixel;

deciding a depth of each vertical line in the object region based on the three-dimensional coordinate of a pixel on the vertical line;

calculating a change amount of depth between adjacent vertical lines in the object region;

grouping a set of vertical lines in which the change amount of depth is less than or equal to a predetermined threshold;

describing each group in a predetermined data format; and

determining an entry region as a place where a difference of depths between a right end of a certain group and a left end of an adjacent group is within a predetermined range.

15. An image processing device, comprising:

a range image obtaining unit that obtains a range image from a captured image;

a coordinate calculator that calculates a three-dimensional coordinate in each pixel of the range image;

a reference surface detector that detects a reference surface having a predetermined height in the range image based on the three-dimensional coordinate of each pixel;

an object region detector that detects an object region existing on the reference surface in the range image based on the three-dimensional coordinate of each pixel;

a depth calculator that decides a depth of each vertical line in the object region based on the three-dimensional coordinate of a pixel on the vertical line; and

a grouping unit that calculates a change amount of depth between adjacent vertical lines in the object region, groups a set of vertical lines in which the change amount of depth is less than or equal to a predetermined threshold, and describes each group in a predetermined data format,

wherein the grouping unit determines that a place where a difference of depth between a right end of a certain group and a left end of an adjacent group falls within a predetermined range as an entry region.

16. The image processing device according to claim 15 , wherein a correction amount correcting an error of a self-position is calculated by comparing a position of the entry region and cartographic information.