Method, apparatus, and system for providing real-world distance information from a monocular image
An approach is provided for estimating a real-world depth information from a monocular image. The approach, for example, involves determining a vanishing point of the monocular image captured by a camera. The approach also involves generating a vanishing point ray from an optical center of the camera through the vanishing point on an image plane of the monocular image to infinity. The approach further involves generating a center line ray from the optical center through a geometric center of the image plane to a feature line that is parallel to the vanishing point ray at a lateral distance. The approach further involves generating a feature ray from the optical center through a location of the feature on the image plane to the feature line. The approach further involves computing the real-world distances of the feature based on image coordinates of the rays, lines, angles derived therefrom, and a known pixel-wise distance of the monocular image.
1. A method for computing real-world distance information from a monocular image comprising:
determining a vanishing point of the monocular image captured by a camera;
generating a vanishing point ray from an optical center of the camera through the vanishing point on an image plane of the monocular image to infinity;
generating a center line ray from the optical center of the camera through a geometric center of the image plane to a feature line, wherein the feature line is parallel to the vanishing point ray at a lateral distance corresponding to a feature location of a feature depicted in the monocular image;
generating a feature ray from the optical center of the camera through a feature pixel location of the feature on the image plane to the feature location on the feature line; and
computing a horizontal distance, a depth, or a combination of the feature location based on image coordinate data corresponding to the vanishing point ray, the center line ray, the feature ray, one or more angles derived therefrom, and a known pixel-wise distance of the monocular image.
2. The method of claim 1 , wherein the known pixel-wise distance is determined based on a field of view and a focal length of the camera to generate the horizontal distance, the depth, or a combination thereof.
3. The method of claim 1 , wherein the horizontal distance, the depth, or a combination thereof is determined with respect to a top view, a side view, or a combination thereof of the vanishing point ray, the center line ray, the feature ray, or a combination thereof.
4. The method of claim 1 , wherein the vanishing point ray, the center line ray, the feature ray, or a combination thereof are further based on a height of the camera from a surface plane.
5. The method of claim 1 , wherein the feature line corresponds to a road edge of a road depicted in the monocular image.
6. The method of claim 5 , further comprising:
generating a central road line passing through a center of the road;
generating a first line from the feature as depicted in the monocular image to the central road line, wherein the first line is perpendicular to the central road line;
generating a second line from the feature as depicted in the monocular image to the road edge, wherein the second line is perpendicular to the road edge; and
mapping the feature as depicted in the monocular image to the feature pixel location on the feature line based on an angle formed by the first line and the second line.
7. The method of claim 1 , further comprising:
computing a camera pose of the camera based on the vanishing point ray, the center line ray, the feature ray, one or more angles derived therefrom, and a known pixel-wise distance of the monocular image.
8. The method of claim 1 , further comprising:
providing the horizontal distance, the depth, or a combination thereof as an output for use by a location-based application or service.
9. The method of claim 1 , wherein the feature is a base of a pole-like feature.
10. The method of claim 1 , wherein the vanishing point is determined based on a segmenting of the monocular image into road pixels and non-road pixels.
11. An apparatus for determining a vanishing point of an image comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
segment the image into a plurality of road pixels and a plurality of non-road pixels;
extract a road mask from the image based on the segmenting, wherein the road mask comprises the plurality of road pixels;
determine a closest pixel of the road mask to a horizon of the image; and
provide the closest pixel as an initial estimation of the vanishing point of the image.
12. The apparatus of claim 11 , wherein the apparatus is further caused to:
iteratively compute a subsequent estimation of the vanishing point at an offset from the initial estimation or a previous estimation of the vanishing point until a difference between the initial estimation or the previous estimation and the subsequent estimation is below a threshold value.
13. The apparatus of claim 12 , wherein the apparatus is further caused to:
determine a left edge and a right edge of the road based on the road mask,
wherein the subsequent estimation is computed based on a mid-point between the left edge and the right edge of the road at the offset.
14. The apparatus of claim 12 , wherein the difference is based on a Euclidean distance between the initial estimation or the previous destination and the subsequent estimation of the vanishing point.
15. The apparatus of claim 11 , wherein the apparatus is further caused to:
provide the vanishing point as an output for estimating real-world depth information from a monocular image.
16. A non-transitory computer-readable storage medium for computing real-world distance information from a monocular image, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:
determining a vanishing point of the monocular image captured by a camera;
generating a vanishing point ray from an optical center of the camera through the vanishing point on an image plane of the monocular image to infinity;
generating a center line ray from the optical center of the camera through a geometric center of the image plane to a feature line, wherein the feature line is parallel to the vanishing point ray at a lateral distance corresponding to a feature location of a feature depicted in the monocular image;
generating a feature ray from the optical center of the camera through a feature pixel location of the feature on the image plane to the feature location on the feature line; and
computing a horizontal distance, a depth, or a combination of the feature location based on image coordinate data corresponding to the vanishing point ray, the center line ray, the feature ray, one or more angles derived therefrom, and a known pixel-wise distance of the monocular image.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the known pixel-wise distance is determined based on a field of view and a focal length of the camera to generate the horizontal distance, the depth, or a combination thereof.
18. The non-transitory computer-readable storage medium of claim 16 , wherein the horizontal distance, the depth, or a combination thereof is determined with respect to a top view, a side view, or a combination thereof of the vanishing point ray, the center line ray, the feature ray, or a combination thereof.
19. The non-transitory computer-readable storage medium of claim 16 , wherein the vanishing point ray, the center line ray, the feature ray, or a combination thereof are further based on a height of the camera from a surface plane.
20. The non-transitory computer-readable storage medium of claim 16 , wherein the feature line corresponds to a road edge of a road depicted in the monocular image.