Granted Patent B2

US 12,639,840 · App. 18/517,255 · Granted May 26, 2026

Inventors: Jung Hyun Lee (Seoul, KR); Tae Hun Lim (Seoul, KR); Dong Hoon Koo (Seoul, KR); Young Hyun Kim (Seoul, KR)

Assignees: HYUNDAI MOTOR COMPANY; KIA COPORATION

G06T7/536G06T7/55G06T2207/20084

Abstract

Claims (52)

1 . A method for estimating a vanishing point, the method comprising:

receiving an input image; and

estimating a vanishing point for the input image, using an artificial intelligence network pre-trained by vanishing point ground truth (GT) data generated based on real image data,

wherein estimating of the vanishing point includes

estimating a depth map or an optical flow map for the input image,

estimating a gradient map for the depth map or the optical flow map,

estimating the vanishing point for the input image based on the gradient map and a predetermined reference gradient map, and

generating the predetermined reference gradient map,

wherein generating the predetermined reference gradient map includes

generating a reference space where there is only a static object and a reference vanishing point,

generating a reference depth map or a reference optical flow map based on the reference space and the reference vanishing point, and

generating a gradient map for the reference depth map or the reference optical flow map to generate the predetermined reference gradient map.

2 . The method of claim 1 , wherein estimating the vanishing point includes estimating a heat map for a static object based on the gradient map and the predetermined reference gradient map and estimating the vanishing point based on the heat map.

3 . The method of claim 2 , wherein estimating the vanishing point includes performing two-dimensional (2D) Gaussian fitting of the heat map for the static object and estimating a center point of 2D Gaussian as the vanishing point.

4 . The method of claim 2 , wherein the artificial intelligence network includes a network for estimating a keypoint corresponding to the vanishing point in the heat map.

5 . The method of claim 1 , wherein estimating the depth map or the optical flow map includes:

detecting at least one predetermined dynamic object from the input image; and

estimating a depth map of a static object except for an area of the detected at least one predetermined dynamic object, when estimating the depth map.

6 . The method of claim 1 , wherein the artificial intelligence network includes CenterNet.

7 . The method of claim 1 , wherein the vanishing point GT data is generated based on a heat map and a keypoint, when a depth map or an optical flow map for an image is generated, when a gradient map for the depth map or the optical flow map is generated, when the heat map for the gradient map is generated by means of Gaussian fitting of the gradient map and the keypoint is generated using a Gaussian center point by means of the Gaussian fitting, and when coordinates for the Gaussian center point are identical to coordinates of a vanishing point detected by at least one predetermined vanishing point detection technique.

8 . An apparatus for estimating a vanishing point, the apparatus comprising:

a receiver configured to receive an input image;

a generator; and

an estimation device configured to estimate a vanishing point for the input image, using an artificial intelligence network pre-trained by vanishing point GT data generated based on real image data,

wherein the estimation device is configured to estimate a depth map or an optical flow map for the input image, estimate a gradient map for the depth map or the optical flow map, and estimate the vanishing point for the input image based on the gradient map and a predetermined reference gradient map,

wherein the generator is configured to:

generate the predetermined reference gradient map;

generate a reference space where there is only a static object and a reference vanishing point;

generate a reference depth map or a reference optical flow map based on the reference space and the reference vanishing point; and

generate a gradient map for the reference depth map or the predetermined reference gradient map to generate the predetermined reference gradient map.

9 . The apparatus of claim 8 , wherein the estimation device is further configured to estimate a heat map for a static object based on the gradient map and the predetermined reference gradient map and estimate the vanishing point based on the heat map.

10 . The apparatus of claim 9 , wherein the estimation device is further configured to perform 2D Gaussian fitting of the heat map for the static object and estimate a center point of 2D Gaussian as the vanishing point.

11 . The apparatus of claim 9 , wherein the artificial intelligence network includes a network for estimating a keypoint corresponding to the vanishing point in the heat map.

12 . The apparatus of claim 8 , wherein the estimation device is further configured to detect at least one predetermined dynamic object from the input image and estimate a depth map of a static object except for an area of the detected at least one predetermined dynamic object, when estimating the depth map.

13 . The apparatus of claim 8 , wherein the artificial intelligence network includes CenterNet.

14 . The apparatus of claim 8 , wherein the vanishing point GT data is generated based on a heat map and a keypoint, when a depth map or an optical flow map for an image is generated, when a gradient map for the depth map or the optical flow map is generated, when the heat map for the gradient map is generated by means of Gaussian fitting of the gradient map and the keypoint is generated using a Gaussian center point by means of the Gaussian fitting, and when coordinates for the Gaussian center point are identical to coordinates of a vanishing point detected by at least one predetermined vanishing point detection technique.

15 . A method for generating vanishing point GT, the method comprising:

generating a depth map or an optical flow map for an input image;

generating a gradient map for the depth map or the optical flow map;

generating a heat map for the gradient map by means of Gaussian fitting of the gradient map and generating a keypoint using a Gaussian center point by means of the Gaussian fitting;

generating the vanishing point GT based on the heat map and a key point, when coordinates for the Gaussian center point are identical to coordinates of a vanishing point detected by at least one predetermined vanishing point detection technique;

receiving the input image; and

estimating the vanishing point for the input image, using an artificial intelligence network pre-trained by the vanishing point GT data generated based on real image data,

wherein estimating of the vanishing point includes

estimating the depth map or the optical flow map for the input image,

estimating the gradient map for the depth map or the optical flow map,

estimating the vanishing point for the input image based on the gradient map and a predetermined reference gradient map, and

generating the predetermined reference gradient map,

wherein generating the predetermined reference gradient map includes

generating a reference space where there is only a static object and a reference vanishing point,

generating a reference depth map or a reference optical flow map based on the reference space and the reference vanishing point, and

generating a gradient map for the reference depth map or the reference optical flow map to generate the predetermined reference gradient map.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (22)