Granted Patent B2

US 12,372,370 · App. 17/532,846 · Granted Jul 29, 2025

Inventors: Seung Jun Han (Daejeon, KR); Jungyu Kang (Daejeon, KR); Kyoung-Wook Min (Daejeon, KR); Taeghyun An (Daejeon, KR); Jeong Dan Choi (Daejeon, KR)

Assignee: ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE

G01C21/3815G01C21/3833G01C21/3881G06T7/10G06T2207/10028G06T2207/30252

Abstract

Claims (105)

1. A method for generating a map for autonomous driving, comprising:

obtaining three-dimensional (3D) coordinate information corresponding to a 3D space;

obtaining a spherical range image by projecting the 3D coordinate information onto a two-dimensional (2D) plane;

generating a semantic segmented image by performing semantic segmentation on the spherical range image; and

generating map data including the spherical range image, the semantic segmented image, and lane attribute information,

wherein the map has a graph-based structure, the map data is mapped for each node constituting a graph,

wherein the generating of a map data comprises:

obtaining map data of neighboring nodes around a node corresponding to a map data generation point in a state in which map data of the node is generated; and

synthesizing map data of the node and map data of the neighboring nodes and using them as map data for the node,

wherein the synthesizing of map data of the node and map data of the neighboring nodes comprises:

synthesizing a spherical range image of the node and a spherical range image of the neighboring node;

synthesizing a semantic segmented image of the node and a semantic segmented image of the neighboring node;

generating the lane attribute information based on the synthesized semantic segmented image; and

using data including the synthesized spherical range image, the synthesized semantic segmented image, and the lane attribute information as map data for the node.

2. The method of claim 1 , wherein

the generating of a semantic segmented image comprises:

performing semantic segmentation on the spherical range image; and

generating the semantic segmented image including road information by removing a moving object from the image obtained by the semantic segmentation,

wherein the road information is used as segmentation information including a pattern of a road surface, a road pattern, and a pattern around a road, and the attribute information of a land road is obtained using the segmentation information included in the semantic segmented image.

3. The method of claim 1 , wherein edge information between nodes includes pose information, and

the pose information is a result of movement and pose estimation using a gradient of the spherical range image or a gradient of the semantic segmented image.

4. The method of claim 1 , wherein

in the synthesizing of a spherical range image, a synthesized spherical range image is obtained based on a method of selecting a middle value from values accumulated for each pixel, and

in the synthesizing of a semantic segmented image, a synthesized semantic segmented image is obtained based on a method of selecting a value having a largest proportion among values accumulated for each pixel.

5. The method of claim 4 , wherein

when synthesizing spherical range images or synthesizing semantic segmented images,

the synthesizing is performed in a way in which coordinate transformation on an image of the node and an image of the neighboring node is performed based on the node, and a value of a corresponding pixel of the image of the node and a value of a corresponding pixel of the image of the neighboring node are accumulated and stored for each pixel, 2D image data is obtained by selecting one of values accumulated for each pixel, and for each pixel of the 2D image data, a final value of a pixel is obtained based on a result of comparing values of neighboring pixels with values of the pixel.

6. The method of claim 1 , further comprising

compressing the map data,

wherein in the compressing,

for each pixel, the spherical range image and the semantic segmented image is combined into one image and stored, and for a pixel composed of set bits, distance information of the spherical range image is placed in a first bit among the set bits and segmentation information of the semantic segmented image is placed in a second bit among the set bits.

7. The method of claim 6 , wherein

the compressing comprises:

compressing the spherical range image and the semantic segmented image, respectively;

quantizing distance information of the compressed spherical range image, and placing the quantized distance information in an upper bit among the set bits of a pixel; and

quantizing segmentation information of the compressed semantic segmented image, and placing the quantized segmentation information in a lower bit among the set bits of a pixel.

8. The method of claim 1 , further comprising

registering the map data,

wherein the registering comprises:

registering a generated map data as new map data when there is no existing map data in surrounding coordinates of the generated map data; and

updating the existing map data based on the generated map data when there is existing map data in the surrounding coordinates of the generated map data.

9. The method of claim 8 , wherein

the updating comprises:

repeatedly performing an operation of storing the generated map data as additional map data together with time information; and

comparing the stored map data and updating the existing map data based on the additional map data when different changes in the stored map data are maintained for a preset time.

10. The method of claim 1 , further comprising

optimizing the graph constituting a map,

wherein the optimizing comprises:

connecting a global positioning system (GPS) node having GPS data to a corresponding node when GPS data is obtained for each node;

connecting corresponding nodes when loop closure is detected in the graph; and

combining lane attribute information included in each node using pose information.

11. A method for recognizing a location based on a map for autonomous driving, comprising:

obtaining a spherical range image based on data input from a sensor at a current node, and generating a semantic segmented image from the spherical range image;

obtaining an initial location based on global positioning system (GPS) data or a previous driving route;

searching map data from the map based on the initial location; and

obtaining a current location based on the searched map data,

wherein the map data of the map includes the spherical range image, the semantic segmented image, and lane attribute information,

wherein the map has a graph-based structure, the map data is mapped for each node constituting a graph,

the method further comprises:

generating map data including a spherical range image, a semantic segmented image, and lane attribute information,

wherein the generating map data including a spherical range image, a semantic segmented image, and lane attribute information comprises:

obtaining map data of neighboring nodes around a node corresponding to a map data generation point in a state in which map data of the node is generated;

synthesizing a spherical range image of the node and a spherical range image of the neighboring node;

synthesizing a semantic segmented image of the node and a semantic segmented image of the neighboring node;

generating the lane attribute information based on the synthesized semantic segmented image; and

using data including the synthesized spherical range image, the synthesized semantic segmented image, and the lane attribute information as map data for the node.

12. The method of claim 11 , wherein

edge information between nodes includes pose information, and

the pose information is a result of movement and pose estimation using a gradient of the spherical range image or a gradient of the semantic segmented image.

13. The method of claim 12 , wherein

the obtaining of a current location comprises:

estimating pose information of the current node with respect to the searched map data;

obtaining the current location using the map data when the estimated pose information and pose information according to movement of the current node satisfy a setting condition; and

additionally obtaining the current location using the GPS data when there is GPS data in the current node.

14. An apparatus for recognizing a location, comprising:

an interface device; and

a processor connected to the interface device and configured to perform location recognition, wherein

the processor is configured to perform operations by:

projecting three-dimensional (3D) coordinate information corresponding to a 3D space inputted through the interface device onto a two-dimensional (2D) plane to obtain a spherical range image;

generating a semantic segmented image by performing semantic segmentation on the spherical range image;

obtaining an initial location based on global positioning system (GPS) data or a previous driving route;

searching map data from the map based on the initial location; and

obtaining a current location based on the searched map data,

wherein the map data of the map includes the spherical range image, the semantic segmented image, and lane attribute information,

wherein the map has a graph-based structure, the map data is mapped for each node constituting a graph,

wherein

the processor is configured to further perform an operation by

generating map data including a spherical range image, a semantic segmented image, and lane attribute information, and

when performing the operation of generating map data, the processor is specifically configured to perform operations by:

obtaining map data of neighboring nodes around a node corresponding to a map data generation point in a state in which map data of the node is generated;

synthesizing a spherical range image of the node and a spherical range image of the neighboring node;

synthesizing a semantic segmented image of the node and a semantic segmented image of the neighboring node;

generating the lane attribute information based on the synthesized semantic segmented image; and

using data including the synthesized spherical range image, the synthesized semantic segmented image, and the lane attribute information as map data for the node.

15. The apparatus of claim 14 , wherein

edge information between nodes includes pose information, and

the pose information is a result of movement and pose estimation using a gradient of the spherical range image or a gradient of the semantic segmented image.

16. The apparatus of claim 14 , wherein

when performing the operation of generating a semantic segmented image, the processor is specifically configured to perform operations by:

performing semantic segmentation on the spherical range image; and

generating the semantic segmented image including road information by removing a moving object from the image obtained by the semantic segmentation,

wherein the road information is used as segmentation information including a pattern of a road surface, a road pattern, and a pattern around a road, and the attribute information of a land road is obtained using the segmentation information included in the semantic segmented image.

17. The apparatus of claim 14 , wherein

the map data is compressed and stored,

wherein when compressing, for each pixel, the spherical range image and the semantic segmented image are combined into one image and stored, and for a pixel composed of set bits, distance information of the spherical range image is placed in a first bit among the set bits and segmentation information of the semantic segmented image is placed in a second bit among the set bits.

Assignments (1)

Priority Claims (2)

Continuity (1)

References Cited (39)

Cited By (1)

US 12,670,727