Granted Patent B2

US 12,487,090 · App. 17/960,253 · Granted Dec 2, 2025

Inventors: Byoung Tak Zhang (Seoul, KR); Dong Sig Han (Seoul, KR); Hyun Do Lee (Seoul, KR); Jae In Kim (Seoul, KR); Gang Hun Lee (Seoul, KR); Yoon Sung Kim (Seoul, KR)

Assignees: AGENCY FOR DEFENSE DEVELOPMENT; SEOUL NATIONAL UNIVERSITY R&DB FOUNDATION

G01C21/32G01C21/3833G01C21/387G06N3/045G06N3/08G06V10/82

Abstract

Claims (39)

1 . A map merging method for an electronic apparatus, the map merging method comprising:

obtaining information about a first local map of a first apparatus, a pose of the first apparatus in the first local map, a second local map of a second apparatus, a pose of the second apparatus in the second local map, and an image of the second apparatus obtained by the first apparatus;

identifying a relative pose of the second apparatus relative to the first apparatus from the image using a first trained artificial neural network;

transforming the second local map to correspond to the first local map based on the relative pose, the pose of the first apparatus, and the pose of the second apparatus, to align origin and coordinate axes of the second local map with those of the first local map; and

merging the first local map and a transformed second local map transformed in the transforming the second local map to output a merged map,

wherein the image is an image of the second apparatus obtained through an image sensor of the first apparatus,

wherein the first trained artificial neural network includes a first convolutional neural network trained to process an image input thereto and output a relative pose of an apparatus with respect to an image sensor.

2 . The map merging method of claim 1 , wherein the merging the first local map and the transformed second local map includes removing noise in the merged map using a trained autoencoder.

3 . The map merging method of claim 2 , wherein the merging the first local map and the transformed second local map includes removing the noise in the merged map by using a generative adversarial network configured to improve a performance of the trained autoencoder in competition with a neural network that compares restored data and original data in a course of training the trained autoencoder.

4 . The map merging method of claim 1 , wherein the relative pose includes a rotation value with respect to a specific coordinate axis,

the transforming the second local map includes determining an error of the rotation value based on the first local map and the transformed second local map, and correcting the transformed second local map based on the error of the rotation value using a second trained artificial neural network, and

the merging the first local map and the transformed second local map includes merging the first local map and a corrected second local map corrected in the correcting the transformed second local map.

5 . The map merging method of claim 4 , wherein the second trained artificial neural network includes a second convolutional neural network trained to process the first local map and the transformed second local map and output the error of the rotational value of the transformed second local map.

6 . The map merging method of claim 1 , wherein the first local map and the second local map are occupancy grid maps.

7 . The map merging method of claim 1 , further comprising: outputting the merged map in real time.

8 . An electronic apparatus comprising:

a memory configured to store at least one program; and

a processor configured to execute the at least one program having instructions for

obtaining information about a first local map of a first apparatus, a pose of the first apparatus in the first local map, a second local map of a second apparatus, a pose of the second apparatus in the second local map, and an image of the second apparatus obtained by the first apparatus;

identifying a relative pose of the second apparatus relative to the first apparatus from the image using a first trained artificial neural network;

transforming the second local map to correspond to the first local map based on the relative pose, the pose of the first apparatus, and the pose of the second apparatus, to align origin and coordinate axes of the second local map with those of the first local map; and

merging the first local map and a transformed second local map transformed in the transforming the second local map to output a merged map,

wherein the image is an image of the second apparatus obtained through an image sensor of the first apparatus,

wherein the first trained artificial neural network includes a first convolutional neural network trained to process an image input thereto and output a relative pose of an apparatus with respect to an image sensor.

9 . A non-transitory computer-readable medium storing a program that causes a computer to execute a map merging method, the program having instructions for:

obtaining information about a first local map of a first apparatus, a pose of the first apparatus in the first local map, a second local map of a second apparatus, a pose of the second apparatus in the second local map, and an image of the second apparatus obtained by the first apparatus;

identifying a relative pose of the second apparatus relative to the first apparatus from the image using a first trained artificial neural network;

transforming the second local map to correspond to the first local map based on the relative pose, the pose of the first apparatus, and the pose of the second apparatus, to align origin and coordinate axes of the second local map with those of the first local map; and

merging the first local map and a transformed second local map transformed in the transforming the second local map to output a merged map,

wherein the image is an image of the second apparatus obtained through an image sensor of the first apparatus,

wherein the first trained artificial neural network includes a first convolutional neural network trained to process an image input thereto and output a relative pose of an apparatus with respect to an image sensor.

10 . The non-transitory computer-readable medium of claim 9 , wherein the merging the first local map and the transformed second local map includes removing noise in the merged map using a trained autoencoder.

11 . The non-transitory computer-readable medium of claim 10 , wherein the merging the first local map and the transformed second local map includes removing the noise in the merged map by using a generative adversarial network configured to improve a performance of the trained autoencoder in competition with a neural network that compares restored data and original data in a course of training the trained autoencoder.

12 . The non-transitory computer-readable medium of claim 9 , wherein the relative pose includes a rotation value with respect to a specific coordinate axis,

the transforming the second local map includes determining an error of the rotation value based on the first local map and the transformed second local map, and correcting the transformed second local map based on the error of the rotation value using a second trained artificial neural network, and

the merging the first local map and the transformed second local map includes merging the first local map and a corrected second local map corrected in the correcting the transformed second local map.

13 . The non-transitory computer-readable medium of claim 12 , wherein the second trained artificial neural network includes a second convolutional neural network trained to process the first local map and the transformed second local map and output the error of the rotational value of the transformed second local map.

14 . The non-transitory computer-readable medium of claim 9 , wherein the first local map and the second local map are occupancy grid maps.

15 . The non-transitory computer-readable medium of claim 9 , further comprising instructions for: outputting the merged map in real time.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (15)