Granted Patent B2

US 12,626,397 · App. 18/106,184 · Granted May 12, 2026

Inventors: Xiongfeng Peng (Beijing, CN); Zhihua Liu (Beijing, CN); Qiang Wang (Beijing, CN); Yuntae Kim (Suwon-si, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06T7/74G06F16/538G06F16/55G06F16/5854G06T7/60

Abstract

Claims (90)

1 . A simultaneous localization and mapping (SLAM) method performed by an electronic device, the method comprising:

acquiring a search image based on a query image;

acquiring first spatial features of the query image and second spatial features of the search image;

acquiring feature matching results by matching at least one of the first spatial features with at least one of the second spatial features; and

estimating a relative pose between the query image and the search image based on the feature matching results,

wherein the acquiring feature matching results comprises:

generating one or more first feature matching pairs by performing a coarse matching operation based on the first spatial features and the second spatial features, and

generating one or more second feature matching pairs by performing a fine matching operation based on the first spatial features and the second spatial features.

2 . The SLAM method of claim 1 ,

wherein the first spatial features and the second spatial features each comprise three dimensional (3D) point sets, and

wherein the acquiring the first spatial features and the second spatial features each comprises:

extracting image feature points that comprise image keypoints and feature descriptors; and

estimating the 3D point sets by performing stereo matching on the image feature points.

3 . The SLAM method of claim 1 ,

wherein the acquiring the feature matching results comprises generating the one or more first feature matching pairs between results of clustering of the query image and results of clustering of the search image by clustering the 3D point sets of the query image and the 3D point sets of the search image.

4 . The SLAM method of claim 3 , wherein the generating of the one or more first feature matching pairs comprises:

determining one or more first cubes by clustering the 3D point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image;

determining first cluster centroids of the respective first cubes and second cluster centroids of the respective second cubes;

determining the second cluster centroids that respectively match the first cluster centroids; and

determining the one or more first feature matching pairs based on the first cluster centroids and the second cluster centroids determined to match each other.

5 . The SLAM method of claim 3 ,

wherein the acquiring the feature matching results further comprises acquiring the one or more second feature matching pairs between the 3D point sets of the query image and the 3D point sets of the search image by performing nearest neighbor search and mutual verification on 3D points of the one or more first feature matching pairs, and

wherein the determining the relative pose comprises determining the relative pose based on the one or more second feature matching pairs.

6 . The SLAM method of claim 5 , wherein the feature matching results further comprises third feature matching pairs,

wherein the relative pose comprises a coarse relative pose and a fine matching pose,

wherein the acquiring the feature matching results further comprises:

estimating the coarse relative pose between the query image and the search image based on the one or more second feature matching pairs; and

determining the third feature matching pairs between the 3D point sets of the query image and the 3D point sets of the search image by projecting the 3D point sets of the search image onto a coordinate system of the query image according to the coarse relative pose, and

wherein the determining of the fine relative pose comprises determining the relative pose based on the third feature matching pairs.

7 . The SLAM method of claim 1 , wherein the determining of the relative pose comprises:

estimating a prior relative pose between the query image and the search image based on the feature matching results;

determining local points of the search image corresponding to keypoints of the query image based on the prior relative pose, and generating point matching pairs based on the local points corresponding to the keypoints; and

estimating the relative pose based on the point matching pairs.

8 . The SLAM method of claim 1 , further comprising acquiring an optimized global map by optimizing a current global map based on the relative pose.

9 . The SLAM method of claim 8 , wherein the acquiring the optimized global map comprises:

determining pose drift information based on the relative pose; and

acquiring the optimized global map by determining an optimization strategy based on the pose drift information and optimizing the current global map according to the optimization strategy.

10 . The SLAM method of claim 9 , wherein the acquiring the optimized global map further comprises:

acquiring the optimized global map by adjusting a prior global map through incremental bundle adjustment when the pose drift information satisfies a preset error condition; or

acquiring the optimized global map by adjusting the prior global map through full bundle adjustment when the pose drift information does not satisfy the error condition.

11 . The SLAM method of claim 10 , wherein the acquiring the optimized global map further comprises:

acquiring a first global map by optimizing a multi-degree-of-freedom pose of a keyframe of the prior global map based on the relative pose; and

acquiring the optimized global map by optimizing a keyframe pose and map points of the first global map through whole bundle adjustment.

12 . The SLAM method of claim 1 , wherein the generating of the one or more first feature matching pairs comprises:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image; and

determining the one or more first feature matching pairs based on a comparison between the one or more first cubes and the one or more second cubes.

13 . The SLAM method of claim 1 , wherein the generating of the one or more first feature matching pairs comprises:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image;

determining first cluster centroids of the respective one or more first cubes and second cluster centroids of the respective one or more second cubes; and

determining the one or more first feature matching pairs based on a comparison between the first cluster centroids and the second cluster centroids.

14 . An electronic device comprising:

at least one processor;

a memory; and

at least one application program stored in the memory and configured to be executed by the at least one processor, the at least one application program being configured to perform simultaneous localization and mapping (SLAM) by:

acquiring a search image based on a query image,

acquiring first spatial features of the query image and second spatial features of the search image,

acquiring feature matching results by matching at least one of the first spatial features with at least one of the second spatial features, and

estimating a relative pose between the query image and the search image based on the feature matching results,

wherein the acquiring feature matching results comprises:

generating one or more first feature matching pairs by performing a coarse matching operation based on the first spatial features and the second spatial features, and

generating one or more second feature matching pairs by performing a fine matching operation based on the first spatial features and the second spatial features.

15 . The electronic device of claim 14 , wherein the at least one application program is further configured to generate the one or more first feature matching pairs by:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image; and

determining the one or more first feature matching pairs based on a comparison between the one or more first cubes and the one or more second cubes.

16 . The electronic device of claim 14 , wherein the at least one application program is further configured to generate the one or more first feature matching pairs by:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image;

determining first cluster centroids of the respective one or more first cubes and second cluster centroids of the respective one or more second cubes; and

determining the one or more first feature matching pairs based on a comparison between the first cluster centroids and the second cluster centroids.

17 . A non-transitory computer-readable storage medium having recorded thereon a program for executing simultaneous localization and mapping method comprising:

acquiring a search image based on a query image,

acquiring first spatial features of the query image and second spatial features of the search image,

acquiring feature matching results by matching at least one of the first spatial features with at least one of the second spatial features, and

estimating a relative pose between the query image and the search image based on the feature matching results,

wherein the acquiring feature matching results comprises:

generating one or more first feature matching pairs by performing a coarse matching operation based on the first spatial features and the second spatial features, and

generating one or more second feature matching pairs by performing a fine matching operation based on the first spatial features and the second spatial features.

18 . The non-transitory computer-readable storage medium of claim 17 , wherein the generating of the one or more first feature matching pairs comprises:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image; and

determining the one or more first feature matching pairs based on a comparison between the one or more first cubes and the one or more second cubes.

19 . The non-transitory computer-readable storage medium of claim 17 , wherein the generating of the one or more first feature matching pairs comprises:

determining one or more first cubes by clustering three dimensional (3D) point sets of the query image;

determining one or more second cubes by clustering the 3D point sets of the search image;

determining first cluster centroids of the respective one or more first cubes and second cluster centroids of the respective one or more second cubes; and

determining the one or more first feature matching pairs based on a comparison between the first cluster centroids and the second cluster centroids.

Assignments (1)

Priority Claims (2)

Continuity (1)

References Cited (9)