Granted Patent B2

US 12,374,079 · App. 18/021,291 · Granted Jul 29, 2025

Inventors: Royston Rodrigues (Tokyo, JP); Masahiro Tani (Tokyo, JP)

Assignee: NEC CORPORATION

G06V10/761G06V10/44G06V10/771G06V10/82G06V20/17

Abstract

Claims (57)

1. An image matching apparatus comprising:

at least one processor; and

memory storing instructions,

wherein the at least one processor is configured to execute the instructions to:

acquire a ground-view image and an aerial-view image;

generate ground-view feature maps that are feature maps derived from the ground-view image, the ground-view feature maps including: a first non-attention ground-view feature map that is generated by extracting features from the ground-view image; a first attention ground-view feature map that is generated by applying an attention map on the first non-attention ground-view feature map; a second non-attention ground-view feature map that is generated by extracting features from the first non-attention ground-view feature map; and a second attention ground-view feature map that is generated by applying an attention map on the second non-attention ground-view feature map;

generate aerial-view feature maps that are feature maps derived from the aerial-view image, the aerial-view feature maps including: a first non-attention aerial-view feature map that is generated by extracting features from the aerial-view image; a first attention aerial-view feature map that is generated by applying an attention map on the first non-attention aerial-view feature map; a second non-attention aerial-view feature map that is generated by extracting features from the first non-attention aerial-view feature map; and a second attention aerial-view feature map that is generated by applying an attention map on the second non-attention aerial-view feature map;

compute a similarity score based on the ground-view feature maps and the aerial-view feature maps, the similarity score indicating a degree of similarity between the ground-view image and the aerial-view image; and

determine whether the ground-view image matches the aerial-view image based on the similarity score.

2. The image matching apparatus according to claim 1 ,

wherein the computation of the similarity score includes to compute a degree of similarity between a concatenation of the ground-view feature maps and a concatenation of the aerial-view feature maps as the similarity score.

3. The image matching apparatus according to claim 2 ,

wherein the degree of similarity between two data is computed as a distance therebetween, a cosine similarity therebetween, or a neural network based similarity that is computed by a neural network trained to output the degree of similarity between two data.

4. The image matching apparatus according to claim 1 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a maximum value of the computed degrees of similarity as the similarity score.

5. The image matching apparatus according to claim 1 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a statistical value of some or all of the computed degrees of similarity as the similarity score.

6. A control method performed by a computer, comprising:

acquiring a ground-view image and an aerial-view image;

generating ground-view feature maps that are feature maps derived from the ground-view image, the ground-view feature maps including: a first non-attention ground-view feature map that is generated by extracting features from the ground-view image; a first attention ground-view feature map that is generated by applying an attention map on the first non-attention ground-view feature map; a second non-attention ground-view feature map that is generated by extracting features from the first non-attention ground-view feature map; and a second attention ground-view feature map that is generated by applying an attention map on the second non-attention ground-view feature map;

generating aerial-view feature maps that are feature maps derived from the aerial-view image, the aerial-view feature maps including: a first non-attention aerial-view feature map that is generated by extracting features from the aerial-view image; a first attention aerial-view feature map that is generated by applying an attention map on the first non-attention aerial-view feature map; a second non-attention aerial-view feature map that is generated by extracting features from the first non-attention aerial-view feature map; and a second attention aerial-view feature map that is generated by applying an attention map on the second non-attention aerial-view feature map;

computing a similarity score based on the ground-view feature maps and the aerial-view feature maps, the similarity score indicating a degree of similarity between the ground-view image and the aerial-view image; and

determining whether the ground-view image matches the aerial-view image based on the similarity score.

7. The control method according to claim 6 ,

wherein the computation of the similarity score includes to compute a degree of similarity between a concatenation of the ground-view feature maps and a concatenation of the aerial-view feature maps as the similarity score.

8. The control method according to claim 6 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a maximum value of the computed degrees of similarity as the similarity score.

9. The control method according to claim 6 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a statistical value of some or all of the computed degrees of similarity as the similarity score.

10. The control method according to claim 6 ,

wherein the degree of similarity between two data is computed as a distance therebetween, a cosine similarity therebetween, or a neural network based similarity that is computed by a neural network trained to output the degree of similarity between two data.

11. A non-transitory computer readable storage medium storing a program that causes a computer to execute:

acquiring a ground-view image and an aerial-view image;

generating ground-view feature maps that are feature maps derived from the ground-view image, the ground-view feature maps including: a first non-attention ground-view feature map that is generated by extracting features from the ground-view image; a first attention ground-view feature map that is generated by applying an attention map on the first non-attention ground-view feature map; a second non-attention ground-view feature map that is generated by extracting features from the first non-attention ground-view feature map; and a second attention ground-view feature map that is generated by applying an attention map on the second non-attention ground-view feature map;

generating aerial-view feature maps that are feature maps derived from the aerial-view image, the aerial-view feature maps including: a first non-attention aerial-view feature map that is generated by extracting features from the aerial-view image; a first attention aerial-view feature map that is generated by applying an attention map on the first non-attention aerial-view feature map; a second non-attention aerial-view feature map that is generated by extracting features from the first non-attention aerial-view feature map; and a second attention aerial-view feature map that is generated by applying an attention map on the second non-attention aerial-view feature map;

computing a similarity score based on the ground-view feature maps and the aerial-view feature maps, the similarity score indicating a degree of similarity between the ground-view image and the aerial-view image; and

determining whether the ground-view image matches the aerial-view image based on the similarity score.

12. The storage medium according to claim 11 ,

wherein the computation of the similarity score includes to compute a degree of similarity between a concatenation of the ground-view feature maps and a concatenation of the aerial-view feature maps as the similarity score.

13. The storage medium according to claim 11 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a maximum value of the computed degrees of similarity as the similarity score.

14. The storage medium according to claim 11 ,

wherein the computation of the similarity score includes to:

compute a degree of similarity for each one of possible pairs of the ground-view feature map and the aerial-view feature map;

use a statistical value of some or all of the computed degrees of similarity as the similarity score.

15. The storage medium according to claim 11 ,

wherein the degree of similarity between two data is computed as a distance therebetween, a cosine similarity therebetween, or a neural network based similarity that is computed by a neural network trained to output the degree of similarity between two data.

Assignments (1)

Continuity (1)

References Cited (10)