Granted Patent B2

US 12,536,680 · App. 18/273,612 · Granted Jan 27, 2026

Inventors: Royston Rodrigues (Tokyo, JP); Masahiro Tani (Tokyo, JP)

Assignee: NEC CORPORATION

G06T7/35G06T2207/10032G06T2207/20081

Abstract

Claims (62)

1 . A training apparatus comprising:

at least one memory that is configured to store instructions; and

at least one processor that is configured to execute the instructions to:

acquire a ground-view image whose field of view is less than 360 degree, an aerial-view image, and alignment information; and

train a discriminator that determines whether or not the ground-view image matches the aerial-view image, the discriminator extracting a plurality of partial aerial regions from the aerial-view image and determining that the ground-view image matches the aerial-view image when there is the partial aerial region that matches the ground-view image,

wherein the training of the discriminator includes:

extracting a correctly-aligned partial aerial region and a mis-aligned partial aerial region from the aerial-view image, the correctly-aligned partial aerial region being the partial aerial region that is indicated by the alignment information and that corresponds to the ground-view image, the mis-aligned partial aerial region being the partial aerial region partially overlapping the correctly-aligned partial aerial region;

training the discriminator using a set of the ground-view image and the correctly-aligned partial aerial region and a set of the ground-view image and the mis-aligned partial aerial region, both the correctly-aligned partial aerial region and the mis-aligned partial aerial region being used as positive examples of the partial aerial region.

2 . The training apparatus according to claim 1 , wherein

the alignment information indicates a first radial line and a second radial line,

the extraction of the correctly-aligned partial aerial region includes extracting a region between the first line and the second line in the aerial-view image as the correctly-aligned partial aerial region, and

the extraction of the mis-aligned partial aerial region includes:

rotating the first radial line and the second radial line; and

extracting a region between the rotated first line and the rotated second line in the aerial-view image as the mis-aligned partial aerial region.

3 . The training apparatus according to claim 2 , wherein

the extraction of the mis-aligned partial aerial region includes sampling a value from a probability distribution, and

the first radial line and the second radial line are rotated by an angle that is determined by the sampled value.

4 . The training apparatus according to claim 3 , wherein

the probability distribution is a zero mean Gaussian distribution.

5 . The training apparatus according to claim 1 , wherein

the at least one processor is further configured to extract a circular region from the aerial-view image, and

the correctly-aligned partial aerial region and the mis-aligned partial aerial region are extracted from the circular region.

6 . A control method performed by a computer, comprising:

acquiring a ground-view image whose field of view is less than 360 degree, an aerial-view image, and alignment information; and

training a discriminator that determines whether or not the ground-view image matches the aerial-view image, the discriminator extracting a plurality of partial aerial regions from the aerial-view image and determining that the ground-view image matches the aerial-view image when there is the partial aerial region that matches the ground-view image,

wherein the training of the discriminator includes:

extracting a correctly-aligned partial aerial region and a mis-aligned partial aerial region from the aerial-view image, the correctly-aligned partial aerial region being the partial aerial region that is indicated by the alignment information and that corresponds to the ground-view image, the mis-aligned partial aerial region being the partial aerial region partially overlapping the correctly-aligned partial aerial region;

training the discriminator using a set of the ground-view image and the correctly-aligned partial aerial region and a set of the ground-view image and the mis-aligned partial aerial region, both the correctly-aligned partial aerial region and the mis-aligned partial aerial region being used as positive examples of the partial aerial region.

7 . The control method according to claim 6 , wherein

the alignment information indicates a first radial line and a second radial line,

the extraction of the correctly-aligned partial aerial region includes extracting a region between the first line and the second line in the aerial-view image as the correctly-aligned partial aerial region, and

the extraction of the mis-aligned partial aerial region includes:

rotating the first radial line and the second radial line; and

extracting a region between the rotated first line and the rotated second line in the aerial-view image as the mis-aligned partial aerial region.

8 . The control method according to claim 7 , wherein

the extraction of the mis-aligned partial aerial region includes sampling a value from a probability distribution, and

the first radial line and the second radial line are rotated by an angle that is determined by the sampled value.

9 . The control method according to claim 8 , wherein

the probability distribution is a zero mean Gaussian distribution.

10 . The control method according to claim 6 , further comprising:

extracting a circular region from the aerial-view image,

wherein the correctly-aligned partial aerial region and the mis-aligned partial aerial region are extracted from the circular region.

11 . A non-transitory computer-readable storage medium storing a program that causes a computer to execute:

acquiring a ground-view image whose field of view is less than 360 degree, an aerial-view image, and alignment information; and

training a discriminator that determines whether or not the ground-view image matches the aerial-view image, the discriminator extracting a plurality of partial aerial regions from the aerial-view image and determining that the ground-view image matches the aerial-view image when there is the partial aerial region that matches the ground-view image,

wherein the training of the discriminator includes:

extracting a correctly-aligned partial aerial region and a mis-aligned partial aerial region from the aerial-view image, the correctly-aligned partial aerial region being the partial aerial region that is indicated by the alignment information and that corresponds to the ground-view image, the mis-aligned partial aerial region being the partial aerial region partially overlapping the correctly-aligned partial aerial region;

training the discriminator using a set of the ground-view image and the correctly-aligned partial aerial region and a set of the ground-view image and the mis-aligned partial aerial region, both the correctly-aligned partial aerial region and the mis-aligned partial aerial region being used as positive examples of the partial aerial region.

12 . The storage medium according to claim 11 , wherein

the alignment information indicates a first radial line and a second radial line,

the extraction of the correctly-aligned partial aerial region includes extracting a region between the first line and the second line in the aerial-view image as the correctly-aligned partial aerial region, and

the extraction of the mis-aligned partial aerial region includes:

rotating the first radial line and the second radial line; and

extracting a region between the rotated first line and the rotated second line in the aerial-view image as the mis-aligned partial aerial region.

13 . The storage medium according to claim 12 , wherein

the extraction of the mis-aligned partial aerial region includes sampling a value from a probability distribution, and

the first radial line and the second radial line are rotated by an angle that is determined by the sampled value.

14 . The storage medium according to claim 13 , wherein

the probability distribution is a zero mean Gaussian distribution.

15 . The storage medium according to claim 11 , wherein

the program further causes the compute to extract a circular region from the aerial-view image, and

the correctly-aligned partial aerial region and the mis-aligned partial aerial region are extracted from the circular region.

Assignments (1)

Continuity (1)

References Cited (20)