Method and image processing arrangement for estimating a likely pose in respect of a spatial region

1 . A method for estimating a likely pose of an image processing arrangement in respect of a spatial region, wherein the image processing arrangement is coupled to an imaging capturing arrangement that is configured to capture one or more 2-D images of a scene within the spatial region, wherein the image processing arrangement is provided with spatial coordinates of a plurality of 3-D point locations within the spatial region, wherein the method comprises:

(i) identifying key feature points present in the one or more 2-D images;

(ii) identifying a correspondence of one or more clusters of the plurality of 3-D point locations to key feature points present in the one or more 2-D images, wherein the one or more clusters are a variable size set of correspondences between the plurality of 3-D point locations and key feature points present in the one or more 2-D images; and

(iii) from the correspondence, estimating the likely pose of the image processing arrangement relative to the spatial region by using a multi-match likelihood function that finds for each 3-D point location a closest 2-D feature point among k possible 2-D key feature points, wherein k is an integer greater than one, and

wherein the method further comprises implementing the multi-match likelihood function as follows:

L

⁡

(

Θ

)

=

Σ

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein

Q i =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q i : =point coordinates of the 3-D feature point projected Q i into a target image;

m i :=an i-th 2-D feature point in the target image potentially corresponding to the 3-D feature point Q i ; and

ε:=a constant of uniform outlier distance distribution, and

wherein pose parameters Φ are chosen such that a multi-match likelihood L(Φ) is maximized to determine the likely pose.

2 . The method of claim 1 , further comprising determining the spatial coordinates of the plurality of 3-D point locations within the spatial region from a plurality of images captured from mutually different angular perspectives.

3 . The method of claim 1 , further comprising implementing the multi-match likelihood function as a sampling-based framework to search for a best pose for the likely pose by computing a maximum value for the multi-match likelihood function.

4 . The method of claim 1 , further comprising using an optimization process to adaptively select a closest 2-D key feature point among k best matches, to capture inliers within the 2-D image.

5 . The method of claim 1 , wherein the multi-match likelihood function is implemented as a single-match robust likelihood function as follows:

(

Θ

)

=

∑

i

N

⁢

log

⁡

(

exp

⁡

(

min

j

=

1

k

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

-

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein an index k is typically a small number, optionally in a range of 0 to 10;

Q i =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q i : =point coordinates of the 3-D feature point projected Q i into a target image;

m ij :=an ij-th 2-D feature point in the target image potentially corresponding to the 3-D feature point Q i ; and

ε:=a constant of uniform outlier distance distribution,

wherein, for each given pose parameters P (Φ), the method includes firstly finding a spatially closest point m ij , j=l, . . . ,k, and then computing a multi-match likelihood L(Φ) based on the closest point m ij .

6 . The method of claim 1 , wherein the multi-match likelihood function is implemented as a multi-match robust likelihood function as follows:

For

⁢

all

⁢

⁢

i

:

m

i

=

min

j

=

1

k

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

L

⁡

(

Θ

)

=

Σ

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein

Q i : =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q=point coordinates of the 3D feature point Q i projected into a target image to obtain q i ;

m ij : =ij-th 2D feature points in the target image potentially corresponding to 3D feature point Q i ;

ε: =a constant of uniform outlier distance distribution,

D p , D m : =descriptor vectors of q and m, respectively; and

ƒ(Φ, Q i ,M ij , D q , D m ij )=a function to determine a distance depending on both the descriptor similarity as well as spatial distance between the projection q of the 3D point Q into the image and its potential correspondence m,

wherein, for each given pose parameter P(Φ), the method includes firstly finding a closest point m ij among k m ij ,j=1, . . . ,k, w.r.t. f(), and then computing a multi-match likelihood L(Φ) based on the closest point m ij .

7 . The method of claim 1 , wherein the multi-match likelihood function is implemented as a multi-match robust likelihood function as follows:

For

⁢

all

⁢

⁢

i

:

m

i

=

min

j

=

1

k

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

L

⁡

(

Θ

)

=

Σ

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein a function f() is defined by

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

=

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

+

λ

*

❘

"\[LeftBracketingBar]"

D

q

-

D

m

ij

❘

"\[RightBracketingBar]"

2

⁢

or

f

⁡

(

Θ

,

Q

i

,

m

j

,

D

q

,

D

m

ij

)

=

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

+

λ

*

(

1

-

D

q

⁢

D

m

ij

❘

"\[LeftBracketingBar]"

D

q

❘

"\[RightBracketingBar]"

⁢

❘

"\[LeftBracketingBar]"

D

m

ij

❘

"\[RightBracketingBar]"

)

wherein a position of q in a given image depends both on camera pose parameters Φ and their 3D position Q;

q

i

=

P

⁡

(

Θ

)

⁢

Q

i

wherein, for a given feature point q, there are k potential correspondences, and

m ij : =i, j-th 2D feature points in a target image potentially corresponding to a 3D feature point Q i ;

ε: =a constant of uniform outlier distance distribution; and

D q , D m : =descriptor vectors of q and m, respectively.

8 . An image processing arrangement for estimating a likely pose in respect of a spatial region, wherein the image processing arrangement is coupled to an imaging capturing arrangement that is configured to capture one or more 2-D images of a scene within the spatial region, wherein the image processing arrangement is provided with spatial coordinates of a plurality of 3-D point locations within the spatial region, wherein the image processing arrangement is configured to:

(i) identify key feature points present in the one or more 2-D images;

(ii) identify a correspondence of one or more clusters of the plurality of 3-D point locations to key feature points present in the one or more 2-D images, wherein the one or more clusters are a variable size set of correspondences between the plurality of 3D point locations and key features points present in the one or more 2D images; and

(iii) based on the correspondence, estimate the likely pose of the image processing arrangement relative to the spatial region by using a multi-match likelihood function that finds for each 3-D point location a closest 2-D feature point among k possible 2-D key feature points, wherein k is an integer greater than one,

wherein the image processing arrangement is further configured to implement the multi-match likelihood function as follows:

(

Θ

)

=

∑

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein

Q i =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q i : =point coordinates of the 3-D feature point projected Q i into a target image;

m ij : an i-th 2-D feature point in a target image potentially corresponding to a 3D feature point Q i ;

ε: =a constant of uniform outlier distance distribution, and

wherein pose parameters Φ are chosen such that a multi-match likelihood L(Φ) is maximized to determine the likely pose.

9 . The image processing arrangement of claim 8 , wherein the image processing arrangement is further configured to determine the spatial coordinates of the plurality of 3-D point locations within the spatial region from a plurality of images captured from mutually different angular perspectives.

10 . A non-transitory computer-readable medium containing computer-executable instructions, wherein the computer-executable instructions cause a computer to execute a method for estimating a likely pose of an image processing arrangement in respect of a spatial region, wherein the image processing arrangement is coupled to an imaging capturing arrangement that is configured to capture one or more 2-D images of a scene within the spatial region, wherein the image processing arrangement is provided with spatial coordinates of a plurality of 3-D point locations within the spatial region, wherein the method comprises:

(i) identifying key feature points present in the one or more 2-D images;

(ii) identifying a correspondence of one or more clusters of the plurality of 3-D point locations to key feature points present in the one or more 2-D images, wherein the one or more clusters are a variable size set of correspondences between the plurality of 3-D point locations and key feature points in the one or more 2-D images; and

(iii) from the correspondence, estimating the likely pose of the image processing arrangement relative to the spatial region by using a multi-match likelihood function that finds for each 3-D point location a closest 2-D feature point among k possible 2-D key feature points, wherein k is an integer greater than one, and

ps wherein the method further comprises implementing the multi-match likelihood function as follows:

L

⁡

(

Θ

)

=

Σ

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein

Q i =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q i : =point coordinates of the 3-D feature point projected Q i into a target image;

m i : =an i-th 2-D feature point in the target image potentially corresponding to the 3-D feature point Q i ; and

ε: =a constant of uniform outlier distance distribution, and

wherein pose parameters Φ are chosen such that a multi-match likelihood L(Φ) is maximized to determine the likely pose.

11 . The non-transitory computer-readable medium of claim 10 , further comprising determining the spatial coordinates of the plurality of 3-D point locations within the spatial region from a plurality of images captured from mutually different angular perspectives.

12 . The non-transitory computer-readable medium of claim 10 , further comprising implementing the multi-match likelihood function as a sampling-based framework to search for a best pose for the likely pose by computing a maximum value for the multi-match likelihood function.

13 . The non-transitory computer-readable medium of claim 10 , further comprising using an optimization process to adaptively select a closest 2-D key feature point among k best matches, to capture inliers within the 2-D image.

14 . The non-transitory computer-readable medium of claim 10 , wherein the multi-match likelihood function is implemented as a single-match robust likelihood function as follows:

(

Θ

)

=

∑

i

N

⁢

log

⁡

(

exp

⁡

(

min

j

=

1

k

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

-

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein an index k is typically a small number, optionally in a range of 0 to 10;

Q i : =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ;

P(Φ) Q i : =point coordinates of the 3D feature point Q i projected into a target image;

m ij : =ij-th 2D feature points in the target image potentially corresponding to 3D feature point Q i ; and

ε: =a constant of uniform outlier distance distribution,

wherein, for each given pose parameter P(Φ), the method includes firstly finding a spatially closest point m ij among k m ij , j=1, . . . ,k, and then computing a multi-match likelihood L(Φ) based on the closest point m ij .

15 . The non-transitory computer-readable medium of claim 10 , wherein the multi-match likelihood function is implemented as a multi-match robust likelihood function as follows:

For

⁢

all

⁢

i

:

m

i

=

min

j

=

1

k

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

(

Θ

)

=

∑

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein

Q i : =an i-th 3D feature point;

P(Φ): =an inverse camera pose parameterized by Φ:

q i =P(Φ) Q i : point coordinates of the 3D feature point projected Q i into a target image;

m ij : ij-th 2-D feature point in a target image potentially corresponding to a 3D feature point Q i ;

ε: =a constant of uniform outlier distance distribution;

D q , D m : =descriptor vectors of q and m, respectively; and

ƒ(Θ, Q i ,m ij , D q , D m ij )=a function to determine a distance depending on both the descriptor similarity as well as spatial distance between the projection q of the 3D point Q into the image and its potential correspondence m,

wherein for each given pose parameter P(Φ), the method includes firstly finding a closest point m ij among k m ij , j=1, . . . ,k, w.r.t. f( ) and then computing a multi-match likelihood L(Φ) based on the closest point m ij .

16 . The non-transitory computer-readable medium of claim 10 , wherein the multi-match likelihood function is implemented as a multi-match robust likelihood function as follows:

For

⁢

all

⁢

i

:

m

i

=

min

j

=

1

k

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

(

Θ

)

=

∑

i

N

⁢

log

⁡

(

exp

⁡

(

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

i

❘

"\[RightBracketingBar]"

2

2

⁢

σ

2

)

+

ε

)

wherein a function f( ) is defined by

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

=

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

+

λ

*

❘

"\[LeftBracketingBar]"

D

q

-

D

m

ij

❘

"\[RightBracketingBar]"

2

or

f

⁡

(

Θ

,

Q

i

,

m

ij

,

D

q

,

D

m

ij

)

=

❘

"\[LeftBracketingBar]"

P

⁡

(

Θ

)

⁢

Q

i

-

m

ij

❘

"\[RightBracketingBar]"

2

+

λ

*

(

1

-

D

q

⁢

D

m

ij

❘

"\[LeftBracketingBar]"

D

q

❘

"\[RightBracketingBar]"

⁢

❘

"\[LeftBracketingBar]"

D

m

ij

❘

"\[RightBracketingBar]"

)

wherein a position of q in a given image depends both on camera pose parameters Φ and their 3D position Q:

q

t

=

P

⁡

(

Θ

)

⁢

Q

t

wherein, for a given feature point q, there are k potential correspondences, and

m ij : =i, j-th 2D feature points in a target image potentially corresponding to a 3D feature point Q i ;

ε: =a constant of uniform outlier distance distribution; and

D q , D m :=descriptor vectors of q and m, respectively.