Two-View Geometry Scoring without Correspondences

1 . A computer-implemented method comprising:

receiving a pair of overlapping images of a scene;

calculating a relative pose between the pair of overlapping images;

applying a two-view geometry model to predict an error in the relative pose between the pair of overlapping images; and

providing content for display at a client device accounting for the error.

2 . The method of claim 1 , wherein the two-view geometry model uses an epipolar attention mechanism to predict the error in the relative pose between the pair of overlapping images.

3 . The method of claim 1 , further comprising:

computing feature maps for each of the pair of overlapping images; and

using a self-attention layer and a cross-attention layer to form transformed feature maps for each of the feature maps.

4 . The method of claim 3 , further comprising:

applying cross-attention along epipolar lines to embed a fundamental matrix hypothesis into corresponding transformed feature maps to form final feature maps for the pair of overlapping images.

5 . The method of claim 4 wherein applying cross-attention along the epipolar lines to embed the fundamental matrix hypothesis into corresponding transformed feature maps is performed such that a resolution of the final feature maps is less than a resolution of the transformed feature maps.

6 . The method of claim 4 , wherein applying the two-view geometry model to predict the error in the relative pose between the pair of overlapping images, comprises:

predicting an angular translation error and a rotation error associated with the fundamental matrix hypothesis using the final feature maps.

7 . The method of claim 6 , wherein the two-view geometry model does not use correspondences to predict the error in the relative pose between the pair of overlapping images.

8 . The method of claim 1 , further comprising:

identifying a number of correspondences between the pair of overlapping images; and

electing to use the two-view geometry model responsive to the number of correspondences being below a threshold.

9 . The method of claim 1 , wherein applying the two-view geometry model comprises:

generating a pool of fundamental matrix hypotheses;

reducing the pool of fundamental matrix hypotheses based on rankings of the fundamental matrix hypotheses;

applying the two-view geometry model to calculate a hypothesis error for each fundamental matrix hypothesis in the reduced pool of fundamental matrix hypotheses; and

using a fundamental matrix hypothesis having a lowest hypothesis error to determine the relative pose between the pair of overlapping images, wherein the error in the lowest hypothesis error.

10 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor of a client device, cause the client device to perform operations including:

receiving a pair of overlapping images of a scene;

calculating a relative pose between the pair of overlapping images;

applying a two-view geometry model to predict an error in the relative pose between the pair of overlapping images; and

presenting content that accounts for the error.

11 . The computer program product of claim 10 , wherein the two-view geometry model is configured to use an epipolar attention mechanism to predict the error in the relative pose between the pair of overlapping images.

12 . The computer program product of claim 11 , wherein the operations further comprise:

computing feature maps for each of the pair of overlapping images; and

using a self-attention layer and a cross-attention layer to form transformed feature maps for each of the feature maps.

13 . The computer program product of claim 11 , wherein the operations further comprise:

applying cross-attention along epipolar lines to embed a fundamental matrix hypothesis into corresponding transformed feature maps to form final feature maps for the pair of overlapping images.

14 . The computer program product of claim 13 , wherein applying the cross-attention along the epipolar lines to embed the fundamental matrix hypothesis into corresponding transformed feature maps is performed such that a resolution of the final feature maps is less than a resolution of the transformed feature maps.

15 . The computer program product of claim 13 , wherein applying the two-view geometry model to predict the error in the relative pose between the pair of overlapping images further comprises:

predicting an angular translation error and a rotation error associated with the fundamental matrix hypothesis using the final feature maps.

16 . The computer program product of claim 15 , wherein the two-view geometry model does not use correspondences to predict the error in the relative pose between the pair of overlapping images.

17 . A client device comprising:

one or more cameras configured to capture a pair of overlapping images of a scene;

a display configured to present content;

a processor; and

a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the processor, cause the processor to:

calculate a relative pose between the pair of overlapping images,

apply a two-view geometry model to predict an error in the relative pose between the pair of overlapping images, and

instruct the display to present content, wherein the content accounts for the error.

18 . The client device of claim 17 , wherein the two-view geometry model is configured to use an epipolar attention mechanism to predict the error in the relative pose between the pair of overlapping images.

19 . The client device of claim 17 , further comprising instructions that when executed cause the client device to:

compute feature maps for each of the pair of overlapping images; and

use a self-attention layer and a cross-attention layer to form transformed feature maps for each of the feature maps.

20 . The client device of claim 17 , further comprising instructions that when executed cause the client device to:

apply cross-attention along epipolar lines to embed a fundamental matrix hypothesis into corresponding transformed feature maps to form final feature maps for the pair of overlapping images; and

predict an angular translation error and a rotation error associated with the fundamental matrix hypothesis using the final feature maps.