Systems and methods for detecting and matching keypoints between different views of a scene

1. A system for detecting and matching keypoints between different views of a scene, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing:

a data encoding module including instructions that when executed by the one or more processors cause the one or more processors to:

acquire a first image of the scene and a second image of the scene, wherein the second image of the scene differs from the first image of the scene;

subdivide the first image of the scene into a first plurality of cells; and

subdivide the second image of the scene into a second plurality of cells; and

a keypoint analysis module including instructions that when executed by the one or more processors cause the one or more processors to:

process the first plurality of cells and the second plurality of cells using a neural keypoint detection network to identify, for a particular cell in the first plurality of cells, a first keypoint within the first plurality of cells and to identify, for a particular cell in the second plurality of cells, a second keypoint in the second plurality of cells, wherein at least one of the first keypoint and the second keypoint lies in a cell other than the particular cell in the first or second plurality of cells for which it was identified; and

classify the first keypoint and the second keypoint as a matching keypoint pair based, at least in part, on a comparison between a first descriptor associated with the first keypoint and a second descriptor associated with the second keypoint.

2. The system of claim 1 , wherein the data encoding module includes further instructions that when executed by the one or more processors cause the one or more processors to encode the first image of the scene and the second image of the scene into a reduced-resolution embedding space to produce a first encoded image of the scene and a second encoded image of the scene and the keypoint analysis module includes further instructions that when executed by the one or more processors cause the one or more processors to up sample the first encoded image of the scene and the second encoded image of the scene prior to regressing the first descriptor and the second descriptor.

3. The system of claim 1 , wherein the instructions in the keypoint analysis module to identify the first keypoint and the second keypoint include instructions to identify the first keypoint with respect to a center of the particular cell in the first plurality of cells and to identify the second keypoint with respect to a center of the particular cell in the second plurality of cells.

4. The system of claim 1 , wherein the instructions in the keypoint analysis module to classify the first keypoint and the second keypoint as a matching keypoint pair based, at least in part, on the comparison between the first descriptor associated with the first keypoint and the second descriptor associated with the second keypoint include instructions to compute a Euclidian distance between the first descriptor and the second descriptor.

5. The system of claim 1 , further comprising a training module including instructions that when executed by the one or more processors cause the one or more processors to train the neural keypoint detection network in accordance with a self-supervised training process in which the neural keypoint detection network processes sample image segments and corresponding transformed sample image segments under constraint of three direct supervisory signals that include a keypoint-location loss function, a keypoint-descriptor loss function, and a keypoint-quality-score loss function.

6. The system of claim 1 , wherein the system is integrated with a vehicle.

7. The system of claim 6 , wherein the vehicle is an autonomous vehicle.

8. The system of claim 1 , wherein the system is integrated with a humanoid robot.

9. The system of claim 1 , wherein the first image of the scene and the second image of the scene are in red-blue-green (RGB) format.

10. A non-transitory computer-readable medium for detecting and matching keypoints between different views of a scene and storing instructions that when executed by one or more processors cause the one or more processors to:

acquire a first image of the scene and a second image of the scene, wherein the second image of the scene differs from the first image of the scene;

subdivide the first image of the scene into a first plurality of cells;

subdivide the second image of the scene into a second plurality of cells;

process the first plurality of cells and the second plurality of cells using a neural keypoint detection network to identify, for a particular cell in the first plurality of cells, a first keypoint within the first plurality of cells and to identify, for a particular cell in the second plurality of cells, a second keypoint in the second plurality of cells, wherein at least one of the first keypoint and the second keypoint lies in a cell other than the particular cell in the first or second plurality of cells for which it was identified; and

classify the first keypoint and the second keypoint as a matching keypoint pair based, at least in part, on a comparison between a first descriptor associated with the first keypoint and a second descriptor associated with the second keypoint.

11. The non-transitory computer-readable medium of claim 10 , further comprising instructions that when executed by one or more processors cause the one or more processors to:

encode the first image of the scene and the second image of the scene into a reduced-resolution embedding space to produce a first encoded image of the scene and a second encoded image of the scene; and

upsample the first encoded image of the scene and the second encoded image of the scene prior to regressing the first descriptor and the second descriptor.

12. The non-transitory computer-readable medium of claim 10 , wherein the instructions include instructions to identify the first keypoint with respect to a center of the particular cell in the first plurality of cells and to identify the second keypoint with respect to a center of the particular cell in the second plurality of cells.

13. The non-transitory computer-readable medium of claim 10 , wherein the instructions to classify the first keypoint and the second keypoint as a matching keypoint pair based, at least in part, on the comparison between the first descriptor associated with the first keypoint and the second descriptor associated with the second keypoint include instructions to compute a Euclidian distance between the first descriptor and the second descriptor.

14. The non-transitory computer-readable medium of claim 10 , further comprising instructions that when executed by one or more processors cause the one or more processors to train the neural keypoint detection network in accordance with a self-supervised training process in which the neural keypoint detection network processes sample image segments and corresponding transformed sample image segments under constraint of three direct supervisory signals that include a keypoint-location loss function, a keypoint-descriptor loss function, and a keypoint-quality-score loss function.

15. A method of detecting and matching keypoints between different views of a scene, the method comprising:

acquiring a first image of the scene and a second image of the scene, wherein the second image of the scene differs from the first image of the scene;

subdividing the first image of the scene into a first plurality of cells;

subdividing the second image of the scene into a second plurality of cells;

processing the first plurality of cells and the second plurality of cells using a neural keypoint detection network to identify, for a particular cell in the first plurality of cells, a first keypoint within the first plurality of cells and to identify, for a particular cell in the second plurality of cells, a second keypoint in the second plurality of cells, wherein at least one of the first keypoint and the second keypoint lies in a cell other than the particular cell in the first or second plurality of cells for which it was identified; and

classifying the first keypoint and the second keypoint as a matching keypoint pair based, at least in part, on a comparison between a first descriptor associated with the first keypoint and a second descriptor associated with the second keypoint.

16. The method of claim 15 , further comprising:

encoding the first image of the scene and the second image of the scene into a reduced-resolution embedding space to produce a first encoded image of the scene and a second encoded image of the scene; and

upsampling the first encoded image of the scene and the second encoded image of the scene prior to regressing the first descriptor and the second descriptor.

17. The method of claim 15 , wherein the first keypoint is identified with respect to a center of the particular cell in the first plurality of cells and the second keypoint is identified with respect to a center of the particular cell in the second plurality of cells.

18. The method of claim 15 , wherein the comparison includes computation of a Euclidian distance between the first descriptor and the second descriptor.

19. The method of claim 15 , further comprising training the neural keypoint detection network in accordance with a self-supervised training process in which the neural keypoint detection network processes sample image segments and corresponding transformed sample image segments under constraint of three direct supervisory signals that include a keypoint-location loss function, a keypoint-descriptor loss function, and a keypoint-quality-score loss function.

20. The method of claim 15 , wherein the first image of the scene and the second image of the scene are in red-blue-green (RGB) format.