IP Library Granted Patent US 11,010,630
Granted Patent B2
US 11,010,630 · App. 15/965,039 · Granted May 18, 2021

Systems and methods for detecting landmark pairs in images

Inventors: Deshan Yang (St. Louis, MO); Ye Duan (Columbia, MO)
Assignees: Washington University; The Curators of The University of Missouri
G06K9/4609G06K9/40G06K9/44G06K9/4671G06K9/6212G06T7/0016G06T7/246G06T7/33G06T7/337G06K2209/05G06T2207/10081G06T2207/10088G06T2207/20016G06T2207/20028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,010,630
App. No.
15/965,039
Granted
May 18, 2021
Kind
B2
Abstract

A computer-implemented method for detecting landmark pairs in a pair of images is provided. The method includes receiving a pair of images, sampling the pair of images to generate reduced-resolution pairs of images, identifying features in the reduced-resolution pairs of images, matching the features in the image pairs, using the matched features in an increased resolution pair of images as guides for feature matching, and through iteratively guiding feature matching, identifying landmarks in the full-resolution pair of images.

Claims (38)

1. A computer-implemented method for detecting landmark pairs in a pair of full-resolution images, the method implemented using at least one processor in communication with at least one memory, the method comprising:

down-sampling, using the at least one processor, the pair of full-resolution images to generate a reduced-resolution pair of images, the reduced-resolution pair of images comprising an image resolution less than a corresponding resolution of the pair of full-resolution images;

detecting, using the at least one processor, a plurality of features in the reduced-resolution pair of images;

determining, using the at least one processor, in the reduced-resolution pair of images:

a plurality of matched feature pairs in a first iteration comprising unguided feature matching; and

at least one additional plurality of matched feature pairs in at least one additional iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and at least one additional plurality of matched feature pairs from previous iterations; and

determining, using the at least one processor, in the full-resolution pair of images, a plurality of landmark pairs in at least one iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and the at least one additional plurality of matched feature pairs.

2. The method of claim 1 , wherein the first iteration to determine the plurality of matched feature pairs uses a first matching threshold and each additional iteration of the at least one additional iteration to determine the at least one additional plurality of matched feature pairs uses an additional matching threshold, wherein the first matching threshold is higher than the additional matching threshold.

3. The method of claim 1 , wherein each iteration of the at least one additional iteration to determine the plurality of matched feature pairs uses a matching threshold that is lower than a prior matching threshold used for a prior additional iteration of the at least one additional iteration to determine the at least one additional plurality of matched feature pairs.

4. The method of claim 1 further comprising preprocessing the pair of full-resolution images, the preprocessing comprising creating a skin mask, removing voxels outside the skin mask, and applying a 3D bilateral filter to remove noise from the pair of full-resolution images.

5. The method of claim 1 further comprising down-sampling the reduced-resolution pair of images to generate an additional reduced-resolution pair of images, the additional reduced-resolution pair of images comprising an image resolution less than a corresponding resolution of the reduced-resolution pair of images.

6. The method of claim 5 , wherein down-sampling is performed using a sampling filter.

7. The method of claim 1 , wherein identifying a plurality of features in the reduced-resolution pair of images comprises at least one of using a 3D SIFT algorithm to detect SIFT features, using a 3D Harris-Laplacian algorithm to detect corner features, and computing SIFT descriptors for the SIFT features and corner features.

8. The method of claim 1 , wherein determining a plurality of landmark pairs in the pair of full-resolution images further comprises inverse-consistent matching of the full-resolution pair of images.

9. The method of claim 1 further comprising generating a SIFT descriptor for each feature of the plurality of features.

10. The method of claim 9 further comprising extracting and processing voxel information around a feature point to generate the SIFT descriptor.

11. A feature pair detection (FPD) computing device for detecting landmark pairs in a pair of full-resolution images, the FPD computing device including at least one processor in communication with at least one memory device, the at least one processor programmed to:

down-sample the pair of full-resolution images to generate a reduced-resolution pair of images, the reduced-resolution pair of images comprising an image resolution less than a corresponding resolution of the pair of full-resolution images;

detect a plurality of features in the reduced-resolution pair of images;

determine, in the reduced-resolution pair of images:

a plurality of matched feature pairs in a first iteration comprising unguided feature matching; and

at least one additional plurality of matched feature pairs in at least one additional iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and at least one additional plurality of matched feature pairs from previous iterations; and

determine, in the full-resolution pair of images, a plurality of landmark pairs in at least one iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and the at least one additional plurality of matched feature pairs.

12. The FPD computing device of claim 11 , wherein the first iteration to determine the plurality of matched feature pairs uses a first matching threshold and each additional iteration of the at least one additional iteration to determine the at least one additional plurality of matched feature pairs uses an additional matching threshold, wherein the first matching threshold is higher than the additional matching threshold.

13. The FPD computing device of claim 11 , wherein each additional iteration of the at least one additional iteration to determine the plurality of matched feature pairs uses a matching threshold that is lower than a prior matching threshold used for a prior additional iteration of the at least one additional iteration to determine the at least one additional plurality of matched feature pairs.

14. The FPD computing device of claim 11 , wherein the at least one processor is further configured to preprocess the pair of full-resolution images by creating a skin mask, removing voxels outside the skin mask, and applying a 3D bilateral filter to remove noise from the pair of full-resolution images.

15. The FPD computing device of claim 11 , wherein the at least one processor is further configured to sample the reduced-resolution pair of images to generate an additional reduced-resolution pair of images, the additional reduced-resolution pair of images having a resolution of half the resolution of the reduced-resolution pair of images.

16. The FPD computing device of claim 15 , wherein the at least one processor is further configured to down-sample by applying a sampling filter.

17. The FPD computing device of claim 11 , wherein the at least one processor is further configured to identify the plurality of features in the reduced-resolution pair of images using at least one of detecting SIFT features by applying a 3D SIFT algorithm, detecting corner features using a 3D Harris-Laplacian algorithm, and computing SIFT descriptors for the SIFT features and the corner features.

18. At least one non-transitory computer-readable storage medium having computer-executable instructions for implementing a feature pair detection (FPD) program embodied thereon, wherein when executed by a feature pair detection computing device including at least one processor in communication with at least one memory, the computer-executable instructions cause the at least one processor to:

down-sample a pair of full-resolution images to generate a reduced-resolution pair of images, the reduced-resolution pair of images comprising an image resolution less than a corresponding resolution of the pair of full-resolution images;

detect a plurality of features in the reduced-resolution pair of images;

determine, in the reduced-resolution pair of images:

a plurality of matched feature pairs in a first iteration comprising unguided feature matching; and

at least one additional plurality of matched feature pairs in at least one additional iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and at least one additional plurality of matched feature pairs from previous iterations; and

determine, in the full-resolution pair of images, a plurality of landmark pairs in at least one iteration comprising guided feature matching with a plurality of guide pairs selected from the plurality of matched feature pairs and the at least one additional plurality of matched feature pairs.

19. The at least one non-transitory computer-readable storage medium of claim 18 , wherein the first iteration to determine the plurality of matched feature pairs uses a first matching threshold and each additional iteration of the at least one additional iteration to determine the at least one additional plurality of matched feature pairs uses an additional matching threshold, wherein the first matching threshold is higher than the additional matching threshold.

20. The at least one non-transitory computer-readable storage medium of claim 18 , wherein each iteration of the at least one iterations to determine the plurality of matched feature pairs uses a matching threshold that is lower than a prior matching threshold used for a prior iteration of the at least one iterations to determine the at least one additional plurality of matched feature pairs.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 25, 2018
From: YANG, DESHAN
To: WASHINGTON UNIVERSITY
Reel/Frame 045900/0441 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 25, 2018
From: DUAN, YE
To: THE CURATORS OF THE UNIVERSITY OF MISSOURI
Reel/Frame 045900/0464 →
Continuity (2)
Provisional Application 62490728 · Apr 27, 2017
Related Publication 20180314906A1 · Nov 1, 2018