IP Library › Granted Patent US 12,664,669
Granted Patent B2
US 12,664,669 · App. 18/124,548 · Granted Jun 23, 2026

Computer vision systems and methods for unsupervised learning for progressively aligning noisy contours

Inventors: Venkata Subbarao Veeravarasapu (Munich, DE); Abhishek Goel (Rohini, IN); Deepak Mittal (Haryana, IN); Maneesh Kumar Singh (Princeton, NJ)
Assignee: Insurance Services Office, Inc.
G06T7/337G06N3/088G06T3/18G06T2207/20016G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,664,669
App. No.
18/124,548
Filed
Mar 21, 2023
Granted
Jun 23, 2026
Kind
B2
Art Unit
2669
USPC
382/100
Abstract

Computer vision systems and methods for noisy contour alignment are provided. The system generates a loss function and trains a convolutional neural network with the loss function and a plurality of images of a dataset to learn to align contours with progressively increasing complex forward and backward transforms over increasing scales. The system can align an attribute of an image of the dataset by the trained neural network.

Claims (33)

1 . A computer vision system for performing noisy contour alignment comprising:

a memory; and

a processor in communication with the memory, the processor:

generating a loss function;

training a neural network to learn to align contours using the loss function, a cascade of warp predictors to predict transformations to align source features to target features, and a plurality of images; and

aligning an attribute of an image using the trained neural network;

wherein the cascade of warp predictors predicts transformations to align the source features to the target features at different scales; and

wherein each warp predictor concatenates the source features warped by an upsampled warp field, the target features, and the upsampled warp field.

2 . The system of claim 1 , wherein the processor determines an upper bound of the loss function.

3 . The system of claim 1 , wherein the loss function is a local shape-dependent Chamfer upper bound loss function that measures proximity and local shape similarity.

4 . The system of claim 1 , wherein the neural network is a convolutional neural network.

5 . The system of claim 1 , wherein the loss function is generated from a Modified National Institute of Standards and Technology (MNIST) digit image dataset or a geo-parcel to aerial image alignment dataset.

6 . The system of claim 1 , where the processor aligns a noisy digit contour present in the image or a parcel present in the image.

7 . A method for performing noisy contour alignment by a computer vision system, comprising the steps of:

generating a loss function;

training a neural network to learn to align contours using the loss function, a cascade of warp predictors to predict transformations to align source features to target features, and a plurality of images; and

aligning an attribute of an image using the trained neural network;

wherein the cascade of warp predictors predicts transformations to align the source features to the target features at different scales; and

wherein each warp predictor concatenates the source features warped by an upsampled warp field, the target features, and the upsampled warp field.

8 . The method of claim 7 , further comprising the step of determining an upper bound of the loss function.

9 . The method of claim 7 , wherein the loss function is a local shape-dependent Chamfer upper bound loss function that measures proximity and local shape similarity.

10 . The method of claim 8 , wherein the neural network is a convolutional neural network.

11 . The method of claim 7 , wherein the loss function is generated from a Modified National Institute of Standards and Technology (MNIST) digit image dataset or a geo-parcel to aerial image alignment dataset.

12 . The method of claim 7 , further comprising the step of aligning the attribute of the image by the trained neural network by aligning a noisy digit contour present in the image or a parcel present in the image.

13 . A non-transitory computer readable medium having instructions stored thereon for performing noisy contour alignment by a computer vision system which, when executed by a processor, causes the processor to carry out the steps of:

generating a loss function;

training a neural network to learn to align contours using the loss function, a cascade of warp predictors to predict transformations to align source features to target features, and a plurality of images; and

aligning an attribute of an image using the trained neural network;

wherein the cascade of warp predictors predicts transformations to align the source features to the target features at different scales; and

wherein each warp predictor concatenates the source features warped by an upsampled warp field, the target features, and the upsampled warp field.

14 . The non-transitory computer readable medium of claim 13 , the processor further carrying out the step of determining an upper bound of the loss function.

15 . The non-transitory computer readable medium of claim 13 , wherein the loss function is a local shape-dependent Chamfer upper bound loss function that measures proximity and local shape similarity.

16 . The non-transitory computer readable medium of claim 13 , the processor further carrying out the step of aligning the attribute of the image by the trained neural network by aligning a noisy digit contour present in the image or a parcel present in the image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 21, 2023
From: VEERAVARASAPU, VENKATA SUBBARAO; GOEL, ABHISHEK; MITTAL, DEEPAK; SINGH, MANEESH KUMAR
To: INSURANCE SERVICES OFFICE, INC.
Reel/Frame 063052/0688 →
Continuity (3)
Continuation 17104698 · Nov 25, 2020
Provisional Application 62939770 · Nov 25, 2019
Related Publication 20240005533A1 · Jan 4, 2024
References Cited (51)
US 11610322B2 · Veeravasarapu et al. · 2023 [cited by applicant]
US 20210049733A1 · Kang · 2021 [cited by examiner]
US 20210142539A1 · Ayush et al. · 2021 [cited by applicant]
US 20210158549A1 · Veeravasarapu et al. · 2021 [cited by applicant]
Hanocka, Rana, et al. “ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning.” arXiv preprint arXiv:1804.08497v2 (2018). (Year: 2018). [cited by examiner]
Gavrila, Dariu M. “Multi-feature hierarchical template matching using distance transforms.” Proceedings. Fourteenth international conference on pattern recognition (Cat. No. 98EX170). vol. 1. IEEE, 1998. (Year: 1998). [cited by examiner]
Cohen, Gregory, et al. “EMNIST: Extending MNIST to handwritten letters.” 2017 international joint conference on neural networks (IJCNN). IEEE, 2017. (Year: 2017). [cited by examiner]
Rocco, et al., “Convolutional Neural Network Architecture for Geometric Matching.” IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 41, No. 11, Nov. 2019 (15 pages). [cited by applicant]
Veeravasarapu, et al., “ProAlignNet: Unsupervised Learning for Progressively Aligning Noisy Contours.” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (9 pages). [cited by applicant]
Hanocka, et al. “ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning.” arXiv preprint arXiv:1804.08497v2 (2018) (14 pages). [cited by applicant]
Sun, et al. “PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (10 pages). [cited by applicant]
Wang, et al. “Occlusion Aware Unsupervised Learning of Optical Flow,” 2018 I EEE/CVF Conference on Computer Vision and Pattern Recognition (10 pages). [cited by applicant]
Notice of Allowanced dated Aug. 24, 2022, issued in connection with U.S. Appl. No. 17/104,698 (13 pages). [cited by applicant]
Examiner-Initiated Interview Summary dated Aug. 19, 2022, issued in connection with U.S. Appl. No. 17/104,698 (1 page). [cited by applicant]
Acuna, et al., “Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotations,” arXiv:1904.07934v2, Jun. 9, 2019 (9 pages). [cited by applicant]
Alexiadis, et al., “Evaluating a Dancer's Performance Using Kinect-Based Skeleton Tracking,” in Proceedings of the 19th ACM international conference on Multimedia (2011) (4 pages). [cited by applicant]
Borgefors, “Distance Transformations in Arbitrary Dimensions,” Computer Vision, Graphics, and Image Processing, 27 (1984) (25 pages). [cited by applicant]
Brox, et al., “High Accuracy Optical Flow Estimation Based on a Theory for Warping,” in European Conference on Computer Vision, Springer, 2004 (13 pages). [cited by applicant]
Butt, et al., “Optimum Design of Chamfer Distance Transforms,” IEEE Transactions on Image Processing, vol. 7, No. 10, Oct. 1998 (8 pages). [cited by applicant]
Cordts, et al., “The Cityscapes Dataset for Semantic Urban Scene Understanding,” arXiv:1604.01685v2, Apr. 7, 2016 (29 pages). [cited by applicant]
Bob D. de Vos, et al., “End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Sprin… [cited by applicant]
Deng, “The MNIST Database of Handwritten Digit Images for Machine Learning Research,” [Best of the Web], IEEE Signal Processing Magazine, Nov. 2012 (2 pages). [cited by applicant]
Deng, et al., “Probabilistic Neural Programmed Networks for Scene Generation,” 32nd Conference on Neural Information Processing Systems (2018) (11 pages). [cited by applicant]
Eslami, et al., “Attend, Infer, Repeat: Fast Scene Understanding with Generative Models,” arXiv:1603.08575v3, Aug. 12, 2016 (17 pages). [cited by applicant]
Gavrila, et al., “Multi-Feature Hierarchical Template Matching Using Distance Transforms,” in Proc, IEEE International Conference on Pattern Recognition (1998) (7 pages). [cited by applicant]
Grauman, et al., “Fast Contour Matching Using Approximate Earth Mover's Distance,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2004 (8 pages). [cited by applicant]
Guan, et al., “Deformable Cardiovascular Image Registration via Multi-Channel Convolutional Neural Network,” IEEE Access (2019) (11 pages). [cited by applicant]
Hanocka, et al., “ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning,” arXiv:1804.08497v2, Oct. 30, 2018 (14 pages). [cited by applicant]
Hirshberg, et al., “Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape,” in European Conference on Computer Vision. Springer, 2012 (14 pages). [cited by applicant]
Jaderberg, et al., “Spatial Transformer Networks,” arXiv:1506.02025v3, Feb. 4, 2016 (15 pages). [cited by applicant]
Kanazawa, et al., “Warpnet: Weakly Supervised Matching for Single-View Reconstruction,” arXiv:1604.05592v2, Jun. 20, 2016 (9 pages). [cited by applicant]
Karnewar, et al., “MSG-GAN: Multi-Scale Gradient GAN for Stable Image Synthesis,” arXiv:1903.06048v2, Mar. 22, 2019 (9 pages). [cited by applicant]
Klein, et al., “Elastix: A Toolbox for Intensity-Based Medical Image Registration,” IEEE Transactions on Medical Imaging, vol. 29, No. 1, Jan. 2010 (10 pages). [cited by applicant]
Miao, et al., “Dilated FCN for Multi-Agent 2D/3D Medical Image Registration,” the 32nd AAAI Conference on Artificial Intelligence (2018) (8 pages). [cited by applicant]
Moore, “The Evolution of the Concept of Homeomorphism,” Historia Mathematica, 34 (2007) (11 pages). [cited by applicant]
Nacken, “Chamfer Metrics in Mathematical Morphology,” Journal of Mathematical Imaging and Vision, 4 (1994) (21 pages). [cited by applicant]
Ronneberger, et al. “U-Net: Convolutional Networks for Biomedical Image Segmentation,” arXiv:1505.04597v1, May 18, 2015 (8 pages). [cited by applicant]
Rother, et al., “Grabcut: Interactive Foreground Extraction Using Iterated Graph Cuts,” in ACM transactions on graphics (TOG) (2004) (6 pages). [cited by applicant]
Saxena, et al., “A survey of Recent and Classical Image Registration Methods,” International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 7, No. 4 (2014) (10 pages). [cited by applicant]
Sun, et al., “Learning Optical Flow,” in European Conference on Computer Vision, Springer, 2008 (16 pages). [cited by applicant]
Yu, et al., “CASENet: Deep Category-Aware Semantic Edge Eetection,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) (10 pages). [cited by applicant]
Yu, et al., “Simultaneous Edge Alignment and Learning,” arXiv:1808.01992v3, Oct. 26, 2018 (30 pages). [cited by applicant]
Zlateski, et al., “On the Importance of Label Quality for Semantic Segmentation,” in Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (9 pages). [cited by applicant]
Lin, et al., “Feature Pyramid Networks for Object Detection,” arXiv:1612.03144v2, Apr. 19, 2017 (10 pages). [cited by applicant]
Lindeberg, “Scale-Space Theory: A Basic Tool for Analyzing Structures at Different Scales,” Journal of Applied Statistics (1994) (47 pages). [cited by applicant]
Lowe, “Object Recognition from Local Scale-Invariant Features,” Proc. of the International Conference on Computer Vision (1999) (8 pages). [cited by applicant]
Pan, et al., “Fully Convolutional Neural Networks with Full-Scale-Features for Semantic Segmentation,” In 31st AAAI Conference on Artificial Intelligence (2017) (7 pages). [cited by applicant]
Philbin, et al., “Object Retrieval with Large Vocabularies and Fast Spatial Matching,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition (8 pages). [cited by applicant]
Redmon, et al., “YOLO9000: Better, Faster, Stronger,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017) (9 pages). [cited by applicant]
Sun, et al., “Secrets of Optical Flow Estimation and Their Principles,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (8 pages). [cited by applicant]
Notice of Allowanced dated Oct. 31, 2022, issued in connection with U.S. Appl. No. 17/104,698 (5 pages). [cited by applicant]