IP Library Granted Patent US 12,737,637
Granted Patent B2
US 12,737,637 · App. 18/177,839 · Granted Sep 15, 2026

Machine learning model training with adversarial learning and triplet loss regularization

Inventors: Xuan-Hong Dang (Chappaqua, NY); Dinesh C. Verma (New Castle, NY); Seraphin Bernard Calo (Cortlandt Manor, NY); Petros Zerfos (New York, NY)
Assignee: International Business Machines Corporation
G06N3/094G06N3/045G06N3/096
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,737,637
App. No.
18/177,839
Granted
Sep 15, 2026
Kind
B2
Abstract

A method, computer system, and a computer program product for training a machine learning model are provided. A first set of labelled training data from a source domain is obtained. A second set of labelled training data from a target domain is obtained. A number of labelled samples of the first set is greater than a number of labelled samples of the second set. The first machine learning model is trained with the first set and the second set and with a discriminator so that the discriminator is unable to distinguish whether a sample is from the first set or from the second set. The first machine learning model is trained with triplet loss regularization using the first set and the second set.

Claims (63)

1 . A method comprising:

obtaining a first set of labelled training data from a source domain;

obtaining a second set of labelled training data from a target domain, a number of labelled samples of the first set being greater than a number of labelled samples of the second set;

training a first machine learning model with the first set and the second set and with a discriminator so that the discriminator is unable to distinguish whether a sample, represented by an embedding produced via the first machine learning model in response to receiving the sample, is from the first set or from the second set;

training the first machine learning model with triplet loss regularization using the first set and the second set; and

training a classifier of the first machine learning model using classification loss from the first set and the second set;

wherein the training of the first machine learning model with the discriminator and with the triplet loss regularization occurs iteratively in multiple epochs based on refining of a sample pool of the first set and the second set so that a first epoch of the multiple epochs is performed with a first version of the sample pool and a second epoch of the multiple epochs is performed with a second version of the sample pool, the second version being modified compared to the first version, wherein the sample pool comprises triplet sets of a respective anchor sample, a respective positive sample, and a respective negative sample, and the refining occurs by evaluating relevancy of the triplet sets in a latent common embedding space between the first set and the second set.

2 . The method of claim 1 , wherein the respective anchor sample and the respective positive sample together form a respective matching pair for a respective one of the triplet sets, wherein the respective anchor sample and the respective negative sample together form a respective non-matching pair for the respective one of the triplet sets, and

wherein the refining further comprises comparing a first distance between the matching pair to a second distance between the non-matching pair to determine a triplet function value.

3 . The method of claim 2 , wherein the refining further comprises discarding a first triplet of the triplet sets for the iterative training in response to the triplet function value for the first triplet not exceeding a threshold value.

4 . The method of claim 1 , further comprising:

updating a first generator encoder of the first machine learning model based on:

domain loss from the training of the discriminator,

distance loss from the training with the triplet loss regularization, and

the classification loss;

wherein the trained first machine learning model for use in an inference phase comprises the updated first generator encoder and the trained classifier.

5 . The method of claim 4 , further comprising performing classification with the trained first machine learning model via:

inputting a new sample into the updated first generator encoder so that the updated first generator encoder generates an embedding in an embedding space and

inputting the embedding into the trained classifier so that the trained classifier produces a class prediction.

6 . The method of claim 1 , wherein a first generator encoder of the first machine learning model is updated based on:

domain loss from the training with the discriminator,

distance loss from the training with the triplet loss regularization, and

the classification loss.

7 . The method of claim 6 , further comprising updating a second generator encoder based on:

the domain loss from the training with the discriminator, and

the classification loss;

wherein the first generator encoder and the second generator encoder are used for the training with the discriminator and for the training of the classifier.

8 . The method of claim 7 , further comprising:

comparing at least one of a shape and a dimension of the labelled training data of the second set to a corresponding at least one of a shape and a dimension of the labelled training data of the first set; and

initializing weights of the first generator encoder based on the comparing.

9 . The method of claim 8 , wherein via the comparison the at least one of the shape and the dimension of the labelled training data of the second set passes a similarity threshold with the corresponding at least one of a shape and a dimension of the labelled training data of the first set; and

wherein the initialization comprises weights of the second generator encoder being implemented as the weights for the first generator encoder.

10 . The method of claim 8 , wherein via the comparison the at least one of the shape and the dimension of the labelled training data of the second set fails a similarity threshold with the corresponding at least one of a shape and a dimension of the labelled training data of the first set; and

wherein the initialization comprises randomly initializing the weights of the first generator encoder.

11 . The method of claim 1 , wherein the triplet loss regularization comprises penalization in response to samples from the second set being mapped at a distance greater than a distance threshold from samples from the first set and from the second set having same class labels as the samples from the second set.

12 . The method of claim 1 , wherein the triplet loss regularization comprises penalization in response to samples from the second set being mapped at a distance less than a distance threshold from samples from the first set and from the second set having different class labels as the samples from the second set.

13 . The method of claim 1 , wherein the triplet loss regularization comprises:

penalization in response to samples from the second set being mapped at a first distance greater than a first distance threshold from samples from the first set and from the second set having same class labels as the samples from the second set and

penalization in response to samples from the second set being mapped at a second distance less than a second distance threshold from samples from the first set and from the second set having different class labels as the samples from the second set.

14 . The method of claim 1 , wherein the source domain is of a first type and the target domain is of a second type different than the first type.

15 . The method of claim 1 , further comprising re-calculating distances for all triplet groups of the sample pool in response to a change amount of the sample pool exceeding or equaling a pre-determined threshold, wherein the change amount refers to a percentage of overall samples that are discarded from the sample pool due to being less relevant to the target domain, and wherein the re-calculating of the distances adjusts a shape of embeddings in the latent common embedding space.

16 . The method of claim 1 , wherein a first group of the triplet sets comprises respective anchor samples from the second set and respective positive or negative samples from the first set.

17 . The method of claim 16 , wherein a second group of the triplet sets comprises respective anchor samples from the second set and respective additional samples from the second set, and wherein the first group includes more of the triplet sets than the second group does.

18 . The method of claim 1 , wherein a first group of the triplet sets comprises respective anchor samples from the second set and respective positive and negative samples from the first set.

19 . A computer system comprising:

one or more processors,

one or more computer-readable storage media, and

program instructions stored on at least one of the one or more computer-readable storage media for execution by at least one of the one or more processors to cause the computer system to:

obtain a first set of labelled training data from a source domain;

obtain a second set of labelled training data from a target domain, a number of labelled samples of the first set being greater than a number of labelled samples of the second set;

train a first machine learning model with the first set and the second set and with a discriminator so that the discriminator is unable to distinguish whether a sample, represented by an embedding produced via the first machine learning model in response to receiving the sample, is from the first set or from the second set;

train the first machine learning model with triplet loss regularization using the first set and the second set; and

train a classifier of the first machine learning model using classification loss from the first set and the second set;

wherein the training of the first machine learning model with the discriminator and with the triplet loss regularization occurs iteratively in multiple epochs based on refining of a sample pool of the first set and the second set so that a first epoch of the multiple epochs is performed with a first version of the sample pool and a second epoch of the multiple epochs is performed with a second version of the sample pool, the second version being modified compared to the first version, wherein the sample pool comprises triplet sets of a respective anchor sample, a respective positive sample, and a respective negative sample, and the refining occurs by evaluating relevancy of the triplet sets in a latent common embedding space between the first set and the second set.

20 . A computer program product comprising:

a computer-readable storage medium; and

program instructions stored on the computer-readable storage medium, wherein the program instructions are executable by a processor to cause the processor to:

obtain a first set of labelled training data from a source domain;

obtain a second set of labelled training data from a target domain, a number of labelled samples of the first set being greater than a number of labelled samples of the second set;

train a first machine learning model with the first set and the second set and with a discriminator so that the discriminator is unable to distinguish whether a sample, represented by an embedding produced via the first machine learning model in response to receiving the sample, is from the first set or from the second set;

train the first machine learning model with triplet loss regularization using the first set and the second set; and

training a classifier of the first machine learning model using classification loss from the first set and the second set;

wherein the training of the first machine learning model with the discriminator and with the triplet loss regularization occurs iteratively in multiple epochs based on refining of a sample pool of the first set and the second set so that a first epoch of the multiple epochs is performed with a first version of the sample pool and a second epoch of the multiple epochs is performed with a second version of the sample pool, the second version being modified compared to the first version, wherein the sample pool comprises triplet sets of a respective anchor sample, a respective positive sample, and a respective negative sample, and the refining occurs by evaluating relevancy of the triplet sets in a latent common embedding space between the first set and the second set.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 3, 2023
From: DANG, XUAN-HONG; VERMA, DINESH C.; CALO, SERAPHIN BERNARD; ZERFOS, PETROS
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 062866/0447 →
Continuity (1)
Related Publication 20240296334A1 · Sep 5, 2024
References Cited (45)
US 10354199B2 · Clinchant · 2019 [cited by applicant]
US 11010637B2 · Marco · 2021 [cited by applicant]
US 11106182B2 · Hosseini-Asl · 2021 [cited by applicant]
US 11361254B2 · Butvinik · 2022 [cited by applicant]
US 11443242B2 · Velizhev · 2022 [cited by applicant]
US 11448753B2 · Rostami · 2022 [cited by applicant]
US 20190286073A1 · Hosseini-Asl · 2019 [cited by applicant]
US 20190354807A1 · Tsai · 2019 [cited by examiner]
US 20200264300A1 · Rostami · 2020 [cited by applicant]
US 20210019629A1 · Chidlovskii · 2021 [cited by examiner]
US 20210192363A1 · Kolouri · 2021 [cited by applicant]
US 20210264318A1 · Butvinik · 2021 [cited by applicant]
US 20210295208A1 · Xu · 2021 [cited by applicant]
US 20210326749A1 · Velizhev · 2021 [cited by applicant]
US 20220180200A1 · Yu · 2022 [cited by applicant]
US 20220198339A1 · Zhao · 2022 [cited by applicant]
US 20220261633A1 · Butvinik · 2022 [cited by applicant]
US 20220261657A1 · Puzanov · 2022 [cited by applicant]
US 20220318567A1 · Muhamedrahimov · 2022 [cited by examiner]
CN 112541580A · 2021 [cited by examiner]
Gong et al., Domain Adaptation with Conditional Transferable Components, 2016 (Year: 2016). [cited by examiner]
Shuang Li et al., Domain Invariant and Class Discriminative Feature Learning for Visual Domain Adaptation, Jun. 2018 (Year: 2018). [cited by examiner]
Lei Li et al., Unsupervised Domain Adaptation via Discriminative Feature Learning and Classifier Adaptation from Center-based Distances, May 2022 (Year: 2022). [cited by examiner]
Luo, Deep learning based forecasting of photovoltaic power generation by incorporating domain knowledge, Mar. 2021 (Year: 2021). [cited by examiner]
Bekkouch et al., Triplet Loss Network for Unsupervised Domain Adaptation, May 2019 (Year: 2019). [cited by examiner]
Dash et al., A Review of Some Techniques for Inclusion of Domain-Knowledge into Deep Neural Networks, 2022 (Year: 2022). [cited by examiner]
Deng et al., Rethinking Triplet Loss for Domain Adaptation, Jan. 2021 (Year: 2021). [cited by examiner]
Lagunes-Fortiz et al., Learning Discriminative Embeddings for Object Recognition On-the-fly, 2019 (Year: 2019). [cited by examiner]
Yin et al., Metric-learning-assisted Domain Adaptation, May 2021 (Year: 2021). [cited by examiner]
Zonoozi et al., A Survey on Adversarial Domain Adaptation, Jul. 2022 (Year: 2022). [cited by examiner]
Cao et al., TripletGAN: Training a Generative Model with Triplet Loss, Nov. 2017 (Year: 2017). [cited by examiner]
Guo and Lovell, Domain-aware Triplet loss in Domain Generalization, Mar. 1, 2023 (Year: 2023). [cited by examiner]
Hacohen and Weinshall, On The Power of Curriculum Learning in Training Deep Networks, May 2019 (Year: 2019). [cited by examiner]
Komatsuzaki, One Epoch is All You Need, Jun. 2019 (Year: 2019). [cited by examiner]
Blitzer et al., “Domain Adaptation with Structural Correspondence Learning,” Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), Jul. 2006, https://aclanthology.org/W06-1… [cited by applicant]
Fu et al., “Unsupervised Domain Adaptation Based on Pseudo-Label Confidence,” IEEE Access, vol. 9, originally published Jun. 9, 2021, date of current version Jun. 23, 2021, pp. 87049-87057. [cited by applicant]
Hermans et al., “In Defense of the Triplet Loss for Person Re-Identification,” arXiv:1703.07737v4 [cs.CV] Nov. 21, 2017, https://arxiv.org/pdf/1703.07737.pdf, 17 pages. [cited by applicant]
Li et al., “Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation,” originally published Apr. 2021, EEE, Jun. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2505-2514. [cited by applicant]
Li et al., “Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation,” arXiv:2010.04647v3 [cs.LG] Apr. 4, 2021, https://arxiv.org/abs/2010.04647, 14 pages. [cited by applicant]
Motiian et al., “Few-Shot Adversarial Domain Adaptation,” 31st Conference on Neural Information Processing Systems (NIPS 2017), arXiv:1711.02536v1 [cs.CV] Nov. 5, 2017, https://arxiv.org/pdf/1711.02536v1.pdf, 11 pages. [cited by applicant]
Nguyen et al., “STEM: An approach to Multi-source Domain Adaptation with Guarantees,” IEEE, Oct. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9332-9343. [cited by applicant]
Rakshit et al., “Open-Set Domain Adaptation Under Few Source-Domain Labeled Samples,” IEEE, Jun. 2022 IEEE/ Cvf Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 4028-4037. [cited by applicant]
Singla et al., “Preparing Network Intrusion Detection Deep Learning Models with Minimal Data Using Adversarial Domain Adaptation,” ACM, ASIA CCS '20, Oct. 5-9, 2020, https://dl.acm.org/doi/pdf/10.1145/3320269.3384718, p… [cited by applicant]
Tian et al., “VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, No. 6, Jun. 2022, pp. 3749-3760. [cited by applicant]
Wang et al., “Generalizing from a Few Examples: A Survey on Few-Shot Learning,” ACM Comput. Surv., vol. 1, No. 1, Article 1, Mar. 2020, arXiv:1904.05046v3 [cs.LG] Mar. 29, 2020, https://arxiv.org/pdf/1904.05046.pdf, 34 … [cited by applicant]