IP Library › Granted Patent US 12,657,469
Granted Patent B2
US 12,657,469 · App. 17/740,383 · Granted Jun 16, 2026

Training machine-trained models by directly specifying gradient elements

Inventors: Hong Xuan (Bellevue, WA); Xi Chen (Bellevue, WA); Saurajit Mukherjee (Kirkland, WA); Li Huang (Sammamish, WA); Kun Wu (Bellevue, WA); Arun Kumar Sacheti (Sammamish, WA); Kamal Ginotra (Kirkland, WA); Meenaz Aliraza Merchant (Kirkland, WA)
Assignee: Microsoft Technology Licensing, LLC
G06N3/084G06N3/045
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,657,469
App. No.
17/740,383
Granted
Jun 16, 2026
Kind
B2
Abstract

A computer-implemented technique performs machine learning that bypasses the traditional design of loss functions. The technique includes receiving plural instances of gradient objective information. Each of the plural instances includes a particular combination of plural gradient elements. The technique produces plural sets of machine-trained parameter values using the plural respective instances of gradient objective information. The technique performs this operation based on the plural instances of gradient objective information as given, without calculating the plural instances of gradient objective information using loss functions. The technique then measures performance of the plural sets of machine-trained parameter values in an application system. Based on the measured performance, the technique provides output information that identifies a particular set of machine-trained parameter values that satisfies a prescribed test.

Claims (60)

1 . A computer-implemented method for performing machine learning to produce parameters values of a neural network, comprising:

receiving plural instances of gradient objective information, each of the plural instances of gradient objective information including a particular combination of plural gradient elements selected from a data store of gradient elements, the plural instances of gradient objective information including different respective combinations of gradient elements;

producing plural sets of machine-trained parameter values of the neural network using the plural respective instances of gradient objective information, said producing being performed without calculation of the plural instances of gradient objective information using loss functions during training,

the producing performing training on training examples in successive forward and backward passes to produce each set of machine-trained parameters of the neural network, the producing performing each backward pass by back-propagating an instance of instantiated gradient information backward through levels of the neural network;

measuring performance of the plural sets of machine-trained parameter values in an application system; and

based on test results generated by said measuring, producing output information that identifies a selected set of machine-trained parameter values from the plural sets of machine-trained parameter values that most effectively satisfies a specified standard of performance, the selected set of parameter values being produced using a corresponding selected instance of gradient information having a selected combination of gradient elements,

the application system using the selected set of machine-trained parameter values to produce an output result, the output result having fewer errors compared to output results produced by other considered instances of gradient information having other corresponding combinations of gradient elements.

2 . The computer-implemented method of claim 1 , wherein a particular instance of gradient objective information of the plural instances of gradient objective information includes:

a first gradient element that is part of a gradient of a first loss function; and

a second gradient element that is part of a gradient of a second loss function, the second loss function being different than the first loss function,

the first loss function and the second loss function being deep metric learning (DML) loss functions.

3 . The computer-implemented method of claim 1 , wherein a particular instance of gradient objective information of the plural instances of gradient objective information includes:

a first pair-based gradient element that is based on similarity information that depends on comparison of two input items of a triplet, including an anchor item, and a positive item or a negative item; and

a second triplet-based gradient element that is based on similarity information that depends on comparison of three input items of the triplet, including the anchor item and the positive item, or the anchor item and the negative item.

4 . The computer-implemented method of claim 1 , wherein a first set of machine-trained parameter values is produced for a first instance of gradient objective information, of the plural instances of gradient objective information, by:

mapping a training example that includes at least an anchor item, positive item, and negative item to an embedding space using the neural network, to respectively produce at least an anchor-item vector, a positive-item vector, and a negative-item vector, respectively;

generating similarity information based on at least the anchor-item vector, the positive-item vector, and the negative-item vector, a triplet relationship between the anchor item, positive item, and negative item being established prior to said generating of similarity information, or after said generating of similarity information;

producing instantiated gradient information based on the similarity information and the first instance of gradient input information, said producing the instantiated gradient information using the first instance of gradient objective information as received without calculation of the first instance of gradient objective information from a loss function during training;

back-propagating the instantiated gradient information through the neural network and performing optimization, to produce model update information, and using the model update information to update the first set of machine-trained values; and

repeating said mapping, generating, producing instantiated gradient information, and back-propagating plural times for other training examples.

5 . The computer-implemented method of claim 4 , wherein the training example includes at least one image.

6 . The computer-implemented method of claim 5 , wherein the training example also includes at least at least one text item.

7 . The computer-implemented method of claim 4 , wherein the neural network includes a first neural network for mapping the anchor item into the anchor-item vector, and a second neural network for mapping the positive item and the negative item into the positive-item vector and negative-item vector, respectively.

8 . The computer-implemented method of claim 7 , wherein the first neural network processes images and the second neural network processes text items.

9 . The computer-implemented method of claim 7 , wherein the first neural network and/or the second neural network has a transformer-based architecture.

10 . The computer-implemented method of claim 7 , wherein the first neural network and/or the second neural network has a convolutional neural network architecture.

11 . The computer-implemented method of claim 1 , wherein the application system is a search application that identifies a target item that matches an input query.

12 . The computer-implemented method of claim 1 , wherein the application system performs a control action based on output results produced by the selected set of parameter values.

13 . A computing system, comprising:

a computer-implemented application system, including a hardware processor, that performs an application task based on a neural network, neural network using a selected set of machine-trained parameter values produced by a computer-implemented training system, the training system also including a hardware processor,

the selected set of parameter values being produced by the training system, using the hardware processor provided by the training system, by:

receiving plural instances of gradient objective information, each of the plural instances of gradient objective information including a particular combination of plural gradient elements selected from a data store of gradient elements, the plural instances of gradient objective information including different respective combinations of gradient elements;

producing plural sets of machine-trained parameter values using the plural respective instances of gradient objective information, said producing being performed without calculation of the plural instances of gradient objective information using loss functions during training,

the producing performing training on training examples in successive forward and backward passes to produce each set of machine-trained parameters of the neural network, the producing performing each backward pass by back-propagating an instance of instantiated gradient information backward through levels of the neural network;

measuring performance of the plural sets of machine-trained parameter values in the application system; and

based on test results generated by said measuring, producing output information that identifies the selected set of machine-trained parameter values from the plural sets of machine-trained parameter values that most effectively satisfies a specified standard of performance, the selected set of parameter values being produced using a corresponding selected instance of gradient information having a selected combination of gradient elements,

the application system using the selected set of machine-trained parameter values to produce an output result, the output result having fewer errors compared to output results produced by other considered instances of gradient information having other corresponding combinations of gradient elements.

14 . The computing system of claim 13 , wherein a particular instance of gradient objective information of the plural instances of gradient objective information includes:

a first gradient element that is part of a gradient of a first loss function; and

a second gradient element that is part of a gradient of a second loss function, the second loss function being different than the first loss function.

15 . The computing system of claim 13 , wherein a first set of machine-trained parameter values is produced for a first instance of gradient objective information, of the plural instances of gradient objective information, by:

mapping a training example that includes at least an anchor item, positive item, and negative item to an embedding space using the neural network, to respectively produce at least an anchor-item vector, a positive-item vector, and a negative-item vector, respectively, the positive item matching the anchor item, and the negative item not matching the anchor item;

generating similarity information based on at least the anchor-item vector, the positive-item vector, and the negative-item vector, a triplet relationship between the anchor item, positive item, and negative item being established prior to said generating of similarity information, or after said generating of similarity information;

producing instantiated gradient information based on the similarity information and the first instance of gradient input information, said producing instantiated gradient information using the first instance of gradient objective information as received and without calculation of the first instance of gradient objective information from a loss function during training;

back-propagating the instantiated gradient information through the neural network and performing optimization, to produce model update information, and using the model update information to update the first set of machine-trained values; and

repeating said mapping, generating, producing instantiated gradient information, and back-propagating plural times for other training examples.

16 . The computing system of claim 15 , wherein the neural network includes a first neural network for mapping the anchor item into the anchor-item vector, and a second neural network for mapping the positive item and the negative item into the positive-item vector and negative-item vector, respectively.

17 . The computing system of claim 16 , wherein the first neural network processes images and the second neural network processes text items.

18 . The computing system of claim 13 , wherein the application task performed by the application system is a search application that identifies a target item that matches an input query.

19 . A computer-readable storage medium for storing computer-readable instructions, one or more hardware processors executing the computer-readable instructions to perform a method that comprises:

receiving plural instances of gradient objective information, each of the plural instances of gradient objective information including a particular combination of plural gradient elements selected from a data store of gradient elements, the plural instances of gradient objective information including different respective combinations of gradient elements; and

producing plural sets of machine-trained parameter values of a neural network using the plural respective instances of gradient objective information, said producing being performed without calculation of the plural instances of gradient objective information using loss functions during training,

the producing performing training on training examples in successive forward and backward passes to produce each set of machine-trained parameters of the neural network, the producing performing each backward pass by back-propagating an instance of instantiated gradient information backward through levels of the neural network;

said producing including, for a first instance of gradient objective information, of the plural instances of gradient objective information:

mapping a training example that includes at least an anchor item, positive item, and negative item to a vector space using a neural network, to respectively produce at least an anchor-item vector, a positive-item vector, and a negative-item vector, respectively, the positive item matching the anchor item, and the negative item not matching the anchor item;

generating similarity information based on at least the anchor-item vector, the positive-item vector, and the negative-item vector, a triplet relationship between the anchor item, positive item, and negative item being established prior to said generating of similarity information, or after said generating of similarity information;

producing instantiated gradient information based on the similarity information and the first instance of gradient input information, said producing instantiated gradient information using the first instance of gradient objective information as received and without calculation of the first instance of gradient objective information from a loss function during training;

back-propagating the instantiated gradient information through the neural network and performing optimization, to produce model update information, and using the model update information to update the first set of machine-trained values; and

repeating said mapping, generating, producing the instantiated gradient information, and back-propagating plural times for other training examples.

20 . The computer-readable storage medium of claim 19 , wherein the training example includes at least one image and at least one text item.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2022
From: XUAN, HONG; CHEN, XI; MUKHERJEE, SAURAJIT; HUANG, LI; WU, KUN; SACHETI, ARUN KUMAR; GINOTRA, KAMAL; MERCHANT, MEENAZ ALIRAZA
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 059876/0720 →
Continuity (1)
Related Publication 20230368031A1 · Nov 16, 2023
References Cited (73)
US 20150278200A1 · He et al. · 2015 [cited by applicant]
US 20200250476A1 · Sather · 2020 [cited by examiner]
US 20200302340A1 · Durand · 2020 [cited by examiner]
US 20220036127A1 · Lin · 2022 [cited by examiner]
US 20220138252A1 · Hsiao · 2022 [cited by examiner]
Wu, Yiling, et al. “Learning fragment self-attention embeddings for image-text matching.” Proceedings of the 27th ACM international conference on multimedia. 2019. (Year: 2019). [cited by examiner]
Engilberge, Martin. Deep Inside Visual-Semantic Embeddings. Diss. Sorbonne Universite, 2020. (Year: 2020). [cited by examiner]
Wang, Guangzhi. “Dynamic knowledge distillation with cross-modality knowledge transfer.” Proceedings of the 29th ACM International Conference on Multimedia. 2021. (Year: 2021). [cited by examiner]
He, et al., “Deep Residual Learning for Image Recognition,” Open Access version of paper in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 9 pages. [cited by applicant]
Hochreiter, et al., “Long Short-Term Memory,” in Neural Computation, vol. 9, Issue 8, Nov. 1997, 32 pages. [cited by applicant]
Hoffer, et al., “Deep metric learning using Triplet network,” arXiv, Cornell University, arXiv:1412.6622v4 [cs.LG], Dec. 4, 2018, 8 pages. [cited by applicant]
Huang, et al., “Learning Semantic Concepts and Order for Image and Sentence Matching,” Open Access version of paper in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, No. 3, Mar. 2020, 9 pages. [cited by applicant]
Kiros, et al., “Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models,” arXiv, Cornell University, arXiv:1411.2539v1 [cs.LG], Nov. 10, 2014, 13 pages. [cited by applicant]
Krause, et al., “3D Object Representations for Fine-Grained Categorization,” in ICCVW '13: Proceedings of the 2013 EEE International Conference on Computer Vision Workshops, Dec. 2013, 8 pages. [cited by applicant]
Lee, et al., “Stacked Cross Attention for Image-Text Matching,” arXiv, Cornell University, arXiv:1803.08024v1 [cs.CV, Mar. 21, 2018, 23 pages. [cited by applicant]
Li, et al., “Align before Fuse: Vision and Language Representation Learning with Momentum Distillation,” in 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021, 12 pages. [cited by applicant]
Li, et al., “Visual Semantic Reasoning for Image-Text Matching,” Open Access version of paper in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, 9 pages. [cited by applicant]
Li, et al., “Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks,” arXiv, Cornell University, arXiv:2004.06165v5 [cs.CV], Jul. 26, 2020, 21 pages. [cited by applicant]
Liu, et al., “ROBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv:1907.11692v1 [cs.CL], Jul. 26, 2019, 13 pages. [cited by applicant]
Liu, et al., “DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations,” Open Access version of paper in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Jun. 201… [cited by applicant]
Lu, et al., “VILBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks,” in 33rd Conference on Neural Information Processing Systems, 2019, 11 pages. [cited by applicant]
Mahajan, et al., “Exploring the Limits of Weakly Supervised Pretraining,” arXiv, Cornell University, arXiv:1805.00932v1 [cs.CV], May 2, 2018, 23 pages. [cited by applicant]
Paszke, et al., “Automatic differentiation in PyTorch,” in 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017, 4 pages. [cited by applicant]
Schroff, et al., “FaceNet: A Unified Embedding for Face Recognition and Clustering,” Open Access version of paper in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 9 pages. [cited by applicant]
Sohn, Kihyuk, “Improved Deep Metric Learning with Multi-class N-pair Loss Objective,” in Advances in Neural Information Processing Systems 29 (NIPS 2016), 2016, 9 pages. [cited by applicant]
Song, et al., “Deep Metric Learning via Lifted Structured Feature Embedding,” Open Access version of paper of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 9 pages. [cited by applicant]
Sun, et al., “Circle Loss: A Unified Perspective of Pair Similarity Optimization,” Open Access version of paper in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, 10 pages. [cited by applicant]
Tan, et al., “LXMERT: Learning Cross-Modality Encoder Representations from Transformers,” arXiv, Cornell University, arXiv:1908.07490v3 [cs.CL], Dec. 3, 2019, 14 pages. [cited by applicant]
Vaswani, et al., “Attention is All you Need,” in Advances in Neural Information Processing Systems 30 (NIPS 2017), 30, 2017, 11 pages. [cited by applicant]
Anderson, et al., “Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering,” Open Access version of paper in in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 201… [cited by applicant]
Wang, et al., “Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning,” Open Access version of paper in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, 9 pages. [cited by applicant]
Welinder, et al., “Caltech-UCSD Birds 200,” Technical Report CNS-TR-2010-001, California Institute of Technology, 2010, 15 pages. [cited by applicant]
Wu, et al., “Unified Visual-Semantic Embeddings: Bridging Vision and Language with Structured Meaning Representations,” Open Access version of paper 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (C… [cited by applicant]
Xuan, et al., “Hard negative examples are hard, but useful,” arXiv, Cornell University, arXiv:2007.12749v2 [cs.CV], Feb. 25, 2021, 20 pages. [cited by applicant]
Yi, et al., “Deep Metric Learning for Practical Person Re-Identification,” arXiv, Cornell University, arXiv:1407.4979v1 [cs.CV], Jul. 18, 2014, 11 pages. [cited by applicant]
Young, et al., “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,” in Transactions of the Association for Computational Linguistics, 2, 2014, pp. 67-78. [cited by applicant]
Yu, et al., “Gradient Surgery for Multi-Task Learning,” arXiv, Cornell University, arXiv:2001.06782v4 [cs.LG], Dec. 22, 2020, 27 pages. [cited by applicant]
Zeng, et al., “Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts,” arXiv, Cornell University, arXiv:2111.08276v3 [cs.CL], Jun. 1, 2022, 16 pages. [cited by applicant]
Zhang, et al., “VinVL: Revisiting Visual Representations in Vision-Language Models,” Open Access version of paper in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, 10 pag… [cited by applicant]
Zhang, et al., “P2SGrad: Refined Gradients for Optimizing Deep Face Models,” Open Access version of paper in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, 9 pages. [cited by applicant]
Xuan, et al., “Dissecting Deep Metric Learning Losses for Image-Text Retrieval,” arXiv, Cornel University, arXiv:2210.13188v1 [cs.CV], Oct. 21, 2022, 10 pages. [cited by applicant]
Xuan, et al., “Dissecting Deep Metric Learning Losses for Image-Text Retrieval,” Open Access version of paper in 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Jan. 2023, 10 pages. [cited by applicant]
Xuan, Hong, “GitHub—microsoft/VSE_Gradient,” available at https://github.com/microsoft/VSE_Gradient, Dec. 16, 2022, 2 pages. [cited by applicant]
Xuan, et al., “Dissecting the impact of different loss functions with gradient surgery,” arXiv, Cornell University, arXiv:2201.11307v1 [cs.CV], Jan. 27, 2022, 13 pages. [cited by applicant]
Cao, et al., “Image-text Retrieval: A Survey on Recent Research and Development,” arXiv, Cornell University, arXiv:2203.14713v2 [cs.IR], Apr. 28, 2022, 8 pages. [cited by applicant]
PCT Search Report and Written Opinion for PCTUS2023/013398, date of mailing Jun. 1, 2023, 18 pages. [cited by applicant]
Wang, et al., “Consensus-Aware Visual-Semantic Embedding for Image-Text Matching,” arXiv, Cornell University, arXiv:2007.08883v2 [cs.CV], Feb. 1, 2021, 17 pages. [cited by applicant]
Chen, et al., “Learning the Best Pooling Strategy for Visual Semantic Embedding,” Open Access version of paper in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, 10 pages. [cited by applicant]
Chen, et al., “Microsoft COCO Captions: Data Collection and Evaluation Server,” arXiv, Cornell University, arXiv:1504.00325v2 [cs.CV], Apr. 3, 2015, 7 pages. [cited by applicant]
Chen, et al., “UNITER: UNiversal Image-TExt Representation Learning,” in Proceedings of the 16th European Conference on Computer Vision, Aug. 2020, 17 pages. [cited by applicant]
Devlin, et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of NAACL-HLT 2019, Jun. 2019, pp. 4171-4186. [cited by applicant]
Dosovitskiy, et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv, Cornell University, arXiv:2010.11929v2 [cs.CV], Jun. 3, 2021, 22 pages. [cited by applicant]
Faghri, et al., “VSE++: Improving Visual-Semantic Embeddings with Hard Negative,” in Proceedings of the British Machine Vision Conference (BMVC), 2018, 13 pages. [cited by applicant]
Frome, et al., “DeViSE: A Deep Visual-Semantic Embedding Model,” in Advances in Neural Information Processing, 2013, 9 pages. [cited by applicant]
Goldberger, et al., “Neighbourhood Components Analysis,” in Advances in Neural Information Processing, 2004,8 pages. [cited by applicant]
Hadsell, et al., “Dimensionality Reduction by Learning an Invariant Mapping,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), 2006, 8 pages. [cited by applicant]
Schroff, et al., “FaceNet: A Unified Embedding for Face Recognition and Clustering,” in arXiv repository, arXiv:1503.03832v3 [cs.CV], Jun. 17, 2015, 10 pages. [cited by applicant]
Sohn, Kihyuk, “Improved Deep Metric Learning with Multi-class N-pair Loss Objective,” in Proceedings of the 30th Conference on Neural Information Processing Systems (NIPS 2016), 2016, 9 pages. [cited by applicant]
Sun, et al., “Circle Loss: A Unified Perspective of Pair Similarity Optimization,” in arXiv repository, arXiv:2002.10857v2 [cs.CV], Jun. 15, 2020, 10 pages. [cited by applicant]
Yi, et al., “Deep Metric Learning for Practical Person Re-Identification,” in arXiv repository, arXiv:1407.4979v1 [cs.CV], Jul. 18, 2014, 11 pages. [cited by applicant]
Wang, et al., “Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning,” in arXiv repository, arXiv:1904.06627v3 [cs.CV], Mar. 23, 2020, 12 pages. [cited by applicant]
Goldberger, et al. “Neighbourhood Components Analysis,” in Proceeding of Advances in Neural Information Processing Systems 17 (NIPS 2004), 2004, 8 pages. [cited by applicant]
Dosovitskiy, et al., “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” in arXiv repository, arXiv:2010.11929v2 [cs.CV], Jun. 3, 2021, 22 pages. [cited by applicant]
He, et al., “Deep Residual Learning for Image Recognition,” in arXiv repository, arXiv:1512.03385v1 [cs.CV], Dec. 10, 2015, 12 pages. [cited by applicant]
Vaswani, et al., “Attention is All You Need,” in arXiv repository, arXiv:1706.03762v5 [cs.CL], Dec. 6, 2017, 15 pages. [cited by applicant]
Devlin, et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in arXiv repository, arXiv:1810.04805v2 [cs.CL] May 24, 2019, 16 pages. [cited by applicant]
“A Gentle Introduction to Torch.Autograd,” available at https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html, accessed on Apr. 5, 2022, 4 pages. [cited by applicant]
“Autograd Mechanics,” available at https://pytorch.org/docs/stable/notes/autograd.html, accessed on Apr. 5, 2022, 8 pages. [cited by applicant]
Patel, Jay, “Digging Deeper in Metric Learning with Loss Functions,” available at https://towardsdatascience.com/metric-learning-loss-functions-5b67b3da99a5, Towards Data Science, Nov. 12, 2019, 17 pages. [cited by applicant]
Kumar, Vaibhav, “PyTorch Autograd: Understanding the heart of Pytroch's magic,” available at https://towardsdatascience.com/, Towards Data Science, Jan. 7, 2019, 15 pages. [cited by applicant]
“Losses—PyTorch Metric Learning,” available at https://kevinmusgrave.github.io/pytorch-metric-learning/losses/, accessed on Apr. 6, 2022, 31 pages. [cited by applicant]
“Triplet loss,” available at https://en.wikipedia.org/wiki/Triplet_loss, Wikipedia article, accessed on Sep. 16, 2025, 4 pages. [cited by applicant]
Vishwakarma, Neha, available at read://https_www.analyticsvidhya.com/?url=https%3A%2F%2Fwww.analyticsvidhya.com%2Fblog%2F2023%2F09%2Fwhat-is-adam-optimizer%2F%23h-steps-involved-in-the-adam-optimization-algorithm, “What… [cited by applicant]