IP Library Granted Patent US 12,626,098
Granted Patent B2
US 12,626,098 · App. 17/945,623 · Granted May 12, 2026

Method and system for creating an ensemble of neural network-based classifiers that optimizes a diversity metric

Inventors: Alejandro E. Brito (Mountain View, CA); Shantanu Rane (Palo Alto, CA)
Assignee: Genesee Valley Innovations, LLC
G06N3/045G06F18/2433G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,626,098
App. No.
17/945,623
Granted
May 12, 2026
Kind
B2
Abstract

One embodiment provides a system which facilitates construction of an ensemble of neural network-based classifiers that optimize a diversity metric. During operation, the system defines a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers. The system includes the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers. The system trains one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric. The system predicts an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

Claims (95)

1 . A computer-executable method, comprising:

defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers;

including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;

replacing a final classification layer in one or more neural networks with the three or more affine classifiers;

training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;

predicting an output for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric;

displaying, on a device associated with a user, information related to the predicted output for the respective data object, the information comprising one or more of:

a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;

a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;

a size of the ensemble of neural network-based classifiers;

an approach associated with the respective data object, including whether the respective data object is associated with diversity;

a type of classifier used in a final classifier layer;

the predicted outcome based on a respective classifier; and

an overall ensemble result;

receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classifier layer, a selected classifier, and the regularization term;

predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and

increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.

2 . The method of claim 1 ,

wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.

3 . The method of claim 1 ,

wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.

4 . The method of claim 1 ,

wherein the three or more affine classifiers comprise a multi-class classification layer.

5 . The method of claim 1 ,

wherein the three or more affine classifiers comprise a one-versus-all classification layer.

6 . The method of claim 1 ,

wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.

7 . The method of claim 1 ,

wherein the three or more affine classifiers comprise an odd number of affine classifiers, and

wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.

8 . The method of claim 1 , further comprising:

predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

9 . The method of claim 8 , further comprising:

wherein predicting the outcome for the testing data object is further based an ensemble decision rule.

10 . The method of claim 1 ,

wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and

wherein training the one or more neural networks is further based on the training data.

11 . A computer system for facilitating data classification, the computer system comprising:

a processor; and

a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising:

defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers;

including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;

replacing a final classification layer in one or more neural networks with the three or more affine classifiers;

training, based on a training data object or a testing data object, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;

predicting an outcome for a respective data object based on the ensemble of neural network-based classifiers which optimize the diversity metric;

displaying, on a device associated with a user, information related to the predicted outcome for the respective data object, the information comprising one or more of:

a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;

a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;

a size of the ensemble of neural network-based classifiers;

an approach associated with the respective data object, including whether the respective data object is associated with diversity;

a type of classifier used in a final classifier layer;

the predicted outcome based on a respective classifier; and

an overall ensemble result;

receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term;

predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and

increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.

12 . The computer system of claim 11 ,

wherein one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers.

13 . The computer system of claim 11 ,

wherein a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers.

14 . The computer system of claim 11 ,

wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.

15 . The method of claim 11 ,

wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal.

16 . The computer system of claim 11 ,

wherein the three or more affine classifiers comprise an odd number of affine classifiers, and

wherein the method further comprises generating a decision based on an ensemble decision rule which takes as input results that are outputted by the odd number of affine classifiers.

17 . The computer system of claim 11 , wherein the method further comprises:

predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric.

18 . The computer system of claim 11 ,

wherein defining the diversity metric is further based on pairwise angles between decision boundaries of the three or more affine classifiers for training data, and

wherein training the one or more neural networks is further based on the training data.

19 . A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:

defining a diversity metric based on pairwise angles between decision boundaries of three or more affine classifiers for training data;

including the diversity metric as a regularization term in a loss function optimization for designing a pair of mutually orthogonal affine classifiers of the three or more affine classifiers;

replacing a final classification layer in one or more neural networks with the three or more affine classifiers;

training, based on the training data, the one or more neural networks such that parameters of the one or more neural networks are consistent with parameters of the affine classifiers to obtain an ensemble of neural network-based classifiers which optimize the diversity metric;

predicting an outcome for a testing data object based on the obtained ensemble of neural-network based classifiers which optimize the diversity metric;

displaying, on a device associated with a user, information related to the predicted outcome for a respective data object, the information comprising one or more of:

a type of the respective data object, including whether the respective data object is clean or modified based on an adversarial attack;

a type associated with the adversarial attack, in response to the respective data object being modified based on an adversarial attack;

a size of the ensemble of neural network-based classifiers;

an approach associated with the respective data object, including whether the respective data object is associated with diversity;

a type of classifier used in a final classifier layer;

the predicted outcome based on a respective classifier; and

an overall ensemble result;

receiving, based on an interaction by the user with the displayed information, a request to change a configuration related to one or more of the type of the respective data object, the type associated with the adversarial attack, the size of the ensemble of neural network-based classifiers, the approach associated with the respective data object, the type of classifier used in the final classification layer, a selected classifier, and the regularization term;

predicting an updated outcome for the respective data object by updating the ensemble of neural network-based classifiers based on the changed configuration; and

increasing accuracy of predicting the updated outcome based on the updated ensemble of neural network-based classifiers.

20 . The non-transitory computer-readable storage medium of claim 19 ,

wherein the decision boundaries of the three or more affine classifiers are pairwise within a predetermined threshold of being mutually orthogonal,

wherein at least one of the following:

one backbone layer or one neural network kernel outputs an intermediate representation to the three or more affine classifiers; and

a plurality of backbone layers or a plurality of neural network kernels each output an intermediate representation to the three or more affine classifiers, and

wherein the three or more affine classifiers comprise at least one of a multi-class classification layer and a one-versus-all classification layer.

Assignments (8)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 31, 2025
From: XEROX CORPORATION
To: GENESEE VALLEY INNOVATIONS, LLC
Reel/Frame 073225/0116 →
SECOND LIEN NOTES PATENT SECURITY AGREEMENT Recorded Jul 2, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 071785/0550 →
FIRST LIEN NOTES PATENT SECURITY AGREEMENT Recorded Apr 11, 2025
From: XEROX CORPORATION
To: U.S. BANK TRUST COMPANY, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 070824/0001 →
SECURITY INTEREST Recorded Feb 13, 2024
From: XEROX CORPORATION
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 066741/0001 →
SECURITY INTEREST Recorded Nov 20, 2023
From: XEROX CORPORATION
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 065628/0019 →
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVAL OF US PATENTS 9356603, 10026651, 10626048 AND INCLUSION OF US PATENT 7167871 PREVIOUSLY RECORDED ON REEL 064038 FRAME 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 28, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064161/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2023
From: PALO ALTO RESEARCH CENTER INCORPORATED
To: XEROX CORPORATION
Reel/Frame 064038/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2022
From: BRITO, ALEJANDRO E.; RANE, SHANTANU
To: PALO ALTO RESEARCH CENTER INCORPORATED
Reel/Frame 061651/0065 →
Continuity (1)
Related Publication 20240095496A1 · Mar 21, 2024
References Cited (40)
US 10719301B1 · Dasgupta · 2020 [cited by examiner]
US 20110302111A1 · Chidlovskii · 2011 [cited by applicant]
US 20240273342A1 · Tawara · 2024 [cited by examiner]
Pang et al., Improving Adversarial Robustness via Promoting Ensemble Diversity, May 29, 2019, https://arxiv.org/pdf/1901.08846 (Year: 2019). [cited by examiner]
Hafiz et al., Digit Image Recognition Using an Ensemble of One-Versus-All Deep Network Classifiers, Oct. 31, 2020, https://arxiv.org/abs/2007.01192 (Year: 2020). [cited by examiner]
Martin et al., Toward Effective Moving Target Defense Against Adversarial AI, MILCOM 2021—2021 IEEE Military Communications Conference (MILCOM), San Diego, CA, USA, 2021, pp. 993-998, doi: 10.1109/MILCOM52596.2021.96529… [cited by examiner]
Yin et al., Learning to Diversify via Weighted Kernels for Classifier Ensemble, Jun. 4, 2014, https://arxiv.org/abs/1406.1167 (Year: 2014). [cited by examiner]
Tsoumakas, Grigorios, loannis Katakis, and loannis Vlahavas. “Effective and efficient multilabel classification in domains with large number of labels.” Proc. ECML/PKDD 2008 Workshop on Mining Multidimensional Data (MMD… [cited by applicant]
Similarity-based Multi-label Learning taken from https://arxiv.org/pdf/1710.10335v1 .pdf (Year: 2017). [cited by applicant]
Ml-knn: A Lazy Learning Approach toMulti-Label Learning taken from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.89.7148&rep=rep1&type=pdf (Year: 2007). [cited by applicant]
Score Normalization and Aggregation for Active Learning in Multi-label Classification taken from https://www.researchgate.net/ publication/222089355_Score_Normalization_and_Aggregation_for_Active_Learningjn_Multi-label_… [cited by applicant]
Ml-Clubas: A Multi Label Bug Classification Algorithm taken from https://file.scirp.org/Html/2-9301529_26203.htm (Year: 2012). [cited by applicant]
A Fuzzy Similarity-Based Approach for Multi-label Document Classification taken from https://www.researchgate.net/publication/232625068_A_Fuzzy_Similarity-Based_Approach_for_Multi-labeLDocument_Classification (Year: 200… [cited by applicant]
Madhieh Abbasi et al. “Robustness to Adversarial Examples Through and Ensemble of Specialists” Computer Vision and Systems Laboratory, Electrical and Computer Engineering Department Universite Laval, Quebec, Canada, Mar… [cited by applicant]
Naveed Akhtar et al. “Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey”, School of Computer Science and Software Engineering, University of Western Australia, Feb. 26, 2018. [cited by applicant]
Maria-Florina Balcan et al. “Agnostic Active Learning”, Jun. 13, 2008. [cited by applicant]
Alexander Bagnall et al. “Training Ensembles to Detect Adversarial Examples”, Dec. 11, 2017. [cited by applicant]
Battista Biggio et al. “Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning”, Jul. 19, 2018. [cited by applicant]
Anirban Chakraborty et al. “Adversarial Attacks and Defences: A Survey”, Sep. 28, 2018. [cited by applicant]
Nicholas Carlini et al. “Towards Evaluating the Robustness of Neural Networks”, Mar. 22, 2017. [cited by applicant]
Yoav Freund et al. “Selective Sampling Using the Query by Committee Algorithm”, Machine Learning, 28, 133-168 (1997), 1997 Kluwer Academic Publishers. [cited by applicant]
Ian J. Goodfellow et al. “Explaining and Harnessing Adversarial Examples”, Mar. 20, 2015. [cited by applicant]
Steve Hanneke “Rates of Convergence in Active Learning”, The Annals of Statistics, 2011, vol. 39, No. 1, 333-361, nstitute of Mathematical Statistics, 2011. [cited by applicant]
Haym Hirsh “Theoretical Underpinnings of Version Spaces”, Computer Science Department Rutgers University, tew Brunswick, NJ. [cited by applicant]
Ralf Herbrich et al. “Bayes Point Machines”, 2001. [cited by applicant]
Kebana “Robust Arrangements of Classifiers in Version Space”. [cited by applicant]
Qiang Liu “A Survey on Security Threats and Defensive Techniques of Machine Learning: A Data Driven View”, IEEE Feb. 2018. [cited by applicant]
Yu Li “On the Decision Boundary of Deep Neural Networks”, Jan. 1, 2019. [cited by applicant]
Tessa Lau “Programming by Demonstration Using Version Space Algebra”, Machine Learning, 53,111-156, 2003. [cited by applicant]
Tom M. Mitchell “Generalization as Search”, Computer Science Department, Rutgers University. [cited by applicant]
Seyed-Mohsen Moosavi-Dezfooli “DeepFool: a simple and accurate method to fool deep neural networks” Jul. 4, 2016. [cited by applicant]
Aamir Mustafa “Adversarial Defense by Restricting the Hidden Space of Deep Neural Networks” Jul. 28, 2019. [cited by applicant]
Nicolas Papernot “Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks” IEEE 2016, Mar. 14, 2016. [cited by applicant]
Shantanu Rane “A Version Space Perspective on Differentially Private Pool-Based Active Learning”, Downloaded Aug. 28, 2020. [cited by applicant]
Daniel Soudry “The Implicit Bias of Gradient Descent on Separable Data”, Journal of Machine Learning Research 19(2018), Dec. 28, 2018. [cited by applicant]
Uiawei Su “One Pixel Attach for Fooling Deep Neural Networks” Oct. 17, 2019. [cited by applicant]
Yichuan Tang “Deep Learning using Linear Support Vector Machines”, Feb. 21, 2015. [cited by applicant]
Simon Tong “Support Vector Machine Active Learning with Applications to Text Classification”, Journal of Machine .Learning Research (2001). [cited by applicant]
Florian Tramer “Ensemble Adversarial Training: Attacks and Defenses” ICLR 2018, Apr. 26, 2020. [cited by applicant]
Yan Zhou “Breaking Transferability of Adversarial Samples with Randomness”, Jun. 17, 2018. [cited by applicant]