IP Library › Granted Patent US 12,205,349
Granted Patent B2
US 12,205,349 · App. 17/698,556 · Granted Jan 21, 2025

System and method for improving robustness of pretrained systems in deep neural networks utilizing randomization and sample rejection

Inventors: Fatemeh Sheikholeslami (Pittsburgh, PA); Wan-Yi Lin (Wexford, PA); Jan Hendrik Metzen (Boeblingen, DE); Huan Zhang (Pittsburgh, PA); Jeremy Kolter (Pittsburgh, PA)
Assignees: Robert Bosch GmbH; Carnegie Mellon University
G06V10/764G06T5/70G06V10/84G06T2207/20076G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,205,349
App. No.
17/698,556
Granted
Jan 21, 2025
Kind
B2
Abstract

A system includes a machine-learning network. The network includes an input interface configured to receive input data from a sensor. The processor is programmed to receive the input data, generate a perturbed input data set utilize the input data, wherein the perturbed input data set includes perturbations of the input data, denoise the perturbed input data set utilizing a denoiser, wherein the denoiser is configured to generate a denoised data set, send the denoised data set to both a pre-trained classifier and a rejector, wherein the pre-trained classifier is configured to classify the denoised data set and the rejector is configured to reject a classification of the denoised data set, train, utilizing the denoised input data set, the a rejector to achieve a trained rejector, and in response to obtaining the trained rejector, output an abstain classification associated with the input data, wherein the abstain classification is ignored for classification.

Claims (38)

1. A computer-implemented method for training a machine-learning network, comprising:

receiving an input data from a sensor, wherein the input data is indicative of image, radar, sonar, or sound information;

generating a perturbed input data set utilizing the input data, wherein the perturbed input data set includes perturbed data;

denoising, via a pre-trained denoiser, the perturbed input data set to generate a denoised data set;

training the machine-learning network utilizing the denoised data set, wherein the machine-learning network is configured to reject the denoised data set, utilizing a rejector, when a classification probability falls below a classification threshold, wherein the classification threshold is associated with classification the denoised data set, wherein the rejector is trained to discriminate between correctly classified inputs and misclassified denoised inputs; and

in response to the classification probability falling below the classification threshold, outputting an abstain classification associated with the input data, wherein the abstain classification is ignored for classifying.

2. The computer-implemented method of claim 1 , wherein the method includes outputting a final classification that has a majority vote in comparison to the abstain classification when classifying.

3. The computer-implemented method of claim 1 , wherein the method includes utilizing a classifier that includes a per-class rejector that is a binary detector for original classes associated with the input data.

4. The computer-implemented method of claim 1 , wherein the method includes utilizing a pre-trained classifier configured to classify the input data and a rejector configured to reject classification of the denoised data set.

5. The computer-implemented method of claim 1 , wherein the method includes utilizing multiple iterations of the denoised data set.

6. The computer-implemented method of claim 1 , wherein method further includes utilizing a classifier configured to classify the input data.

7. A system including a machine-learning network, comprising:

an input interface configured to receive input data from a sensor, wherein the sensor includes a camera, a radar, a sonar, or a microphone;

a processor, in communication with the input interface, wherein the processor is programmed to:

receive the input data, wherein the input data is indicative of image, radar, sonar, or sound information;

generate a perturbed input data set utilizing the input data, wherein the perturbed input data set includes perturbations of the input data;

denoise the perturbed input data set utilizing a denoiser, wherein the denoiser is configured to generate a denoised data set;

send the denoised data set to both a pre-trained classifier and a rejector, wherein the pre-trained classifier is configured to classify the denoised data set and the rejector is configured to reject a classification of the denoised data set, wherein the rejector is configured to be trained via discriminating correctly classified images and misclassified randomly perturbed images;

train, utilizing the denoised input data set, the a rejector to achieve a trained rejector; and

in response to obtaining the trained rejector, output an abstain classification associated with the input data, wherein the abstain classification is ignored for classifying.

8. The system of claim 7 , wherein the denoiser is a pretrained denoiser.

9. The system of claim 7 , wherein the processor is further programmed to output a final classification associated with the input data.

10. The system of claim 7 , wherein training the rejector includes utilizing Monte Carlo sampling associated with the input data.

11. The system of claim 7 , wherein the rejector includes a shared backbone configured to be parameterized.

12. The system of claim 7 , wherein the denoiser is configured to remove or mitigate Gaussian noise added to the input.

13. A computer-program product storing instructions on a non-transitory storage media which, when executed by a computer, cause the computer to:

receive input data from a sensor, wherein the input data is indicative of image, radar, sonar, or sound information;

generate a perturbed input data set utilizing the input data, wherein the perturbed input data set includes perturbations of the input data;

denoise the input data set utilizing a pretrained denoiser, wherein the pretrained denoiser is configured to generate a denoised data set;

classify or reject the denoised data set utilizing a pretrained classifier and a rejector;

train the rejector utilizing the denoised data set, wherein the rejector is configured to reject the denoised data set when a classification probability falls below classification threshold, wherein the classification threshold is associated with classification of the denoised data set, wherein the rejector is trained to discriminate between correctly classified inputs and misclassified denoised inputs; and

in response to the classification probability falling below the classification threshold, output an abstain classification associated with the input data, wherein the abstain classification ignores a classification.

14. The computer-program product of claim 13 , wherein the input data includes an image received from a camera in communication with the computer.

15. The computer-program product of claim 13 , wherein instructions further cause the computer to train the rejector, wherein training includes an upper bound and lower bound defined utilizing Monte Carlo sampling.

16. The computer-program product of claim 13 , wherein the instructions further cause the computer to classify or reject the denoised data set utilizing the pretrained classifier for multiple iterations.

17. The computer-program product of claim 13 , wherein the pretrained denoiser is configured to be trained and mitigate Gaussian noise or remove Gaussian noise.

18. The computer-program product of claim 13 , wherein parameters of the rejector are configured to learn via training the rejector to discriminate a correct classification versus a misclassification.

19. The computer-program product of claim 13 , wherein the input data includes sound information obtained from a microphone.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2022
From: ZHANG, HUAN
To: CARNEGIE MELLON UNIVERSITY
Reel/Frame 060756/0214 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2022
From: SHEIKHOLESLAMI, FATEMEH; LIN, WAN-YI; METZEN, JAN HENDRIK; KOLTER, JEREMY
To: ROBERT BOSCH GMBH
Reel/Frame 061133/0062 →
Continuity (1)
Related Publication 20230298315A1 · Sep 21, 2023
References Cited (54)
US 9171265B1 · Lathia · 2015 [cited by examiner]
US 20160093048A1 · Cheng · 2016 [cited by examiner]
US 20160202074A1 · Woodard · 2016 [cited by examiner]
US 20180122048A1 · Wang · 2018 [cited by examiner]
US 20180165554A1 · Zhang · 2018 [cited by examiner]
US 20190005360A1 · Heide · 2019 [cited by examiner]
US 20190156524A1 · Park · 2019 [cited by examiner]
US 20200116879A1 · Pavlovski · 2020 [cited by examiner]
US 20200241096A1 · Bustin · 2020 [cited by examiner]
US 20200394506A1 · Louizos · 2020 [cited by examiner]
US 20210089842A1 · Rosenfeld · 2021 [cited by examiner]
US 20220027709A1 · Honkala · 2022 [cited by examiner]
US 20220036741A1 · Ozturk · 2022 [cited by examiner]
US 20220051093A1 · Skaljak · 2022 [cited by examiner]
US 20220130084A1 · Litwiller · 2022 [cited by examiner]
US 20220138949A1 · Enzmann · 2022 [cited by examiner]
US 20220180723A1 · Cheng · 2022 [cited by examiner]
US 20220219691A1 · Maleki · 2022 [cited by examiner]
US 20220318376A1 · Cho · 2022 [cited by examiner]
US 20220351496A1 · Dou · 2022 [cited by examiner]
US 20220383502A1 · Teder · 2022 [cited by examiner]
US 20230162341A1 · Yu · 2023 [cited by examiner]
US 20230214288A1 · Moctezuma · 2023 [cited by examiner]
US 20230298315A1 · Sheikholeslami · 2023 [cited by examiner]
Madry et al., “Towards Deep Learning Models Resistant to Adversarial Attacks”, arXiv:1706.06083v1 [stat.ML] Jun. 19, 2017, 22 Pages. [cited by applicant]
Raghunathan et al., “Certified Defenses Against Adversarial Examples”, Published as a conference paper at ICLR 2018, 15 Pages. [cited by applicant]
Gowal et al., “On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models”, arXiv:1810.12715v1 [cs.LG] Oct. 30, 2018, 12 Pages. [cited by applicant]
Wong et al., “Scaling provable adversarial defenses”, 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montreal, Canada, 10 Pages. [cited by applicant]
Zhang et al., “Towards Stable and Efficient Training of Verifiably Robust Neural Networks”, arXiv:1906.06316v2 [cs.LG] Nov. 27, 2019, 25 Pages. [cited by applicant]
Cohen et al., “Certified Adversarial Robustness via Randomized Smoothing”, Proceedings of the 36 th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019, 11 Pages. [cited by applicant]
Laidlaw et al., “Playing it Safe: Adversarial Robustness with an Abstain Option”, arXiv:1911.11253v1 [cs.LG] Nov. 25, 2019, 14 Pages. [cited by applicant]
Tramèr et al., “On Adaptive Attacks to Adversarial Example Defenses”, arXiv:2002.08347v2 [cs.LG] Oct. 23, 2020, 44 Pages. [cited by applicant]
Salman et al., “Denoised Smoothing: A Provable Defense for Pretrained Classifiers”, 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada, 13 Pages. [cited by applicant]
Sheikholeslami et al., “Provably Robust Classification of Adversarial Examples With Detection”, Published as a conference paper at ICLR 2021, 16 Pages. [cited by applicant]
Athalya et al., “Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples”, Proceedings of the 35 th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2… [cited by applicant]
Biggio et al., “Evasion Attacks against Machine Learning at Test Time”, ECML PKDD 2013, Part III, LNAI 8190, 2013, pp. 387-402. [cited by applicant]
Carlini et al., “Towards Evaluating the Robustness of Neural Networks”, 2017 IEEE Symposium on Security and Privacy, pp. 39-57. [cited by applicant]
Fischer et al., “Scalable Certified Segmentation via Randomized Smoothing”, Proceedings of the 38 th International Conference on Machine Learning, PMLR 139, 2021, 12 Pages. [cited by applicant]
Geifman et al., “SelectiveNet: A Deep Neural Network with an Integrated Reject Option”, Proceedings of the 36 th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019, 9 Pages. [cited by applicant]
Goodfellow et al., “Explaining and Harnessing Adversarial Examples”, arXiv:1412.6572v1 [stat.ML] Dec. 20, 2014, 10 Pages. [cited by applicant]
Kumar et al., “Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness”, Proceedings of the 37 th International Conference on Machine Learning, Online, PMLR 119, 2020, 13 Pages. [cited by applicant]
Atlidakis et al., “Certified Robustness to Adversarial Examples with Differential Privacy”, 2019 IEEE Symposium of Security and Privacy, pp. 656-673. [cited by applicant]
Levine et al., “Wasserstein Smoothing: Certified Robustness against Wasserstein Adversarial Attacks”, Proceedings of the 23rdInternational Conference on Artificial Intelligence and Statistics (AISTATS) 2020, Palermo, It… [cited by applicant]
Li et al., “Second-Order Adversarial Attack and Certifiable Robustness”, arXiv: 1809.03113v1 [cs.LG] Sep. 10, 2018, 15 Pages. [cited by applicant]
Ziyin et al., “Deep Gamblers: Learning to Abstain with Portfolio Theory”, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada, 17 Pages. [cited by applicant]
Stutz et al., “Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks”, Proceedings of the 37 th International Conference on Machine Learning, Vienna, Austria, PMLR 119, 2020, 43 Pages. [cited by applicant]
Szegedy et al., “Intriguing properties of neural networks”, arXiv:1312.6199v1 [cs.CV] Dec. 21, 2013, 9 Pages. [cited by applicant]
Tai et al., “MemNet: A Persistent Memory Network for Image Restoration”, Proceedings of the IEEE international conference on computer vision 2017, pp. 4539-4547. [cited by applicant]
Tramèr., “Detecting Adversarial Examples Is (Nearly) as Hard as Classifying Them”, arXiv:2107.11630v1 [cs.LG] Jul. 24, 2021, 6 Pages. [cited by applicant]
Tramèr et al., “Ensemble Adversarial Training: Attacks and Defenses”, arXiv:1705.07204v1 [stat.ML] May 19, 2017, 14 Pages. [cited by applicant]
Uesato et al., “Adversarial Risk and the Dangers of Evaluating Against Weak Attacks”, Proceedings of the 35 th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018, 10 Pages. [cited by applicant]
Kolter et al., “Provable defenses against adversarial examples via the convex outer adversarial polytope”, arXiv:1711.00851v1 [cs.LG] Nov. 2, 2017, 24 pages. [cited by applicant]
Yin et al., “Gat: Generative Adversarial Training for Adversarial Example Detection and Robust Classification”, Published as a conference paper at ICLR 2020, 26 Pages. [cited by applicant]
Zhang et al., “Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising” arXiv:1608.03981v1 [cs.CV] Aug. 13, 2016, 13 Pages. [cited by applicant]