IP Library Granted Patent US 12,190,239
Granted Patent B2
US 12,190,239 · App. 17/429,789 · Granted Jan 7, 2025

Model building apparatus, model building method, computer program and recording medium

Inventors: Kazuya Kakizaki (Tokyo, JP); Kosuke Yoshida (Tokyo, JP)
Assignee: NEC CORPORATION
G06N3/08G06F18/2413G06F21/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,239
App. No.
17/429,789
Granted
Jan 7, 2025
Kind
B2
Abstract

A model building apparatus includes: a building unit that builds a generation model that outputs an adversarial example, which causes misclassification by a learned model, when a source sample is entered into the generation model; and a calculating unit that calculates a first evaluation value and a second evaluation value, wherein the first evaluation value is smaller as a difference is smaller between an actual visual feature of the adversarial example outputted from the generation model and a target visual feature of the adversarial example that are set to be different from a visual feature of the source sample, and the second evaluation value is smaller as there is a higher possibility that the learned model misclassifies the adversarial example outputted from the generation model. The building unit builds the generation model by updating the generation model such that an index value based on the first and second evaluation values is smaller.

Claims (30)

1. A model building apparatus comprising a controller,

the controller being programmed to:

build a generation model that outputs an adversarial example, which causes misclassification by a learned model, when a source sample is entered into the generation model; and

calculate a first evaluation value and a second evaluation value, wherein the first evaluation value is smaller as a difference is smaller between an actual visual feature of the adversarial example outputted from the generation model and a target visual feature of the adversarial example that are set to be different from a visual feature of the source sample, and the second evaluation value is smaller as there is a higher possibility that the learned model misclassifies the adversarial example outputted from the generation model,

wherein

the controller is programmed to build the generation model by updating the generation model such that an index value based on the first and second evaluation values is smaller,

the controller is further programmed to generate an approximate model for approximating the learned model, and

the controller is programmed to calculate the second evaluation value on the basis of a parameter for defining the approximate model.

2. The model building apparatus according to claim 1 , wherein

the controller is programmed to (i) calculate the second evaluation value on the basis of a parameter for defining the learned model when it is possible to obtain the parameter for defining the learned model, and (ii) calculate the second evaluation value on the basis of the parameter for defining the approximate model when it is impossible to obtain the parameter for defining the learned model.

3. The model building apparatus according to claim 1 , wherein

the controller is further programmed to generate the adversarial example by entering the source sample into the generation model built by the building unit.

4. The model building apparatus according to claim 1 , wherein

the controller is further programmed to evaluate the adversarial examples outputted from the generation model.

5. A model building method comprising:

building a generation model that outputs an adversarial example, which causes misclassification by a learned model, when a source sample is entered into the generation model; and

calculating a first evaluation value and a second evaluation value, wherein the first evaluation value is smaller as a difference is smaller between an actual visual feature of the adversarial example outputted from the generation model and a target visual feature of the adversarial example that are set to be different from a visual feature of the source sample, and the second evaluation value is smaller as there is a higher possibility that the learned model misclassifies the adversarial example outputted from the generation model, wherein

building includes building the generation model by updating the generation model such that an index value based on the first and second evaluation values is smaller,

the method further comprising:

generating an approximate model for approximating the learned model, and

calculating the second evaluation value on the basis of a parameter for defining the approximate model.

6. A non-transitory recording medium on which a computer program that allows a computer to execute a model building method comprising:

building a generation model that outputs an adversarial example, which causes misclassification by a learned model, when a source sample is entered into the generation model; and

calculating a first evaluation value and a second evaluation value, wherein the first evaluation value is smaller as a difference is smaller between an actual visual feature of the adversarial example outputted from the generation model and a target visual feature of the adversarial example that are set to be different from a visual feature of the source sample, and the second evaluation value is smaller as there is a higher possibility that the learned model misclassifies the adversarial example outputted from the generation model, wherein

building includes building the generation model by updating the generation model such that an index value based on the first and second evaluation values is smaller,

the method further comprising:

generating an approximate model for approximating the learned model, and

calculating the second evaluation value on the basis of a parameter for defining the approximate model.

7. The model building apparatus according to claim 1 , wherein the parameter includes at least one of variables and coefficients used to determine at least one of behavior, structure, and properties of an approximation model, and the parameter includes at least one of weighting and biases, hyperparameters, and decision thresholds.

8. The non-transitory recording medium according to claim 6 , wherein the parameter includes at least one of variables and coefficients used to determine at least one of behavior, structure, and properties of an approximation model, and the parameter includes at least one of weighting and biases, hyperparameters, and decision thresholds.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2022
From: KAKIZAKI, KAZUYA; YOSHIDA, KOSUKE
To: NEC CORPORATION
Reel/Frame 061250/0654 →
Continuity (1)
Related Publication 20220121991A1 · Apr 21, 2022
References Cited (27)
US 9875440B1 · Commons · 2018 [cited by examiner]
US 10254641B2 · Mailfert · 2019 [cited by examiner]
US 10460235B1 · Truong · 2019 [cited by examiner]
US 10909681B2 · Hsiao · 2021 [cited by examiner]
US 11423263B2 · Kobayashi · 2022 [cited by examiner]
US 11580383B2 · Ishii · 2023 [cited by examiner]
US 11610132B2 · Hewage · 2023 [cited by examiner]
US 20090141969A1 · Yu · 2009 [cited by examiner]
US 20140074762A1 · Campbell · 2014 [cited by examiner]
US 20160096270A1 · Ibarz Gabardos · 2016 [cited by examiner]
US 20170278135A1 · Majumdar · 2017 [cited by examiner]
US 20170316281A1 · Criminisi · 2017 [cited by examiner]
US 20190005386A1 · Chen · 2019 [cited by examiner]
US 20190130216A1 · Tomioka · 2019 [cited by examiner]
US 20190244348A1 · Buckler · 2019 [cited by examiner]
US 20200234162A1 · Jayaraman · 2020 [cited by examiner]
US 20200251213A1 · Tran · 2020 [cited by examiner]
CN 110574120A · 2019 [cited by examiner]
CN 113672197B · 2024 [cited by examiner]
WO WO2016207875A1 · 2016 [cited by examiner]
WO WO2019079182A1 · 2019 [cited by examiner]
WO WO2019207770A1 · 2019 [cited by examiner]
International Search Report for PCT Application No. PCT/JP2019/004822, mailed on Apr. 23, 2019. [cited by applicant]
Nicholas Carlini et al., “Towards Evaluating the Robustness of Neural Networks”, IEEE Symposium on Security and Privacy (SPs), 2017, pp. 1-19. [cited by applicant]
Yunjey Choi et al., “StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 1-15. [cited by applicant]
Shumeet Baluja et al., “Adversarial Transformation Networks Learning to Generate Adversarial examples”, [online], Mar. 28, 2017 [retrieved on Apr. 15, 2019], Retrieved from the Internet: <URL: https://arxiv.org/abs/1703… [cited by applicant]
Chaowei Xiao et al., “Generating Adversarial Examples with Adversarial Networks ”, Proceedings of the Twenty SeSeventh International Joint Conference on Artificial Intelligence(IJCAI-18). International Joint Conferences… [cited by applicant]
Cited By (1)
US 12,455,993