Model performance evaluation

1 . A method for improving accuracy of model anti-attack performance evaluation, comprising:

generating a first sample set comprising a plurality of attack intention samples by extracting the plurality of attack intention samples from a first plurality of test samples, wherein each of the plurality of attack intention samples indicates an attack objective of an attacker;

generating a second sample set comprising a plurality of attack pattern samples by extracting the plurality of attack pattern samples from the first plurality of test samples, wherein each of the plurality of attack pattern samples indicates a model attack technique;

generating a second plurality of test samples by an electronic device, wherein each of the second plurality of test samples is generated based on the electronic device extracting one of the plurality of attack intention samples from the first sample set, extracting one of the plurality of attack pattern samples from the second sample set, and combining the extracted attack intention sample with the extracted attack pattern sample, wherein the second plurality of test samples comprises an increased quantity and diversity of test samples as compared to the first plurality of test samples;

executing the second plurality of test samples by the electronic device using a target model to generate output data and identifying, based on the output data, a subset of the second plurality of test samples for which the target model output failure results in a process of executing the second plurality of test samples, wherein a failure result indicates that a test sample successfully attacked the target model; and

evaluating an anti-attack performance of the target model by the electronic device based on determining a percentage of the second plurality of test samples belonging to the subset of the second plurality of test samples.

2 . The method of claim 1 , wherein generating the first sample set and generating the second sample set comprises:

obtaining, for a first attack sample in the first plurality of test samples, first annotation information of the first attack sample, wherein the first annotation information indicates a first attack intention and a first attack pattern of the first attack sample;

determining a first attack intention sample and a first attack pattern sample from the first attack sample based on the first annotation information;

adding the first attack intention sample to the first sample set; and

adding the first attack pattern sample to the second sample set.

3 . The method of claim 2 , further comprising:

decomposing an attack intention of a second attack intention sample in the first sample set to determine a plurality of attack sub-intentions;

generating a plurality of attack intention samples respectively corresponding to the plurality of attack sub-intentions; and

adding the plurality of generated attack intention samples to the first sample set.

4 . The method of claim 1 , wherein evaluating the anti-attack performance of the target model comprises:

determining a first number of test samples in the second plurality of test samples that are executed by the target model;

determining, based on the output data, a second number of test samples for which the target model output the failure results in the process of executing the first number of test samples; and

determining a first attack success rate for the target model based on a ratio of the first number to the second number, wherein the first attack success rate indicates a protection capability of the target model against a generic attack.

5 . The method of claim 4 , wherein evaluating the anti-attack performance of the target model further comprises:

determining a third number of test samples matching a function of the target model among the first number of test samples; and

determining a second attack success rate for the target model based on a ratio of the third number to the second number, wherein the second attack success rate indicates a protection capability of the target model against an attack matching the function of the target model.

6 . The method of claim 1 , wherein generating the second plurality of test samples comprises:

determining, based on the first sample set, the one or more attack intention samples related to the target model;

determining, based on the second sample set, the one or more attack pattern samples related to the target model; and

combining an attack intention sample in the one or more attack intention samples and an attack pattern sample in the one or more attack pattern samples as the test sample in the second plurality of test samples.

7 . The method of claim 1 , further comprising:

generating a second test sample set based on one or more attack intention samples in the first sample set; and

determining a third attack success rate for the target model based on output data of the target model executing the second test sample set.

8 . The method of claim 7 , further comprising:

determining a proportional relationship among a first attack success rate indicating a protection capability of the target model against a generic attack, a second attack success rate indicating a protection capability of the target model against an attack matching a function of the target model, and the third attack success rate; and

determining a protection capability of the target model based on the proportional relationship.

9 . An electronic device, comprising:

at least one processor; and

at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform acts comprising:

generating a first sample set comprising a plurality of attack intention samples by extracting the plurality of attack intention samples from a first plurality of test samples, wherein each of the plurality of attack intention samples indicates an attack objective of an attacker;

generating a second sample set comprising a plurality of attack pattern samples by extracting the plurality of attack pattern samples from the first plurality of test samples, wherein each of the plurality of attack pattern samples indicates a model attack technique;

generating a second plurality of test samples by an electronic device, wherein each of the second plurality of test samples is generated based on the electronic device extracting one of the plurality of attack intention samples from the first sample set, extracting one of the plurality of attack pattern samples from the second sample set, and combining the extracted attack intention sample with the extracted attack pattern sample, wherein the second plurality of test samples comprises an increased quantity and diversity of test samples as compared to the first plurality of test samples;

executing the second plurality of test samples by the electronic device using a target model to generate output data and identifying, based on the output data, a subset of the second plurality of test samples for which the target model output failure results in a process of executing the second plurality of test samples, wherein a failure result indicates that a test sample successfully attacked the target model; and

evaluating an anti-attack performance of the target model by the electronic device based on determining a percentage of the second plurality of test samples belonging to the subset of the second plurality of test samples.

10 . The electronic device of claim 9 , wherein generating the first sample set and generating the second sample set comprises:

obtaining, for a first attack sample in the first plurality of test samples, first annotation information of the first attack sample, wherein the first annotation information indicates a first attack intention and a first attack pattern of the first attack sample;

determining a first attack intention sample and a first attack pattern sample from the first attack sample based on the first annotation information;

adding the first attack intention sample to the first sample set; and

adding the first attack pattern sample to the second sample set.

11 . The electronic device of claim 10 , the acts further comprising:

decomposing an attack intention of a second attack intention sample in the first sample set to determine a plurality of attack sub-intentions;

generating a plurality of attack intention samples respectively corresponding to the plurality of attack sub-intentions; and

adding the plurality of generated attack intention samples to the first sample set.

12 . The electronic device of claim 9 , wherein evaluating the anti-attack performance of the target model comprises:

determining a first number of test samples in the set second plurality of test samples that are executed by the target model;

determining, based on the output data, a second number of test samples for which the target model output the failure results in the process of executing the first number of test samples; and

determining a first attack success rate for the target model based on a ratio of the first number to the second number, wherein the first attack success rate indicates a protection capability of the target model against a generic attack.

13 . The electronic device of claim 12 , wherein evaluating the anti-attack performance of the target model further comprises:

determining a third number of test samples matching a function of the target model among the first number of test samples; and

determining a second attack success rate for the target model based on a ratio of the third number to the second number, wherein the second attack success rate indicates a protection capability of the target model against an attack matching the function of the target model.

14 . The electronic device of claim 9 , wherein generating the second plurality of test samples comprises:

determining, based on the first sample set, the one or more attack intention samples related to the target model;

determining, based on the second sample set, the one or more attack pattern samples related to the target model; and

combining an attack intention sample in the one or more attack intention samples and an attack pattern sample in the one or more attack pattern samples as the test sample in the second plurality of test samples.

15 . The electronic device of claim 9 , the acts further comprising:

generating a second test sample set based on one or more attack intention samples in the first sample set; and

determining a third attack success rate for the target model based on output data of the target model executing the second test sample set.

16 . The electronic device of claim 15 , the acts further comprising:

determining a proportional relationship among a first attack success rate indicating a protection capability of the target model against a generic attack, a second attack success rate indicating a protection capability of the target model against an attack matching a function of the target model, and the third attack success rate; and

determining a protection capability of the target model based on the proportional relationship.

17 . A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement acts comprising:

generating a first sample set comprising a plurality of attack intention samples by extracting the plurality of attack intention samples from a first plurality of test samples, wherein each of the plurality of attack intention samples indicates an attack objective of an attacker;

generating a second sample set comprising a plurality of attack pattern samples by extracting the plurality of attack pattern samples from the first plurality of test samples, wherein each of the plurality of attack pattern samples indicates a model attack technique;

generating a second plurality of test samples by an electronic device, wherein each of the second plurality of test samples is generated based on the electronic device extracting one of the plurality of attack intention samples from the first sample set, extracting one of the plurality of attack pattern samples from the second sample set, and combining the extracted attack intention sample with the extracted attack pattern sample, wherein the second plurality of test samples comprises an increased quantity and diversity of test samples as compared to the first plurality of test samples;

executing the second plurality of test samples by the electronic device using a target model to generate output data and identifying, based on the output data, a subset of the second plurality of test samples for which the target model output failure results in a process of executing the second plurality of test samples, wherein a failure result indicates that a test sample successfully attacked the target model; and

evaluating an anti-attack performance of the target model by the electronic device based on determining a percentage of the second plurality of test samples belonging to the subset of the second plurality of test samples.