Detecting possible attacks on artificial intelligence models using strengths of causal relationships in training data
Methods and systems for managing artificial intelligence (AI) models are disclosed. To manage AI models, an instance of an AI model may not be re-trained using training data determined to be potentially poisoned. By doing so, malicious attacks intending to influence the AI model using poisoned training data may be prevented. To do so, a first level of strength of a first causal relationship present in historical training data may be compared to a second level of strength of a second causal relationship present in a candidate training data set. The first level of strength and the second level of strength may be expected to be similar within a threshold. If a difference between the first level of strength and the second level of strength is not within the threshold, the candidate training data may be treated as including poisoned training data.
1. A method of managing an artificial intelligence (AI) model, performed by one or more hardware processors, the method comprising:
obtaining, by the one or more hardware processors through a communication system, a candidate training data set usable to update an instance of the AI model;
identifying a historical training data set, the historical training data set being obtained prior to the candidate training data set and the historical training data set already having been considered as trustworthy;
obtaining a quantification of a difference between levels of strengths of similar causal relationships in the candidate training data set and the historical training data set, wherein a level of strength of the levels of strengths is based on a goodness of fit of a function to a portion of the historical training data set, the function defining a causal relationship of the similar causal relationships;
making a determination regarding whether the quantification is within a threshold for the quantification;
in a first instance of the determination in which the quantification is within the threshold: obtaining a second instance of the AI model using at least the candidate training data set; and
in a second instance of the determination in which the quantification is not within the threshold: treating the candidate training data set as comprising poisoned training data.
2. The method of claim 1 , wherein obtaining the quantification comprises:
identifying a first causal relationship of the similar causal relationships in the historical training data set;
identifying a first level of strength of the first causal relationship;
identifying a second causal relationship of the similar causal relationships in the candidate training data set; and
identifying a second level of strength of the second causal relationship.
3. The method of claim 2 , wherein the first causal relationship and the second causal relationship relate same features and same labels.
4. The method of claim 3 , wherein the first causal relationship is based on a first feature present in the historical training data set and a first label present in the historical training data set.
5. The method of claim 4 , wherein the second causal relationship is based on a second feature present in the historical training data set and a second label present in the historical training data set.
6. The method of claim 5 , wherein the first feature is based on first measurements of a quantity during a first period of time and the second feature is based on second measurements of the quantity during a second period of time, the first period of time being prior to the second period of time.
7. The method of claim 6 , wherein the first label is based on third measurements of a second quantity during the first period of time and the second label is based on fourth measurements of the second quantity during the second period of time.
8. The method of claim 1 , wherein the quantification of the difference is based on the goodness of fit and a second goodness of a second fit of a second function to a second portion of the candidate training data set.
9. The method of claim 1 , wherein the threshold is based on a level of tolerance for use of poisoned training data in the AI model.
10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a hardware processor, cause the hardware processor to perform operations for managing an artificial intelligence (AI) model, the operations comprising:
obtaining, through a communication system, a candidate training data set usable to update an instance of the AI model;
identifying a historical training data set, the historical training data set being obtained prior to the candidate training data set and the historical training data set already having been considered as trustworthy;
obtaining a quantification of a difference between levels of strengths of similar causal relationships in the candidate training data set and the historical training data set, wherein a level of strength of the levels of strengths is based on a goodness of fit of a function to a portion of the historical training data set, the function defining a causal relationship of the similar causal relationships;
making a determination regarding whether the quantification is within a threshold for the quantification;
in a first instance of the determination in which the quantification is within the threshold: obtaining a second instance of the AI model using at least the candidate training data set; and
in a second instance of the determination in which the quantification is not within the threshold: treating the candidate training data set as comprising poisoned training data.
11. The non-transitory machine-readable medium of claim 10 , wherein obtaining the quantification comprises:
identifying a first causal relationship of the similar causal relationships in the historical training data set;
identifying a first level of strength of the first causal relationship;
identifying a second causal relationship of the similar causal relationships in the candidate training data set; and
identifying a second level of strength of the second causal relationship.
12. The non-transitory machine-readable medium of claim 11 , wherein the first causal relationship and the second causal relationship relate same features and same labels.
13. The non-transitory machine-readable medium of claim 12 , wherein the first causal relationship is based on a first feature present in the historical training data set and a first label present in the historical training data set.
14. The non-transitory machine-readable medium of claim 13 , wherein the second causal relationship is based on a second feature present in the historical training data set and a second label present in the historical training data set.
15. The non-transitory machine-readable medium of claim 14 , wherein the first feature is based on first measurements of a quantity during a first period of time and the second feature is based on second measurements of the quantity during a second period of time, the first period of time being prior to the second period of time.
16. The non-transitory machine-readable medium of claim 15 , wherein the first label is based on third measurements of a second quantity during the first period of time and the second label is based on fourth measurements of the second quantity during the second period of time.
17. A data processing system, comprising:
a hardware processor; and
a memory coupled to the hardware processor to store instructions, which when executed by the hardware processor, cause the hardware processor to perform operations for managing an artificial intelligence (AI) model, the operations comprising:
obtaining, through a communication system, a candidate training data set usable to update an instance of the AI model;
identifying a historical training data set, the historical training data set being obtained prior to the candidate training data set and the historical training data set already having been considered as trustworthy;
obtaining a quantification of a difference between levels of strengths of similar causal relationships in the candidate training data set and the historical training data set, wherein a level of strength of the levels of strengths is based on a goodness of fit of a function to a portion of the historical training data set, the function defining a causal relationship of the similar causal relationships;
making a determination regarding whether the quantification is within a threshold for the quantification;
in a first instance of the determination in which the quantification is within the threshold: obtaining a second instance of the AI model using at least the candidate training data set; and
in a second instance of the determination in which the quantification is not within the threshold: treating the candidate training data set as comprising poisoned training data.
18. The data processing system of claim 17 , wherein obtaining the quantification comprises:
identifying a first causal relationship of the similar causal relationships in the historical training data set;
identifying a first level of strength of the first causal relationship;
identifying a second causal relationship of the similar causal relationships in the candidate training data set; and
identifying a second level of strength of the second causal relationship.
19. The data processing system of claim 18 , wherein the first causal relationship and the second causal relationship relate same features and same labels.
20. The data processing system of claim 19 , wherein the first causal relationship is based on a first feature present in the historical training data set and a first label present in the historical training data set.