Security action based on anomaly detection using AI model profiles and user profiles

1 . A system comprising:

a processor system; and

a memory that stores computer-executable instructions that are executable by the processor system to at least:

generate a model-session profile that represents semantic meanings of model sessions of an artificial intelligence (AI) model, a model session comprising a first subset of AI prompts received by the AI model during the model session and a first subset of AI responses generated by the AI model in response to the first subset of the AI prompts;

generate a model-response profile that represents semantic meanings of the AI responses;

generate user-session profiles that represent semantic meanings of user sessions of users with regard to the AI model, a user session comprising a second subset of the AI prompts that is received from a user at the AI model during the user session and a second subset of the AI responses that is generated by the AI model in response to the second subset of the AI prompts;

generate user-prompt profiles for the users, a user-prompt profile representing a semantic meaning of a third subset of the AI prompts that is received from a user at the AI model;

assign anomaly scores to an incoming AI prompt, the anomaly scores representing differences between the incoming AI prompt and the model-session profile, the model-response profile, at least a subset of the user-session profiles, and at least a subset of the user-prompt profiles; and

trigger execution of an instruction, which causes a security action to be performed with regard to the incoming AI prompt, as a result of an anomaly score, which is comprised in the anomaly scores, being greater than or equal to an anomaly score threshold.

2 . The system of claim 1 , wherein the anomaly scores correspond to the differences between the incoming AI prompt and a normal distribution of the model sessions represented by the model-session profile, a normal distribution of the AI responses represented by the model-response profile, a normal distribution of the user sessions represented by at least the subset of the user-session profiles, and a normal distribution of subsets of the AI prompts represented by at least the subset of the user-prompt profiles.

3 . The system of claim 2 , wherein the computer-executable instructions are executable by the processor system further to at least:

generate the normal distribution of the model sessions represented by the model-session profile, the normal distribution of the AI responses represented by the model-response profile, the normal distribution of the user sessions represented by at least the subset of the user-session profiles, and the normal distribution of subsets of the AI prompts represented by at least the subset of the user-prompt profiles using a one-class classifier.

4 . The system of claim 1 , wherein the computer-executable instructions are executable by the processor system to at least:

classify the incoming AI prompt as an anomalous AI prompt as the result of the anomaly score, which is comprised in the anomaly scores, being greater than or equal to the anomaly score threshold; and

trigger the execution of the instruction, which causes the security action to be performed with regard to the incoming AI prompt, as a result of the incoming AI prompt being classified as the anomalous AI prompt.

5 . The system of claim 1 , wherein the computer-executable instructions are executable by the processor system to at least:

generate user-response profiles for the users, a user-response profile representing a semantic meaning of a third subset of the AI responses that is generated by the AI model in response to a subset of the AI prompts that is received from a user at the AI model; and

trigger the execution of the instruction causes the security action to be performed with regard to the incoming AI prompt further as a result of a difference between the incoming AI prompt and a user-response profile in the user-response profiles being greater than or equal to a second difference threshold.

6 . The system of claim 1 , wherein the computer-executable instructions are executable by the processor system to at least:

generate a model-session feature vector, which is comprised in the model-session profile, by embedding at least a subset of the model sessions;

generate a model-response feature vector, which is comprised in the model-response profile, by embedding at least a subset of the AI responses;

generate user-session feature vectors, which are comprised in the user-session profiles, by embedding respective second subsets of the AI prompts that are received from the users at the AI model during the user sessions and respective second subsets of the AI responses that are generated by the AI model in response to the respective second subsets of the AI prompts;

generate user-prompt feature vectors, which are comprised in the user-prompt profiles, by embedding respective third subsets of the AI prompts that are received from the users at the AI model; and

trigger the execution of the instruction, which causes the security action to be performed with regard to the incoming AI prompt, as a result of a difference between an embedding that represents the incoming AI prompt and the model-session feature vector, the model-response feature vector, a first identified feature vector that represents at least a subset of the user-session feature vectors, or a second identified feature vector that represents at least a subset of the user-prompt feature vectors being greater than or equal to the difference threshold.

7 . The system of claim 6 , wherein the model-session feature vector is an embedding of a single model session from the model sessions; and

wherein the model-response feature vector is an embedding of a single AI response from the AI responses.

8 . The system of claim 6 , wherein the model-session feature vector is an embedding of an aggregation of the model sessions; and

wherein the model-response feature vector is an embedding of an aggregation of the AI responses.

9 . The system of claim 6 , wherein the first identified feature vector represents a single user-session feature vector in the user-session feature vectors; and

wherein the second identified feature vector represents a single user-prompt feature vector in the user-prompt feature vectors.

10 . The system of claim 6 , wherein the first identified feature vector represents an aggregation of the user-session feature vectors; and

wherein the second identified feature vector represents an aggregation of the user-prompt feature vectors.

11 . The system of claim 6 , wherein the computer-executable instructions are executable by the processor system to at least:

generate the model-session feature vector, the model-response feature vector, the user-session feature vectors, and the user-prompt feature vectors using a cross-lingual language model.

12 . A method implemented by a computing system, the method comprising:

generating a model-session feature vector in a model-session profile, which represents semantic meanings of model sessions of artificial intelligence (AI) models, by embedding at least a subset of the model sessions, a model session comprising a first subset of AI prompts that is received by an AI model during the model session and a first subset of AI responses that is generated by the AI model in response to the first subset of the AI prompts;

generating a model-response feature vector in a model-response profile, which represents semantic meanings of the AI responses that are generated by the AI models, by embedding at least a subset of the AI responses;

generating user-session feature vectors in user-session profiles, which represent semantic meanings of user sessions of users with regard to the AI models, by embedding the user sessions, which comprise respective second subsets of the AI prompts that are received among the AI models from the users during the user sessions and respective second subsets of the AI responses that are generated in response to the respective second subsets of the AI prompts;

generating user-prompt feature vectors in user-prompt profiles for the users by embedding respective third subsets of the AI prompts that are received among the AI models from the users;

assigning anomaly scores to an incoming AI prompt, the anomaly scores representing differences between an embedding that represents the incoming AI prompt and the model-session feature vector, the model-response feature vector, a first identified feature vector that represents at least a subset of the user-session feature vectors, and a second identified feature vector that represents at least a subset of the user-prompt feature vectors; and

triggering execution of an instruction, which causes a security action to be performed with regard to the incoming AI prompt, as a result of an anomaly score, which is comprised in the anomaly scores, being greater than or equal to an anomaly score threshold.

13 . The method of claim 12 , wherein at least the subset of the user-session feature vectors comprises a single user-session feature vector from the user-session feature vectors; and

wherein at least the subset of the user-prompt feature vectors comprises a single user-prompt feature vector from the user-prompt feature vectors.

14 . The method of claim 2 , wherein at least the subset of the user-session feature vectors comprises an aggregation of the user-session feature vectors; and

wherein at least the subset of the user-prompt feature vectors comprises an aggregation of the user-prompt feature vectors.

15 . The method of claim 12 , wherein the model-session feature vector represents a normal distribution of the model sessions represented by the model-session profile;

wherein the model-response feature vector represents a normal distribution of the AI responses represented by the model-response profile;

wherein the user-session feature vectors represent normal distributions of respective subsets of the user sessions; and

wherein the user-prompt feature vectors represent normal distributions of the respective third subsets of the AI prompts that are received among the AI models from the users.

16 . The method of claim 12 , further comprising:

generating user-response feature vectors in user-response profiles for the users by embedding respective fourth subsets of the AI prompts that are received among the AI models from the users; and

assigning an identified anomaly score to the incoming AI prompt, the identified anomaly score representing a difference between the embedding that represents the incoming AI prompt and a third identified feature vector that represents at least a subset of the user-response feature vectors;

wherein triggering the execution of the instruction causes the security action to be performed with regard to the incoming AI prompt further as a result of the identified anomaly score being greater than or equal to a second anomaly score threshold.

17 . The method of claim 12 , wherein the model-session feature vector is an embedding of an aggregation of the model sessions of the AI models.

18 . The method of claim 12 , wherein the model-response feature vector is an embedding of an aggregation of the AI responses that are generated by the AI models.

19 . The method of claim 12 , wherein the model-session feature vector, the model-response feature vector, the user-session feature vectors, and the user-prompt feature vectors are generated using a cross-lingual language model.

20 . A computer program product comprising a computer-readable storage medium having instructions recorded thereon for enabling a processor-based system to perform operations, the operations comprising:

generating a model-session profile that represents semantic meanings of model sessions of artificial intelligence (AI) models, a model session comprising a first subset of AI prompts received by an AI model during the model session and a first subset of AI responses generated by the AI model in response to the first subset of the AI prompts;

generating a model-response profile that represents semantic meanings of a plurality of AI responses that are generated by the AI models in response to a plurality of AI prompts;

generating a model-prompt profile that represents semantic meanings of the plurality of AI prompts;

generating user-session profiles that represent semantic meanings of user sessions of users with regard to the AI models, a user session comprising a second subset of the plurality of AI prompts that is received from a user at an AI model during the user session and a second subset of the plurality of AI responses that is generated by the AI model in response to the second subset of the plurality of AI prompts;

generating user-prompt profiles for the users, a user-prompt profile representing semantic meanings of respective AI prompts in a third subset of the plurality of AI prompts that is received among the AI models from a user;

generating user-response profiles for the users, a user-response profile representing semantic meanings of respective AI responses in a third subset of the plurality of AI responses that is generated by an AI model in response to a subset of the plurality of AI prompts that is received from a user at the AI model;

assigning anomaly scores to an incoming AI prompt, the anomaly scores representing differences between the incoming AI prompt and the model-session profile, the model-response profile, the model-prompt profile, at least a subset of the user-session profiles, at least a subset of the user-prompt profiles, and at least a subset of the user-response profiles; and

triggering execution of an instruction, which causes a security action to be performed with regard to an incoming AI prompt, as a result of an anomaly score, which is comprised in the anomaly scores, being greater than or equal to an anomaly score threshold.

21 . The computer program product of claim 20 , wherein the anomaly scores represent differences between the incoming AI prompt and a normal distribution of the model sessions represented by the model-session profile, a normal distribution of the plurality of AI responses represented by the model-response profile, a normal distribution of the plurality of AI prompts represented by the model-prompt profile, a normal distribution of the user sessions represented by at least the subset of the user-session profiles, a normal distribution of third subsets of the plurality of AI prompts represented by at least the subset of the user-prompt profiles, and a normal distribution of third subsets of the plurality of AI responses represented by at least the subset of the user-response profiles.