Agentic-based approach for reducing false positives in machine learning model output classifications
Techniques are provided for adjudicating suspected-malicious samples using in-context gating and an appeals stage to reduce false positives. A first-stage classifier generates a benign-or-malicious verdict. When malicious is indicated, a context-gating model determines whether the sample is in-context for a target application domain. If out-of-context, the initial verdict is finalized. If in-context, contextual information is assembled by retrieving and condensing items from a knowledge base via similarity search. The sample and the contextual information are then supplied to an appeals model, distinct from the first-stage classifier and the context-gating model, which issues a context-sensitive verdict. The appeals verdict is emitted as the final decision. This architecture separates fast screening from domain-aware reasoning, enabling dynamic incorporation of deployment-specific knowledge without retraining the first-stage classifier, improving accuracy, and providing a principled fallback when the sample is out-of-scope for the application domain.
1 . A computer-implemented method for adjudicating suspected-malicious samples with context gating, the method comprising:
receiving, by one or more computing devices, a sample;
generating, by a first-stage classifier, a first verdict classifying the sample as malicious;
invoking a context-gating model distinct from the first-stage classifier and an appeals model, the context-gating model evaluating whether the sample is in-context for a specific application domain by computing an embedding for the sample and an embedding for context descriptors, comparing a distance between the embeddings against a threshold, and outputting an in-context or out-of-context label;
emitting, as a final verdict, the first verdict if the context-gating model determines the sample is out-of-context;
assembling, if the context-gating model determines the sample is in-context, contextual information by retrieving, via a similarity search over a knowledge base using dense vector retrieval with a distance threshold to select knowledge-base items, one or more items related to the sample;
providing the sample and the contextual information to the appeals model;
generating, by the appeals model, a second verdict based on the contextual information; and
emitting, as the final verdict, the second verdict in preference to the first verdict.
2 . The method of claim 1 , further comprising providing the first verdict and, optionally, first-stage rationale to the appeals model as additional inputs.
3 . The method of claim 1 , wherein the appeals model is an application programming interface-accessed large language model and only processes samples pre-flagged as malicious.
4 . The method of claim 1 , wherein the contextual information includes a deployment-specific system prompt describing the application environment.
5 . The method of claim 1 , wherein the context-gating model is implemented as a two-tower neural network.
6 . The method of claim 1 , wherein the appeals model outputs a natural-language explanation for its verdict.
7 . The method of claim 1 , wherein the knowledge base comprises public or private articles related to the application domain.
8 . The method of claim 1 , wherein the similarity search is performed using a retrieval-augmented generation (RAG) model.
9 . The method of claim 1 , wherein the context-gating model is trained using labeled data indicating in-context and out-of-context samples.
10 . The method of claim 1 , wherein the appeals model is fine-tuned for the application domain.
11 . The method of claim 1 , wherein the method further comprises updating the knowledge base with new context information over time.
12 . The method of claim 1 , further comprising:
initiating, in response to the emitted verdict indicating that the sample is malicious, one or more remediation actions to prevent a machine learning model from behaving in an undesired manner.
13 . The method of claim 1 , further comprising orchestrating, by one or more autonomous software agents, at least one of:
routing the sample among the first-stage classifier, the context-gating model, and the appeals model;
performing semantic task decomposition of the sample;
coordinating data flow among the first-stage classifier, the context-gating model, and the appeals model; or
dynamically selecting and parameterizing the appeals model based on the sample.
14 . The method of claim 1 , further comprising, prior to invoking the context-gating model, preprocessing the sample to extract modality-specific features, the sample comprising at least one of text, image, audio, video, or a multimodal combination thereof, and the preprocessing comprising at least one of tokenization, normalization, sentence embedding generation, or transformation into embedded representations using a pre-trained model.
15 . The method of claim 1 , further comprising, after the context-gating model determines the sample is in-context, invoking an intermediary model configured to validate whether the sample is genuinely relevant to the application domain before providing the sample to the appeals model, the intermediary model operating as a bureaucrat agent that ensures only appropriate cases are escalated to the appeals model.
16 . The method of claim 1 , further comprising, in response to the final verdict indicating that the sample is malicious, initiating one or more remediation actions comprising at least one of:
blocking the sample from ingestion by a machine learning model; redacting personally identifiable information from the sample;
constraining decoding parameters of the machine learning model;
disabling tool invocation capabilities;
rate limiting requests; or
escalating the sample for human review.
17 . The method of claim 1 , further comprising, during the assembling of the contextual information, invoking a context engine that performs a semantic search over a knowledge base to retrieve one or more documents related to the sample, and processing the retrieved documents using a summarizer to generate a condensed summary that is provided to the appeals model as part of the contextual information.
18 . The method of claim 1 , further comprising logging, by a logging module, the first verdict, the second verdict, the contextual information, and process status for audit and retraining purposes.
19 . The method of claim 1 , wherein the context-gating model incorporates an output of the first-stage classifier, including at least one of a verdict and associated reasoning, as additional input features that are integrated with representations of the sample and context descriptors.
20 . The method of claim 1 , wherein the context engine provides different contextual information to the context-gating model and the appeals model, the contextual information provided to each model being selected based on requirements of the respective model's task.
21 . A system for adjudicating suspected-malicious samples with context gating, comprising:
at least one processor; and
at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising:
receiving a sample;
generating, by a first-stage classifier, a first verdict classifying the sample as malicious;
invoking a context-gating model distinct from the first-stage classifier and an appeals model, the context-gating model evaluating whether the sample is in-context for a specific application domain by computing an embedding for the sample and an embedding for context descriptors, comparing a distance between the embeddings against a threshold, and outputting an in-context or out-of-context label;
emitting, as a final verdict, the first verdict if the context-gating model determines the sample is out-of-context;
assembling, if the context-gating model determines the sample is in-context, contextual information by retrieving, via a similarity search over a knowledge base using dense vector retrieval with a distance threshold to select knowledge-base items, one or more items related to the sample;
providing the sample and the contextual information to the appeals model;
generating, by the appeals model, a second verdict based on the contextual information; and
emitting, as the final verdict, the second verdict in preference to the first verdict.
22 . The system of claim 21 , wherein the instructions further cause the system to perform operations comprising: providing the first verdict and, optionally, first-stage rationale to the appeals model as additional inputs.
23 . The system of claim 21 , wherein the appeals model is an application programming interface-accessed large language model and only processes samples pre-flagged as malicious.
24 . The system of claim 21 , wherein the contextual information includes a deployment-specific system prompt describing the application environment.
25 . The system of claim 21 , wherein the context-gating model is implemented as a two-tower neural network.
26 . The system of claim 21 , wherein the appeals model outputs a natural-language explanation for its verdict.
27 . The system of claim 21 , wherein the knowledge base comprises public or private articles related to the application domain.
28 . The system of claim 21 , wherein the similarity search is performed using a retrieval-augmented generation (RAG) model.
29 . The system of claim 21 , wherein the instructions further cause the system to perform operations comprising:
initiating, in response to the emitted verdict indicating that the sample is malicious, one or more remediation actions to prevent a machine learning model from behaving in an undesired manner.
30 . The system of claim 29 , wherein the one or more remediation actions comprise at least one of:
blocking the sample from ingestion by the machine learning model;
redacting personally identifiable information from the sample;
constraining decoding parameters of the machine learning model;
disabling tool invocation capabilities;
rate limiting requests; or
escalating the sample for human review.