Granted Patent B2

US 12,437,239 · App. 17/168,913 · Granted Oct 7, 2025

Inventors: Richard Harang (Alexandria, VA); Felipe Ducau (Oxford, GB)

Assignee: Sophos Limited

G06N20/20G06F18/213G06F18/214G06F18/2178G06F18/22G06N5/01G06N20/00

Abstract

Claims (35)

1. An apparatus, comprising:

a memory; and

a processor operatively coupled to the memory, the processor configured to:

train, at a first time, a machine learning model to output (1) an identification of whether an artifact is malicious and (2) a confidence value associated with the identification of whether the artifact is malicious;

receive a set of artifacts during each time period from a set of time periods, each time period from the set of time periods being after the first time;

for each time period from the set of time periods, provide a feature vector representative of each artifact from the set of artifacts received during that time period to the machine learning model to obtain as an output of the machine learning model an indication of whether that artifact is malicious and a confidence value associated with the indication of whether that artifact is malicious based on a degree of similarity of the feature vector with a set of feature vectors representative of a set of training artifacts used to train the machine learning model at the first time;

calculate a confidence metric for each time period from the set of time periods based on the confidence value associated with a number of artifacts from the set of artifacts received during that time period meeting a confidence value threshold;

calculate a rate of change in confidence based on the confidence metric for at least two time periods from the set of time periods;

in response to the rate of change in confidence meeting a retraining criterion:

receive, over a network and at a second time after the first time, an updated set of training artifacts; and

retrain, based on the updated set of training artifacts, the machine learning model.

2. The apparatus of claim 1 , wherein each time period from the set of time periods overlaps at least one remaining time period from the set of time periods.

3. The apparatus of claim 1 , wherein each time period from the set of time periods is mutually exclusive of remaining time periods from the set of time periods.

4. The apparatus of claim 1 , wherein the machine learning model is at least one of a neural network, a decision tree or a random forest.

5. The apparatus of claim 1 , wherein the confidence metric for each time period from the set of time periods is a percentage associated with (1) the number of artifacts from the set of artifacts received during that time period that have a confidence value above the confidence value threshold and (2) a total number of artifacts from the set of artifacts received during that time period.

6. A method, comprising:

training, at a first time, a machine learning model to output (1) an identification of whether an artifact is malicious and (2) a confidence value associated with the identification of whether the artifact is malicious;

receiving, during a first time period after the first time, a set of artifacts;

for each artifact from the set of artifacts, providing a feature vector representative of that artifact as an input to the machine learning model to obtain as an output of the machine learning model an indication of whether that artifact is malicious and a confidence value associated with the indication of whether that artifact is malicious based on a degree of similarity of the feature vector with a set of feature vectors representative of a set of training artifacts used to train the machine learning model at the first time;

comparing the confidence value for each artifact from the set of artifacts to a confidence criterion to identify a first metric associated with a first number of artifacts having confidence values that do not meet the confidence criterion; and

in response to a rate of change between the first metric and a second metric meeting a retraining criterion:

receiving, over a network and at a second time after the first time, an updated set of training artifacts; and

retraining, based on the updated set of training artifacts, the machine learning model;

the second metric associated with a second number of artifacts including artifacts (1) from a set of artifacts received during a second time period after the first time period and (2) having confidence values not meeting the confidence criterion.

7. The method of claim 6 , wherein the first metric is a percentage associated with (1) the first number of artifacts and (2) a total number of artifacts received during the first time period.

8. The method of claim 6 , wherein the machine learning model is at least one of a neural network, a decision tree or a random forest.

9. The apparatus of claim 1 , wherein the confidence value threshold includes a high confidence criterion and a low confidence criterion, the processor is further configured to:

compare the confidence value for each artifact from the set of artifacts, for a time period from the set of time periods, to the high confidence criterion to identify a number of artifacts having confidence values that meet the high confidence criterion; and

compare the confidence value for each artifact from the set of artifacts, for a time period from the set of time periods, to the low confidence criterion to identify a number of artifacts having confidence values that meet the low confidence criterion;

the confidence metric for that time period from the set of time periods being based on at least one of (1) the number of artifacts having confidence values that meet the high confidence criterion from the set of artifacts received during that time period, or (2) the number of artifacts having confidence values that meet the low confidence criterion from the set of artifacts received during that time period.

10. The apparatus of claim 9 , wherein the confidence metric is based on the number of artifacts, from the set of artifacts received during that time period, having confidence values that meet the high confidence criterion.

11. The apparatus of claim 9 , wherein the confidence metric is based on the number of artifacts, from the set of artifacts received during that time period, having confidence values that meet the low confidence criterion.

12. The apparatus of claim 1 , wherein the degree of similarity is defined by a Euclidean distance between the feature vector of each artifact from the set of artifacts received during that time period and the set of feature vectors of the set of training artifacts used to train the machine learning model at the first time.

13. The method of claim 6 , wherein the degree of similarity is defined by a Euclidean distance between the feature vector of each artifact from the set of artifacts received during the first time period and the set of feature vectors of the set of training artifacts used to train the machine learning model at the first time.

14. The method of claim 6 , wherein the first metric is at least one of a median of the first number of artifacts or a standard deviation of the first number of artifacts.

Assignments (1)

Continuity (3)

References Cited (279)