Identifying relationships in data
View Patent ↗A computer implemented method of clustering computer systems in a plurality of systems to identify computer systems being subject to a common security occurrence, each computer system generating data records corresponding to security events in respect the systems, the method comprising: receiving a set of one or more data records associated with each computer system, each record including a sequence of data fields; generate a vector embedding for each data field in each record; evaluate a distance of each vector embedding from a reference vector as an indicator of semantic distance; identifying similar data records based on a measure of a degree of similarity of the distances of vector embeddings for each record; defining a cluster of computer systems including computer systems associated with the similar data records for applying protective measures to the computer systems in the cluster.
1 . A computer implemented method of clustering computer systems in a plurality of systems to identify computer systems being subject to a common security occurrence, each computer system generating data records corresponding to security events in respect of the systems, the method comprising:
receiving a set of one or more data records associated with each computer system, each record including a sequence of data fields;
generating a vector embedding for each data field in each record;
evaluating a distance of each vector embedding from a reference vector as an indicator of semantic distance;
identifying similar data records based on a measure of a degree of similarity of the distances of vector embeddings for each record;
defining a cluster of computer systems including computer systems associated with the similar data records for applying protective measures to the computer systems in the cluster;
wherein identifying similar data records includes;
generating a model of the measure of a degree of similarity of distances of embeddings for each record including an indication of a number of distances for the record being within a series of predetermined ranges of distance, the indication identifying a degree of concentration of each range of distances for the record; and
identifying records having a degree of concentration meeting a threshold degree in one or more common ranges of distance as similar records based on the model.
2 . The method of claim 1 wherein the vector embedding is a prediction-based vector embedding generated with a neural network trained on a corpus of data records of fields to semantically represent the fields.
3 . The method of claim 1 wherein the reference vector is selected statistically from the vector embeddings for the data fields in the set of data records.
4 . The method of claim 3 wherein the reference vector is an average of the vectors embeddings for the data fields.
5 . The method of claim 1 wherein the data records include textual representations of security events, and the data fields include sets of one or more words in each record.
6 . The method of claim 1 wherein the model is graphically renderable, said model is processed by a further component for identifying clusters graphically based on the indicated concentrations in the model.
7 . A non-transitory computer readable storage medium comprising computer program code to, when loaded into a computer and executed thereon, cause the computer to perform the steps of a method as claimed in claim 1 .
8 . A system for clustering computer systems in a plurality of systems to identify computer systems being subject to a common security occurrence, each computer system generating data records corresponding to security events in respect of the systems, the system including a processor and memory storing computer program code which when executed by the pre causes the system to at least be configured to:
receive a set of one or more data records associated with each computer system, each record including a sequence of data fields;
generate a vector embedding for each data field in each record;
evaluate a distance of each vector embedding from a reference vector as an indicator of semantic distance;
identify similar data records based on a measure ee of similarity of the distances of vector embeddings for each record;
define a cluster of computer systems including computer systems associated with the similar data records for applying protective measures to the computer systems in the cluster;
wherein to identify the similar data records, the system is further configured to:
generate a model of the measure of a degree of similarity of distances of embeddings for each record including an indication of a number of distances for the record being within a series of predetermined ranges of distance, the indication identifying a degree of concentration of each range of distances for the record; and
identify records having a degree of concentration meeting a threshold degree in one or more common ranges of distance as similar records based on the model.
9 . The system of claim 8 wherein the vector embedding is a prediction-based vector embedding generated with a neural network trained on a corpus of data records of fields to semantically represent the fields.
10 . The system of claim 8 wherein the reference vector is selected statistically from the vector embeddings for the data fields in the set of data records.
11 . The system of claim 10 wherein the reference vector is an average of the vectors embeddings for the data fields.
12 . The system of claim 8 wherein the data records include textual representations of security events, and the data fields include sets of one or more words in each record.
13 . The system of claim 8 wherein the model is graphically renderable, said model is processed by a further component for identifying clusters graphically based on the indicated concentrations in the model.
14 . The system of claim 8 wherein the system is further configured to apply the protective measures to the computer systems in the cluster, wherein the computer systems in the cluster are identified as being subject to the common security occurrence.
15 . The system of claim 14 wherein the protective measures include cessation or prevention of network communication by the computer systems in the cluster.
16 . The system of claim 14 wherein the protective measures include use of intrusion detection, virus detection and/or anti-malware by the computer systems in the cluster.
17 . The method of claim 1 further comprising applying the protective measures to the computer systems in the cluster, wherein the computer systems in the cluster are identified as being subject to the common security occurrence.
18 . The method of claim 17 wherein the protective measure is cessation or prevention of network communication by the computer systems in the cluster.
19 . The method of claim 17 wherein the protective measure is the use of intrusion detection, virus detection and/or anti-malware by the computer systems in the cluster.