System and method for transforming network data packets into pixels for pattern analysis
A system for transforming data packets into pixels is disclosed. The system encodes each data packet into a respective encoded value, where each encoded value represents a unique transformation of a respective data packet. The system identifies data patterns within the data packets, where each data pattern indicates a respective data packet type. The system compresses each group of data packets that share a data pattern into a compressed format. The system determines a set of pixel attributes for each encoded, compressed data packet based on data packet attributes. The set of pixel attributes include a pixel color, a pixel color intensity, and a pixel location. The system generates an image comprising pixels based on the sets of pixel attributes.
1 . A system comprising:
a network interface configured to receive a plurality of data packets, wherein each data packet is associated with a set of data packet attributes comprising at least one of a type, content, or a timestamp of communication in a network; and
a processor, operably coupled to the network interface, and configured to:
encode each data packet from among the plurality of data packets into a respective encoded value, wherein a given encoded value represents a unique transformation of a respective data packet;
identify a plurality of data patterns within the plurality of encoded data packets, wherein each data pattern indicates a respective data packet type;
compress each group of encoded data packets that share a data pattern into a compressed format;
for each encoded, compressed data packet, determine a set of pixel attributes for a pixel based, at least in part, upon the set of data packet attributes, wherein the set of pixel attributes comprises:
a pixel color, wherein the pixel color is associated with the type of a respective encoded, compressed data packet;
a pixel color intensity, wherein the pixel color intensity is associated with a value of the content within the respective encoded, compressed data packet; and
a pixel location, wherein the pixel location is associated with the timestamp of communication of the respective encoded, compressed data packet; and
generate, based, at least in part, upon the set of pixel attributes, an image comprising a plurality of pixels, wherein:
each pixel indicates a given encoded, compressed data packet; and
the image visually represents a trend of the plurality of data packets with respect to time.
2 . The system of claim 1 , wherein the pixel location comprises:
an x-axis position associated with the timestamp of the respective compressed, encoded data packet, and
a y-axis position associated with the type of the respective compressed, encoded data packet.
3 . The system of claim 1 , wherein the processor is further configured to:
determine that a pixel region within the image is anomalous, wherein:
the pixel region comprises one or more adjacent pixels;
the anomalous pixel region represents a security breach at one or more data packets represented by the one or more adjacent pixels within the pixel region; and
in response to determining that the pixel region within the image is anomalous, perform one or more countermeasure actions.
4 . The system of claim 3 , wherein the one or more countermeasure actions comprise appending the one or more data packets with an additional header field as indicating anomalous or communicating an alert message that indicates that the one or more data packets are anomalous.
5 . The system of claim 3 , wherein determining that the pixel region within the image is anomalous comprises:
applying a sliding pixel window over the image to evaluate each pixel region; and
comparing the pixel region with a training pixel region that is labeled as anomalous, comprising:
generating a first embedding vector representing a first set of pixel attributes associated with the pixel region;
generating a second embedding vector representing a second set of pixel attributes associated with the training pixel region;
comparing the first embedding vector with the second embedding vector in a vector space; and
determining that a distance between the first embedding vector and the second embedding vector in the vector space is less than a threshold distance.
6 . The system of claim 1 , wherein the plurality of data patterns comprises at least one of the following:
an application programming interface (API) request;
an API response;
a hypertext transfer protocol (HTTP) request;
a structured query language (SQL) query; or
a webhook.
7 . The system of claim 1 , wherein:
each subset of the plurality of data packets is received from a different data source, and
the processor is further configured to convert the plurality of data packets to a configured data structure.
8 . A method comprising:
encoding each data packet from among a plurality of data packets into a respective encoded value, wherein a given encoded value represents a unique transformation of a respective data packet, wherein each data packet is associated with a set of data packet attributes comprising at least one of a type, content, or a timestamp of communication in a network;
identifying a plurality of data patterns within the plurality of encoded data packets, wherein each data pattern indicates a respective data packet type;
compressing each group of encoded data packets that share a data pattern into a compressed format;
for each encoded, compressed data packet, determining a set of pixel attributes for a pixel based, at least in part, upon the set of data packet attributes, wherein the set of pixel attributes comprises:
a pixel color, wherein the pixel color is associated with the type of a respective encoded, compressed data packet;
a pixel color intensity, wherein the pixel color intensity is associated with a value of the content within the respective encoded, compressed data packet; and
a pixel location, wherein the pixel location is associated with the timestamp of communication of the respective encoded, compressed data packet; and
generating, based, at least in part, upon the set of pixel attributes, an image comprising a plurality of pixels, wherein:
each pixel indicates a given encoded, compressed data packet; and
the image visually represents a trend of the plurality of data packets with respect to time.
9 . The method of claim 8 , wherein the pixel location comprises:
an x-axis position associated with the timestamp of the respective compressed, encoded data packet, and
a y-axis position associated with the type of the respective compressed, encoded data packet.
10 . The method of claim 8 , further comprising:
determining that a pixel region within the image is anomalous, wherein:
the pixel region comprises one or more adjacent pixels; and
the anomalous pixel region represents a security breach at one or more data packets represented by the one or more adjacent pixels within the pixel region; and
in response to determining that the pixel region within the image is anomalous, performing one or more countermeasure actions.
11 . The method of claim 10 , wherein the one or more countermeasure actions comprise appending the one or more data packets with an additional header field as indicating anomalous or communicating an alert message that indicates that the one or more data packets are anomalous.
12 . The method of claim 10 , wherein determining that the pixel region within the image is anomalous comprises:
applying a sliding pixel window over the image to evaluate each pixel region; and
comparing the pixel region with a training pixel region that is labeled as anomalous, comprising:
generating a first embedding vector representing a first set of pixel attributes associated with the pixel region;
generating a second embedding vector representing a second set of pixel attributes associated with the training pixel region;
comparing the first embedding vector with the second embedding vector in a vector space; and
determining that a distance between the first embedding vector and the second embedding vector in the vector space is less than a threshold distance.
13 . The method of claim 8 , wherein the plurality of data patterns comprises at least one of the following:
an application programming interface (API) request;
an API response;
a hypertext transfer protocol (HTTP) request;
a structured query language (SQL) query; or
a webhook.
14 . The method of claim 8 , wherein:
each subset of the plurality of data packets is received from a different data source, and
to the method further comprises converting the plurality of data packets to a configured data structure.
15 . A non-transitory computer-readable medium storing instructions that when executed by a processor, cause the processor to:
encode each data packet from among a plurality of data packets into a respective encoded value, wherein a given encoded value represents a unique transformation of a respective data packet, wherein each data packet is associated with a set of data packet attributes comprising at least one of a type, content, or a timestamp of communication in a network;
identify a plurality of data patterns within the plurality of encoded data packets, wherein each data pattern indicates a respective data packet type;
compress each group of encoded data packets that share a data pattern into a compressed format;
for each encoded, compressed data packet, determine a set of pixel attributes for a pixel based, at least in part, upon the set of data packet attributes, wherein the set of pixel attributes comprises:
a pixel color, wherein the pixel color is associated with the type of a respective encoded, compressed data packet;
a pixel color intensity, wherein the pixel color intensity is associated with a value of the content within the respective encoded, compressed data packet; and
a pixel location, wherein the pixel location is associated with the timestamp of communication of the respective encoded, compressed data packet; and
generate, based, at least in part, upon the set of pixel attributes, an image comprising a plurality of pixels, wherein:
each pixel indicates a given encoded, compressed data packet; and
the image visually represents a trend of the plurality of data packets with respect to time.
16 . The non-transitory computer-readable medium of claim 15 , wherein the pixel location comprises:
an x-axis position associated with the timestamp of the respective compressed, encoded data packet, and
a y-axis position associated with the type of the respective compressed, encoded data packet.
17 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to:
determine that a pixel region within the image is anomalous, wherein:
the pixel region comprises one or more adjacent pixels; and
the anomalous pixel region represents a security breach at one or more data packets represented by the one or more adjacent pixels within the pixel region; and
in response to determining that the pixel region within the image is anomalous, perform one or more countermeasure actions.
18 . The non-transitory computer-readable medium of claim 17 , wherein the one or more countermeasure actions comprise appending the one or more data packets with an additional header field as indicating anomalous or communicating an alert message that indicates that the one or more data packets are anomalous.
19 . The non-transitory computer-readable medium of claim 17 , wherein determining that the pixel region within the image is anomalous comprises:
applying a sliding pixel window over the image to evaluate each pixel region;
comparing the pixel region with a training pixel region that is labeled as anomalous, comprising:
generating a first embedding vector representing a first set of pixel attributes associated with the pixel region;
generating a second embedding vector representing a second set of pixel attributes associated with the training pixel region;
comparing the first embedding vector with the second embedding vector in a vector space; and
determining that a distance between the first embedding vector and the second embedding vector in the vector space is less than a threshold distance.
20 . The non-transitory computer-readable medium of claim 15 , wherein compressing each group of encoded data packets that share the data pattern comprises:
identifying a first repeating structure within the plurality of encoded data packets;
replacing the first repeating structure with a first compressed representation;
recursively analyzing the first compressed representation to identify one or more additional repeating structures within the first compressed representation; and
replacing each of the one or more additional repeating structures with a corresponding compressed representation.