Performing digital file analysis using graph-based representations and machine learning techniques

1 . A computer-implemented method comprising:

segmenting at least one digital file into multiple variable-sized portions by identifying portion boundaries within the at least one digital file using at least one rolling hash function, wherein using the at least one rolling hash function comprises:

moving a processing window of a designated size, across multiple steps, through at least parts of the at least one digital file;

calculating at least one hash value for contents of the processing window at at least a portion of the multiple steps;

marking at least part of the portion boundaries in response to determining that the at least one hash value corresponds to at least one designated condition; and

defining the multiple variable-sized portions based at least in part on the marking of the at least part of the portion boundaries;

determining one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file;

generating at least one graph representation of the at least one digital file based at least in part on the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file;

configuring a processor-based machine learning system to process one or more portions of the at least one graph representation, the processor-based machine learning system comprising at least one context-aware graph neural network and a contrastive learning network;

processing the one or more portions of the at least one graph representation in the processor-based machine learning system, wherein processing the one or more portions of the at least one graph representation in the processor-based machine learning system comprises encoding one or more relationships between content of the at least one digital file and structure of the at least one digital file by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network in conjunction with one or more attention mechanism layers and one or more dynamic edge modeling techniques, the encoding further comprising:

(i) implementing the at least one context-aware graph neural network to generate one or more content-aware embeddings, and wherein the at least one content-aware graph neural network comprises, connected in series:

(a) an input layer configured to receive node features derived from content of the multiple variable-sized portions of the at least one digital file, the node features comprising at least one of statistical properties, hash-based representations, and content embeddings of the respective ones of the multiple variable-sized portions, and the input layer having an output coupled to an input of one or more graph convolutional layers;

(b) the one or more graph convolutional layers, each connected in series after the input layer, each of the one or more graph convolutional layers being configured to compute, for each of a plurality of nodes of the at least one graph representation, an updated node feature vector by performing a weighted aggregation of feature vectors of neighboring nodes, wherein the weighted aggregation uses at least one size-aware weighting factor and at least one content-similarity weight, and each of the one or more graph convolutional layers having an output coupled to an input of the one or more attention mechanism layers connected in series after the one or more graph convolutional layers;

(c) the one or more attention mechanism layers configured to compute, for one or more pairs of connected nodes among the plurality of nodes, one or more attention coefficients, wherein each of the one or more attention mechanism layers has an output that is coupled to an input of at least one pooling layer connected in series after the one or more attention mechanism layers; and

(d) the at least one pooling layer configured to adaptively aggregate node feature vectors across the plurality of nodes of the at least one graph representation to produce the one or more content-aware embeddings at a network output of the at least one content-aware graph neural network, the network output of the at least one content-aware graph neural network being coupled to an input of a digital file analysis engine; and

(ii) implementing the contrastive learning network to:

generate, from the at least one graph representation, at least a plurality of augmented graph representations; and

provide a training signal at a training output of the contrastive learning network, the training output of the contrastive learning network being coupled to an input of the at least one content-aware graph neural network such that the contrastive learning network and the at least one content-aware graph neural network form a closed training loop operating in parallel with an inference path of the at least one content-aware graph neural network; and

performing one or more automated actions in connection with at least one data backup system by performing digital file similarity analysis across portions of the at least one digital file based at least in part on results of the processing of the one or more portions of the at least one graph representation, and wherein performing digital file similarity analysis comprises computing one or more similarity scores from the one or more content-aware embeddings, and generating a control signal based at least in part on the one or more similarity scores, the control signal being coupled to an input of the at least one data backup system separate from the processor-based machine learning system;

wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2 . The computer-implemented method of claim 1 , wherein processing one or more portions of the at least one graph representation in the processor-based machine learning system comprises extracting the one or more content-aware embeddings from the one or more portions of the at least one graph representation by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network.

3 . The computer-implemented method of claim 1 , wherein segmenting at least one digital file into multiple variable-sized portions comprises processing the at least one digital file using at least one variable-size chunking algorithm.

4 . The computer-implemented method of claim 1 , wherein determining one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file comprises processing the at least a portion of the multiple variable-sized portions of the at least one digital file using one or more content-based relationship modeling techniques.

5 . The computer-implemented method of claim 1 , wherein generating at least one graph representation of the at least one digital file comprises processing the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file using at least one graph construction algorithm.

6 . A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:

to segment at least one digital file into multiple variable-sized portions by identifying portion boundaries within the at least one digital file using at least one rolling hash function, wherein using the at least one rolling hash function comprises:

moving a processing window of a designated size, across multiple steps, through at least parts of the at least one digital file;

calculating at least one hash value for contents of the processing window at at least a portion of the multiple steps;

marking at least part of the portion boundaries in response to determining that the at least one hash value corresponds to at least one designated condition; and

defining the multiple variable-sized portions based at least in part on the marking of the at least part of the portion boundaries;

to determine one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file;

to generate at least one graph representation of the at least one digital file based at least in part on the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file;

to configure a processor-based machine learning system to process one or more portions of the at least one graph representation, the processor-based machine learning system comprising at least one context-aware graph neural network and a contrastive learning network;

to process the one or more portions of the at least one graph representation in the processor-based machine learning system, wherein processing the one or more portions of the at least one graph representation in the processor-based machine learning system comprises encoding one or more relationships between content of the at least one digital file and structure of the at least one digital file by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network in conjunction with one or more attention mechanism layers and one or more dynamic edge modeling techniques, the encoding further comprising:

(i) implementing the at least one context-aware graph neural network to generate one or more content-aware embeddings, and wherein the at least one content-aware graph neural network comprises, connected in series:

(a) an input layer configured to receive node features derived from content of the multiple variable-sized portions of the at least one digital file, the node features comprising at least one of statistical properties, hash-based representations, and content embeddings of the respective ones of the multiple variable-sized portions, and the input layer having an output coupled to an input of one or more graph convolutional layers;

(b) the one or more graph convolutional layers, each connected in series after the input layer, each of the one or more graph convolutional layers being configured to compute, for each of a plurality of nodes of the at least one graph representation, an updated node feature vector by performing a weighted aggregation of feature vectors of neighboring nodes, wherein the weighted aggregation uses at least one size-aware weighting factor and at least one content-similarity weight, and each of the one or more graph convolutional layers having an output coupled to an input of the one or more attention mechanism layers connected in series after the one or more graph convolutional layers;

(c) the one or more attention mechanism layers configured to compute, for one or more pairs of connected nodes among the plurality of nodes, one or more attention coefficients, wherein each of the one or more attention mechanism layers has an output that is coupled to an input of at least one pooling layer connected in series after the one or more attention mechanism layers; and

(d) the at least one pooling layer configured to adaptively aggregate node feature vectors across the plurality of nodes of the at least one graph representation to produce the one or more content-aware embeddings at a network output of the at least one content-aware graph neural network, the network output of the at least one content-aware graph neural network being coupled to an input of a digital file analysis engine; and

(ii) implementing the contrastive learning network to:

generate, from the at least one graph representation, at least a plurality of augmented graph representations; and

provide a training signal at a training output of the contrastive learning network, the training output of the contrastive learning network being coupled to an input of the at least one content-aware graph neural network such that the contrastive learning network and the at least one content-aware graph neural network form a closed training loop operating in parallel with an inference path of the at least one content-aware graph neural network; and

to perform one or more automated actions in connection with at least one data backup system by performing digital file similarity analysis across portions of the at least one digital file based at least in part on results of the processing of the one or more portions of the at least one graph representation, and wherein performing digital file similarity analysis comprises computing one or more similarity scores from the one or more content-aware embeddings, and generating a control signal based at least in part on the one or more similarity scores, the control signal being coupled to an input of the at least one data backup system separate from the processor-based machine learning system.

7 . The non-transitory processor-readable storage medium of claim 6 , wherein processing one or more portions of the at least one graph representation in the processor-based machine learning system comprises extracting the one or more content-aware embeddings from the one or more portions of the at least one graph representation by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network.

8 . An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

the at least one processing device being configured:

to segment at least one digital file into multiple variable-sized portions by identifying portion boundaries within the at least one digital file using at least one rolling hash function, wherein using the at least one rolling hash function comprises:

moving a processing window of a designated size, across multiple steps, through at least parts of the at least one digital file;

calculating at least one hash value for contents of the processing window at at least a portion of the multiple steps;

marking at least part of the portion boundaries in response to determining that the at least one hash value corresponds to at least one designated condition; and

defining the multiple variable-sized portions based at least in part on the marking of the at least part of the portion boundaries;

to determine one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file;

to generate at least one graph representation of the at least one digital file based at least in part on the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file;

to configure a processor-based machine learning system to process one or more portions of the at least one graph representation, the processor-based machine learning system comprising at least one context-aware graph neural network and a contrastive learning network;

to process the one or more portions of the at least one graph representation in the processor-based machine learning system, wherein processing the one or more portions of the at least one graph representation in the processor-based machine learning system comprises encoding one or more relationships between content of the at least one digital file and structure of the at least one digital file by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network in conjunction with one or more attention mechanism layers and one or more dynamic edge modeling techniques, the encoding further comprising:

(i) implementing the at least one context-aware graph neural network to generate one or more content-aware embeddings, and wherein the at least one content-aware graph neural network comprises, connected in series:

(a) an input layer configured to receive node features derived from content of the multiple variable-sized portions of the at least one digital file, the node features comprising at least one of statistical properties, hash-based representations, and content embeddings of the respective ones of the multiple variable-sized portions, and the input layer having an output coupled to an input of one or more graph convolutional layers;

(b) the one or more graph convolutional layers, each connected in series after the input layer, each of the one or more graph convolutional layers being configured to compute, for each of a plurality of nodes of the at least one graph representation, an updated node feature vector by performing a weighted aggregation of feature vectors of neighboring nodes, wherein the weighted aggregation uses at least one size-aware weighting factor and at least one content-similarity weight, and each of the one or more graph convolutional layers having an output coupled to an input of the one or more attention mechanism layers connected in series after the one or more graph convolutional layers;

(c) the one or more attention mechanism layers configured to compute, for one or more pairs of connected nodes among the plurality of nodes, one or more attention coefficients, wherein each of the one or more attention mechanism layers has an output that is coupled to an input of at least one pooling layer connected in series after the one or more attention mechanism layers; and

(d) the at least one pooling layer configured to adaptively aggregate node feature vectors across the plurality of nodes of the at least one graph representation to produce the one or more content-aware embeddings at a network output of the at least one content-aware graph neural network, the network output of the at least one content-aware graph neural network being coupled to an input of a digital file analysis engine; and

(ii) implementing the contrastive learning network to:

generate, from the at least one graph representation, at least a plurality of augmented graph representations; and

provide a training signal at a training output of the contrastive learning network, the training output of the contrastive learning network being coupled to an input of the at least one content-aware graph neural network such that the contrastive learning network and the at least one content-aware graph neural network form a closed training loop operating in parallel with an inference path of the at least one content-aware graph neural network; and

to perform one or more automated actions in connection with at least one data backup system by performing digital file similarity analysis across portions of the at least one digital file based at least in part on results of the processing of the one or more portions of the at least one graph representation, and wherein performing digital file similarity analysis comprises computing one or more similarity scores from the one or more content-aware embeddings, and generating a control signal based at least in part on the one or more similarity scores, the control signal being coupled to an input of the at least one data backup system separate from the processor-based machine learning system.

9 . The apparatus of claim 8 , wherein processing one or more portions of the at least one graph representation in the processor-based machine learning system comprises extracting the one or more content-aware embeddings from the one or more portions of the at least one graph representation by processing the one or more portions of the at least one graph representation using the at least one context-aware graph neural network.

10 . The apparatus of claim 8 , wherein segmenting at least one digital file into multiple variable-sized portions comprises processing the at least one digital file using at least one variable-size chunking algorithm.

11 . The apparatus of claim 8 , wherein determining one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file comprises processing the at least a portion of the multiple variable-sized portions of the at least one digital file using one or more content-based relationship modeling techniques.

12 . The apparatus of claim 8 , wherein generating at least one graph representation of the at least one digital file comprises processing the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file using at least one graph construction algorithm.

13 . The non-transitory processor-readable storage medium of claim 6 , wherein segmenting at least one digital file into multiple variable-sized portions comprises processing the at least one digital file using at least one variable-size chunking algorithm.

14 . The non-transitory processor-readable storage medium of claim 6 , wherein determining one or more content-based relationships across at least a portion of the multiple variable-sized portions of the at least one digital file comprises processing the at least a portion of the multiple variable-sized portions of the at least one digital file using one or more content-based relationship modeling techniques.

15 . The non-transitory processor-readable storage medium of claim 6 , wherein generating at least one graph representation of the at least one digital file comprises processing the one or more content-based relationships and the at least a portion of the multiple variable-sized portions of the at least one digital file using at least one graph construction algorithm.