Machine learning techniques for efficient data pattern recognition across structured data objects
Various embodiments of the present invention provide methods, apparatus, systems, computing devices, computing entities, and/or the like for performing predictive data analysis with respect to structured data objects. Certain embodiments of the present invention utilize systems, methods, and computer program products that perform predictive data analysis with respect to structured data objects by utilizing at least one of cross-table data similarity score generation machine learning models and unsupervised anomalous table row detection machine learning models.
1 . A computer-implemented method comprising:
generating, by one or more processors, a first table data object and a second table data object;
generating, by the one or more processors, a first row-wise representation for a first table row of the first table data object and a second row-wise representation for a second table row of the second table data object;
generating, by the one or more processors, based at least in part on the first row-wise representation and the second row-wise representation, and by utilizing a shared embedding layer of a cross-table data similarity score generation machine learning model, a first embedded row-wise representation for the first table row and a second embedded row-wise representation for the second table row using data contained in the first table data object and the second table data object without metadata associated with the first table data object and the second table data object, wherein one or more parameters and one or more defined operations of the shared embedding layer are common across all row-wise representations in the first table data object and the second table data object and the cross-table data similarity score generation machine learning model is configured by:
(i) identifying a cross-table overlap ratio and a plurality of training columns within a set of training databases,
(ii) generating a first constructed table data object by sampling a first subset of the plurality of training columns,
(iii) generating a second constructed table data object by sampling a second subset of the plurality of training columns based at least in part on the first subset and the cross-table overlap ratio, wherein the first subset and the second subset comprise a number of overlapping columns from the plurality of training columns based at least in part on the cross-table overlap ratio, and
(iv) configuring the cross-table data similarity score generation machine learning model based at least in part on a plurality of constructed table pairs, wherein a constructed table pair of the plurality of constructed table pairs comprises the first constructed table data object and the second constructed table data object;
generating, by the one or more processors, based at least in part on the first embedded row-wise representation and the second embedded row-wise representation, and by utilizing a convolutional layer of the cross-table data similarity score generation machine learning model, a first context-aware row-wise representation for the first table row and a second context-aware row-wise representation for the second table row;
generating, by the one or more processors, based at least in part on the first context-aware row-wise representation and the second context-aware row-wise representation, and by utilizing a regression layer of the cross-table data similarity score generation machine learning model, a predicted cross-table data similarity score between the first table row and the second table row; and
initiating, by the one or more processors, a merging of the first table data object and the second table data object based at least in part on the predicted cross-table data similarity score satisfying a predicted cross-table similarity threshold.
2 . The computer-implemented method of claim 1 , wherein:
the constructed table pair is associated with a table size value that describes a per-table column count of the first constructed table data object and the second constructed table data object and a total column count of the first constructed table data object and the second constructed table data object.
3 . The computer-implemented method of claim 2 , wherein generating the constructed table pair comprises:
generating, using a random number generator, the table size value that describes the per-table column count for the first constructed table data object and the second constructed table data object; and
generating a non-augmented table pair by generating the first constructed table data object that is associated with the constructed table pair and the second constructed table data object that is associated with the constructed table pair in accordance with the table size value that is associated with the constructed table pair and the cross-table overlap ratio that is associated with the constructed table pair.
4 . The computer-implemented method of claim 3 , wherein generating the constructed table pair further comprises:
performing a row-wise shuffling operation on the non-augmented table pair to generate the constructed table pair, or
performing a column-wise shuffling operation on the non-augmented table pair to generate the constructed table pair.
5 . The computer-implemented method of claim 1 , wherein the first table data object is generated by performing a row-wise shuffling operation on a non-augmented first table data object or performing a column-wise shuffling operation on the non-augmented first table data object.
6 . The computer-implemented method of claim 1 , wherein the second table data object is generated by performing a row-wise shuffling operation on a non-augmented second table data object or performing a column-wise shuffling operation on the non-augmented second table data object.
7 . The computer-implemented method of claim 1 , wherein the regression layer comprises:
a concatenation layer that is configured to generate a first table-wise representation for the first table data object based at least in part on the first context-aware row-wise representation and a second table-wise representation for the second table data object based at least in part on the second context-aware row-wise representation,
a fully-connected layer that is configured to generate an inferred fully-connected output based at least in part on the first table-wise representation and the second table-wise representation, and
an output layer that is configured to generate the predicted cross-table data similarity score based at least in part on the inferred fully-connected output.
8 . The computer-implemented method of claim 1 , further comprising:
generating, based at least in part on the first context-aware row-wise representation and by utilizing an unsupervised anomalous table row detection machine learning model, a row-wise anomaly score for the first table row; and
initiating removal of the first table row based at least in part on the row-wise anomaly score satisfying a row-wise anomaly score threshold.
9 . A system comprising:
one or more processors; and
at least one memory storing processor-executable instructions that, when collectively or independently executed by any one or more of the one or more processors, cause the one or more processors to perform operations comprising:
generating a first table data object and a second table data object;
generating a first row-wise representation for a first table row of the first table data object and a second row-wise representation for a second table row of the second table data object;
generating, based at least in part on the first row-wise representation and the second row-wise representation, and by utilizing a shared embedding layer of a cross-table data similarity score generation machine learning model, a first embedded row-wise representation for the first table row and a second embedded row-wise representation for the second table row using data contained in the first table data object and the second table data object without metadata associated with the first table data object and the second table data object, wherein one or more parameters and one or more defined operations of the shared embedding layer are common across all row-wise representations in the first table data object and the second table data object and the cross-table data similarity score generation machine learning model is configured by:
(i) identifying a cross-table overlap ratio and a plurality of training columns within a set of training databases,
(ii) generating a first constructed table data object by sampling a first subset of the plurality of training columns,
(iii) generating a second constructed table data object by sampling a second subset of the plurality of training columns based at least in part on the first subset and the cross-table overlap ratio, wherein the first subset and the second subset comprise a number of overlapping columns from the plurality of training columns based at least in part on the cross-table overlap ratio, and
(iv) configuring the cross-table data similarity score generation machine learning model based at least in part on a plurality of constructed table pairs, wherein a constructed table pair of the plurality of constructed table pairs comprises the first constructed table data object and the second constructed table data object;
generating, based at least in part on the first embedded row-wise representation and the second embedded row-wise representation, and by utilizing a convolutional layer of the cross-table data similarity score generation machine learning model, a first context-aware row-wise representation for the first table row and a second context-aware row-wise representation for the second table row;
generating, based at least in part on the first context-aware row-wise representation and the second context-aware row-wise representation, and by utilizing a regression layer of the cross-table data similarity score generation machine learning model, a predicted cross-table data similarity score between the first table row and the second table row; and
initiating a merging of the first table data object and the second table data object based at least in part on the predicted cross-table data similarity score satisfying a predicted cross-table similarity threshold.
10 . The system of claim 9 , wherein:
the constructed table pair is associated with a table size value that describes a per-table column count of the first constructed table data object and the second constructed table data object and a total column count of the first constructed table data object and the second constructed table data object.
11 . The system of claim 10 , wherein generating the constructed table pair comprises:
generating, using a random number generator, the table size value that describes the per-table column count for the first constructed table data object and the second constructed table data object; and
generating a non-augmented table pair by generating the first constructed table data object that is associated with the constructed table pair and the second constructed table data object that is associated with the constructed table pair in accordance with the table size value that is associated with the constructed table pair and the cross-table overlap ratio that is associated with the constructed table pair.
12 . The system of claim 11 , wherein generating the constructed table pair further comprises:
performing a row-wise shuffling operation on the non-augmented table pair to generate the constructed table pair, or
performing a column-wise shuffling operation on the non-augmented table pair to generate the constructed table pair.
13 . The system of claim 9 , wherein the first table data object is generated by performing a row-wise shuffling operation on a non-augmented first table data object or performing a column-wise shuffling operation on the non-augmented first table data object.
14 . The system of claim 9 , wherein the second table data object is generated by performing a row-wise shuffling operation on a non-augmented second table data object or performing a column-wise shuffling operation on the non-augmented second table data object.
15 . The system of claim 9 , wherein the regression layer comprises:
a concatenation layer that is configured to generate a first table-wise representation for the first table data object based at least in part on the first context-aware row-wise representation and a second table-wise representation for the second table data object based at least in part on the second context-aware row-wise representation,
a fully-connected layer that is configured to generate an inferred fully-connected output based at least in part on the first table-wise representation and the second table-wise representation, and
an output layer that is configured to generate the predicted cross-table data similarity score based at least in part on the inferred fully-connected output.
16 . The system of claim 9 , wherein the operations comprise:
generating, based at least in part on the first context-aware row-wise representation and by utilizing an unsupervised anomalous table row detection machine learning model, a row-wise anomaly score for the first table row; and
initiating removal of the first table row based at least in part on the row-wise anomaly score satisfying a row-wise anomaly score threshold.
17 . One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
generating a first table data object and a second table data object;
generating a first row-wise representation for a first table row of the first table data object and a second row-wise representation for a second table row of the second table data object;
generating, based at least in part on the first row-wise representation and the second row-wise representation, and by utilizing a shared embedding layer of a cross-table data similarity score generation machine learning model, a first embedded row-wise representation for the first table row and a second embedded row-wise representation for the second table row using data contained in the first table data object and the second table data object without metadata associated with the first table data object and the second table data object, wherein one or more parameters and one or more defined operations of the shared embedding layer are common across all row-wise representations in the first table data object and the second table data object and the cross-table data similarity score generation machine learning model is configured by:
(i) identifying a cross-table overlap ratio and a plurality of training columns within a set of training databases,
(ii) generating a first constructed table data object by sampling a first subset of the plurality of training columns,
(iii) generating a second constructed table data object by sampling a second subset of the plurality of training columns based at least in part on the first subset and the cross-table overlap ratio, wherein the first subset and the second subset comprise a number of overlapping columns from the plurality of training columns based at least in part on the cross-table overlap ratio, and
(iv) configuring the cross-table data similarity score generation machine learning model based at least in part on a plurality of constructed table pairs, wherein a constructed table pair of the plurality of constructed table pairs comprises the first constructed table data object and the second constructed table data object;
generating, based at least in part on the first embedded row-wise representation and the second embedded row-wise representation, and by utilizing a convolutional layer of the cross-table data similarity score generation machine learning model, a first context-aware row- wise representation for the first table row and a second context-aware row-wise representation for the second table row;
generating, based at least in part on the first context-aware row-wise representation and the second context-aware row-wise representation, and by utilizing a regression layer of the cross-table data similarity score generation machine learning model, a predicted cross-table data similarity score between the first table row and the second table row; and
initiating a merging of the first table data object and the second table data object based at least in part on the predicted cross-table data similarity score satisfying a predicted cross-table similarity threshold.
18 . The one or more non-transitory computer-readable storage media of claim 17 , wherein:
the constructed table pair is associated with a table size value that describes a per-table column count of the first constructed table data object and the second constructed table data object and a total column count of the first constructed table data object and the second constructed table data object.