Granted Patent B2

US 12,242,444 · App. 18/545,416 · Granted Mar 4, 2025

Inventors: John Joyce (Newton, MA); Marshall A. Isman (Newton, MA); Sandrick Melbouci (Myersville, MD)

Assignee: Ab Initio Technology LLC

G06F16/215G06F16/2228G06F16/285G06N5/04G06N20/00

Abstract

Claims (56)

1. A method for determining a schema of a data record, the method including:

retrieving a label index that associates a label with a set of one or more fields in the data record, wherein the label identifies a type of information expected in a field of the set of the one or more fields;

accessing an index that associates the type of information indicated by the label with a set of attribute values representing requirements for values of the one or more fields associated with the label, the requirements including logical or syntactical characteristics of the values for the one or more fields; and

for a first field of a particular data record:

identifying, by accessing the label index, a particular label associated with the first field of the particular data record;

retrieving, from the index, an attribute value for the particular label, the attribute value specifying a particular requirement for the first field;

determining that the particular requirement specified by the attribute value includes a schema feature for values included in the first field;

determining, based on the schema feature, a relationship between the first field and a second field in the data record; and

generating an output that indicates the relationship between the first field and the second field in the data record.

2. The method of claim 1 , further including:

determining, based on the schema feature, that the first field having the label includes key values being referenced by values of a second field; and

in response to the determining, updating schema data describing the particular data record to reference that the first field includes the key values being referenced by the values of the second field.

3. The method of claim 1 , further including:

determining, based on the schema feature, that the first field is a key field.

4. The method of claim 1 , wherein the relationship is indicative of a dependency of a value of the second field on a value of the first field or the value of the first field on the value of the second field.

5. The method of claim 1 , wherein the relationship is indicative of a correlation between a value of the first field and a value of the second field.

6. The method of claim 1 , further including obtaining validation data that validates the relationship for each value of the first field and the second field.

7. The method of claim 1 , wherein the relationship is validated for a threshold number of values for the first field and the second field.

8. The method of claim 1 , wherein the schema feature is configured to specify a constraint on a value of the second field based on a value of the first field.

9. The method of claim 1 , wherein the first field and the second field each includes numeric values, and wherein determining that the relationship exists between the first field and the second field includes determining a numerical function that relates values of the first field to values of the second field.

10. The method of claim 1 , wherein determining that the relationship exists between the first field and the second field includes using at least one classifier configured by a machine learning process.

11. The method of claim 1 , further including:

determining that the schema feature indicates that the first field includes primary key values for the particular data record; and

configuring a data quality rule to require that the primary key values are each unique in the first field.

12. The method of claim 1 , wherein the method includes storing data associating the schema feature and the particular label.

13. A data processing system for determining a schema of a data record, the data processing system including:

at least one processing device; and

at least one memory in communication with the at least one processing device, the at least one memory storing instructions that, when executed by the at least one processing device, cause the at least one processing device to perform operations including:

retrieving a label index that associates a label with a set of one or more fields in the data record, wherein the label identifies a type of information expected in a field of the set of the one or more fields;

accessing an index that associates the type of information indicated by the label with a set of attribute values representing requirements for values of the one or more fields associated with the label, the requirements including logical or syntactical characteristics of the values for the one or more fields; and

for a first field of a particular data record:

identifying, by accessing the label index, a particular label associated with the first field of the particular data record;

retrieving, from the index, an attribute value for the particular label, the attribute value specifying a particular requirement for the first field;

determining that the particular requirement specified by the attribute value includes a schema feature for values included in the first field;

determining, based on the schema feature, a relationship between the first field and a second field in the data record; and

generating an output that indicates the relationship between the first field and the second field in the data record.

14. The data processing system of claim 13 , the operations further including:

determining, based on the schema feature, that the first field having the label includes key values being referenced by values of a second field; and

in response to the determining, updating schema data describing the particular data record to reference that the first field includes the key values being referenced by the values of the second field.

15. The data processing system of claim 13 , the operations further including:

determining, based on the schema feature, that the first field is a key field.

16. The data processing system of claim 13 , wherein the relationship is indicative of a dependency of a value of the second field on a value of the first field or the value of the first field on the value of the second field.

17. The data processing system of claim 13 , wherein the relationship is indicative of a correlation between a value of the first field and a value of the second field.

18. The data processing system of claim 13 , further including obtaining validation data that validates the relationship for each value of the first field and the second field.

19. One or more non-transitory computer readable media storing instructions for determining a schema of a data record, wherein the instructions, when executed by at least one processing device, are configured to cause the at least one processing device to perform operations including:

retrieving a label index that associates a label with a set of one or more fields in the data record, wherein the label identifies a type of information expected in a field of the set of the one or more fields;

accessing an index that associates the type of information indicated by the label with a set of attribute values representing requirements for values of the one or more fields associated with the label, the requirements including logical or syntactical characteristics of the values for the one or more fields; and

for a first field of a particular data record:

identifying, by accessing the label index, a particular label associated with the first field of the particular data record;

retrieving, from the index, an attribute value for the particular label, the attribute value specifying a particular requirement for the first field;

determining that the particular requirement specified by the attribute value includes a schema feature for values included in the first field;

determining, based on the schema feature, a relationship between the first field and a second field in the data record; and

generating an output that indicates the relationship between the first field and the second field in the data record.

20. The one or more non-transitory computer readable media of claim 19 , the operations further including:

determining, based on the schema feature, that the first field having the label includes key values being referenced by values of a second field; and

in response to the determining, updating schema data describing the particular data record to reference that the first field includes the key values being referenced by the values of the second field.

Assignments (3)

Continuity (3)

References Cited (95)

Cited By (2)

US 12,456,016 US 12,518,052