Data source correlation user interface
Systems and methods are disclosed for implementing a data stream correlation user interface. The data stream correlation user interface enables users to view information from two sets of records, and identify fields in the two sets of records that can be matched together to “glue” together multiple records. For example, a user may specify that values in an “AcctID” field in one set of records can be matched to values in an “Account_ID” field of a second set of records. Additional identifying fields may be selected, such that multiple values can be chained together. The system can match the records of multiple sets together using designated fields, enabling users to view how many records from one set have a corresponding record in another set.
1. A computer-implemented method, comprising:
determining a first field from a first plurality of fields associated with a first set of records, each record of the first set of records having one or more fields from the first plurality of fields, wherein the first set of records are associated with a first data source;
determining a second field from a second plurality of fields associated with a second set of records, each record of the second set of records having one or more fields from the second plurality of fields, where the second set of records are associated with a second data source that is different from the first data source;
identifying a first record from the first set of records that includes a first value for the first field;
identifying a second record from the second set of records that includes a second value for the second field;
determining, by a computing system and independent of user input specifying that the first value corresponds to the second value, whether the first value corresponds to the second value;
computing, using a result of determining whether the first value corresponds to-the second value, a number of records from the first set of records that can be correlated with one or more records from the second set of records; and
generating a display indicating, based in part on the result of determining whether the first value corresponds to the second value, a potential correlation between the first field of the first set of records and the second field of the second set of records, the display including the first field, the second field, the first value, and the second value, wherein the display associates the first value with the first field and the second value with the second field, wherein the display further includes an input enabling a user to verify that the first field of the first set of records and the second field of the second set of records are correlated, and wherein the display further includes an indication of the number of records from the first set of records that can be correlated with one or more records from the second set of records;
obtaining the input from the user verifying that the first field of the first set of records and the second field of the second set of records are correlated; and
processing a query associated with the first and second sets of records according to the verified correlation.
2. The computer-implemented method of claim 1 , wherein the first value corresponds to the second value when the first value is the same as the second value.
3. The computer-implemented method of claim 2 , wherein the display further includes an indication of whether the first value is the same value as the second value.
4. The computer-implemented method of claim 1 , wherein the indication of the number of records from the first set of records that can be correlated with one or more records from the second set of records is represented as a percentage of records in the first set of records that can be correlated with one or more records from the second set of records.
5. The computer-implemented method of claim 1 , wherein the display further includes an indication of a number of records from the second set of records that can be correlated with one or more records from the first set of records.
6. The computer-implemented method of claim 1 , wherein the display further includes an indication of an average number of records from the second set of records are correlated with each value of the first field.
7. The computer-implemented method of claim 1 , wherein the display further includes an indication of an average number of records from the second set of records are correlated with each value of the first field that corresponds to at least one value of the second field.
8. The computer-implemented method of claim 1 , wherein the first value corresponds to the second value when the first value represents the second value in a different format than the second value.
9. The computer-implemented method of claim 1 , wherein the first value corresponds to the second value when the first and second values are correlated via a lookup table, matching function, or user-specified criteria.
10. The computer-implemented method of claim 1 , wherein the first field or the second field is determined from input obtained using a user interface.
11. The computer-implemented method of claim 1 , wherein a record from the first set of records can be correlated with a record from the second set of records when the first value is the same value as the second value.
12. The computer-implemented method of claim 1 , wherein the first field is associated with a first textual identifier and the second field is associated with a second textual identifier that is different from the first textual identifier.
13. The computer-implemented method of claim 1 further comprising:
identifying a third record from the second set of records that includes the second value for the second field; and
determining that the first record can be correlated with the third record.
14. The computer-implemented method of claim 1 further comprising:
obtaining an indication that an additional field of the second plurality of fields associated with the second set of records is an identifying field;
determining an additional value for the additional field in the second record; and
storing a correlation indicator indicating that records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field can be correlated to one another.
15. The computer-implemented method of claim 1 further comprising:
obtaining an indication that an additional field of the second plurality of fields associated with the second set of records is an identifying field; and
determining an additional value for the additional field in the second record;
wherein the display further includes a listing of correlated records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field.
16. The computer-implemented method of claim 1 further comprising:
obtaining an indication that an additional field of the second plurality of fields associated with the second set of records is an identifying field; and
determining an additional value for the additional field in the second record;
wherein the display further includes:
a listing of correlated records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field; and
a confirmation input selectable by a user to confirm that records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field can be correlated to one another.
17. A system comprising:
a data store including computer-executable instructions; and
a processor configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the system to:
determine a first field from a first plurality of fields associated with a first set of records, each record of the first set of records having one or more fields from the first plurality of fields, wherein the first set of records are associated with a first data source;
determine a second field from a second plurality of fields associated with a second set of records, each record of the second set of records having one or more fields from the second plurality of fields, where the second set of records are associated with a second data source that is different from the first data source;
identify a first record from the first set of records that includes a first value for the first field;
identify a second record from the second set of records that includes a second value for the second field;
determine, by a computing system and independent of user input specifying that the first value corresponds to the second value, whether the first value corresponds to the second value;
compute, using a result of determining whether the first value corresponds to the second value, a number of records from the first set of records that can be correlated with one or more records from the second set of records; and
generate a display indicating, based in part on the result of determining whether the first value corresponds to the second value, a potential correlation between the first field of the first set of records and the second field of the second set of records, the display including the first field, the second field, the first value, and the second value, wherein the display associates the first value with the first field and the second value with the second field, wherein the display further includes an input enabling a user to verify that the first field of the first set of records and the second field of the second set of records are correlated, and wherein the display further includes an indication of the number of records from the first set of records that can be correlated with one or more records from the second set of records;
obtain the input from the user verifying that the first field of the first set of records and the second field of the second set of records are correlated; and
process a query associated with the first and second sets of records according to the verified correlation.
18. The system of claim 17 , wherein execution of the computer-executable instructions further causes the system to:
obtain an indication that an additional field of the second plurality of fields associated with the second set of records is an identifying field;
determine an additional value for the additional field in the second record; and
store a correlation indicator indicating that records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field can be correlated to one another.
19. One or more non-transitory computer-readable media containing computer-executable instructions that, when executed by a computing system, cause the computing system to:
determine a first field from a first plurality of fields associated with a first set of records, each record of the first set of records having one or more fields from the first plurality of fields, wherein the first set of records are associated with a first data source;
determine a second field from a second plurality of fields associated with a second set of records, each record of the second set of records having one or more fields from the second plurality of fields, where the second set of records are associated with a second data source that is different from the first data source;
identify a first record from the first set of records that includes a first value for the first field;
identify a second record from the second set of records that includes a second value for the second field;
determine, by the computing system and independent of user input specifying that the first value corresponds to the second value, whether the first value corresponds to the second value;
compute, using a result of determining whether the first value corresponds to the second value, a number of records from the first set of records that can be correlated with one or more records from the second set of records; and
generate a display indicating, based in part on the result of determining whether the first value corresponds to the second value, a potential correlation between the first field of the first set of records and the second field of the second set of records, the display including the first field, the second field, the first value, and the second value, wherein the display associates the first value with the first field and the second value with the second field, wherein the display further includes an input enabling a user to verify that the first field of the first set of records and the second field of the second set of records are correlated, and wherein the display further includes an indication of the number of records from the first set of records that can be correlated with one or more records from the second set of records;
obtain the input from the user verifying that the first field of the first set of records and the second field of the second set of records are correlated; and
process a query associated with the first and second sets of records according to the verified correlation.
20. The one or more non-transitory computer-readable media of claim 19 , wherein execution of the computer-executable instructions further causes the computing system to:
obtain an indication that an additional field of the second plurality of fields associated with the second set of records is an identifying field;
determine an additional value for the additional field in the second record; and
store a correlation indicator indicating that records having any of the first value for the first field, the second value for the second field, or the additional value for the additional field can be correlated to one another.
21. The one or more non-transitory computer-readable media of claim 19 , wherein the potential correlation is determined based on one of a lookup table, a gluing event, or applying a matching algorithm.