Indicating differences in and reconciling data stored in disparate data storage devices
A method includes receiving a request to reconcile a first set of data stored in a first datastore and a second set of data stored in a second datastore, retrieving, from the first datastore, the first set of data, and retrieving, from the second datastore, the second set of data. The first set of data includes one or more first rows each including a respective first data value and the second set of data includes one or more second rows each including a respective second data value. The method also includes determining a mismatch between a respective one of the first data values and a respective one of the second data values. Based on determining the mismatch, the method also includes updating the respective one of the second data values to match the respective one of the first data values.
1 . A computer-implemented method executing on data processing hardware that causes the data processing hardware to perform operations comprising:
receiving a request to reconcile a first set of data stored in a first datastore and a second set of data stored in a second datastore;
retrieving, from the first datastore, the first set of data, the first set of data comprising one or more first columns and one or more first rows, the one or more first rows each comprising a respective first data value;
retrieving, from the second datastore, the second set of data, the second set of data comprising one or more second columns and one or more second rows, the one or more second rows each comprising a respective second data value;
determining a mismatch between a respective one of the first data values in a respective one of the one or more first rows of the first set of data and a respective one of the second data values in a respective one of the one or more second rows of the second set of data; and
based on determining the mismatch between the respective one of the first data values and the respective one of the second data values, updating the respective one of the second data values in the respective one of the one or more second rows in the second set of data to match the respective one of the first data values in the respective one of the one or more first rows in the second set of data.
2 . The computer-implemented method of claim 1 , wherein the operations further comprise, based on determining the mismatch between the respective one of the first data values and the respective one of the second data values, causing display of the mismatch between the respective one of the first data values and the respective one of the second data values.
3 . The computer-implemented method of claim 1 , wherein determining the mismatch between the respective one of the first data values in the respective one of the one or more first rows of the first set of data and the respective one of the second data values in the respective one of the one or more second rows of the second set of data comprises:
correlating each first row of the one or more first rows to a respective second row of the one or more second rows based on a comparison of a first identifier value stored in each first row of the one or more first rows and a second identifier value stored in each second row of the one or more second rows; and
matching the respective first data value stored in each first row of the one or more first rows to the respective second data value stored in a correlated second row of the one or more second rows.
4 . The computer-implemented method of claim 3 , wherein the first identifier value comprises a combination of two or more data elements stored in each respective first row of the one or more first rows and the second identifier value comprises a combination of two or more data elements stored in each respective second row of the one or more second rows.
5 . The computer-implemented method of claim 1 , wherein the operations further comprise generating a visual indicator for the mismatch.
6 . The computer-implemented method of claim 1 , wherein retrieving the first set of data comprises interfacing with an Application Program Interface (API) of a type corresponding a type of the first set of data.
7 . The computer-implemented method of claim 1 , wherein retrieving the second set of data comprises interfacing with an Application Program Interface (API) of a type corresponding a type of the second set of data.
8 . The computer-implemented method of claim 1 , wherein:
the first set of data comprises clinical trial data; and
the second set of data comprises safety data.
9 . The computer-implemented method of claim 1 , wherein the respective one of the first data values and the respective one of the second data values correspond to a same third column of a set of third columns of a data repository.
10 . The computer-implemented method of claim 1 , wherein the operations further comprise determining that the respective one of the second data values is incorrect based on determining that an accuracy level of the respective first data values in the respective one of the one or more first rows satisfies a threshold amount.
11 . A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware causes the data processing hardware to perform operations comprising:
receiving a request to reconcile a first set of data stored in a first datastore and a second set of data stored in a second datastore;
retrieving, from the first datastore, the first set of data, the first set of data comprising one or more first columns and one or more first rows, the one or more first rows each comprising a respective first data value;
retrieving, from the second datastore, the second set of data, the second set of data comprising one or more second columns and one or more second rows, the one or more second rows each comprising a respective second data value;
determining a mismatch between a respective one of the first data values in a respective one of the one or more first rows of the first set of data and a respective one of the second data values in a respective one of the one or more second rows of the second set of data; and
based on determining the mismatch between the respective one of the first data values and the respective one of the second data values, updating the respective one of the second data values in the respective one of the one or more second rows in the second set of data to match the respective one of the first data values in the respective one of the one or more first rows in the second set of data.
12 . The system of claim 11 , wherein the operations further comprise, based on determining the mismatch between the respective one of the first data values and the respective one of the second data values, causing display of the mismatch between the respective one of the first data values and the respective one of the second data values.
13 . The system of claim 11 , wherein determining the mismatch between the respective one of the first data values in the respective one of the one or more first rows of the first set of data and the respective one of the second data values in the respective one of the one or more second rows of the second set of data comprises:
correlating each first row of the one or more first rows to a respective second row of the one or more second rows based on a comparison of a first identifier value stored in each first row of the one or more first rows and a second identifier value stored in each second row of the one or more second rows; and
matching the respective first data value stored in each first row of the one or more first rows to the respective second data value stored in a correlated second row of the one or more second rows.
14 . The system of claim 13 , wherein the first identifier value comprises a combination of two or more data elements stored in each respective first row of the one or more first rows and the second identifier value comprises a combination of two or more data elements stored in each respective second row of the one or more second rows.
15 . The system of claim 11 , wherein the operations further comprise generating a visual indicator for the mismatch.
16 . The system of claim 11 , wherein retrieving the first set of data comprises interfacing with an Application Program Interface (API) of a type corresponding a type of the first set of data.
17 . The system of claim 11 , wherein retrieving the one or more second data files comprises interfacing with an Application Program Interface (API) of a type corresponding a type of the second set of data.
18 . The system of claim 11 , wherein:
the first set of data comprises clinical trial data; and
the second set of data comprises safety data.
19 . The system of claim 11 , wherein the respective one of the first data values and the respective one of the second data values correspond to a same third column of a set of third columns of a data repository.
20 . The system of claim 11 , wherein the operations further comprise determining that the respective one of the second data values is incorrect based on determining that an accuracy level of the respective first data values in the respective one of the one or more first rows satisfies a threshold amount.