Entity resolution and consolidation using a unique identifier
Examples described herein provide entity resolution and consolidation using a unique identifier for data with complex relationships gathered across a computer network from a plurality of sources. According to an aspect, a computer-implemented method includes receiving, by a processing device, supply chain data. The method also includes analyzing, by the processing device, the supply chain data to identify a unique entity and a doing business as (DBA) name associated with the unique entity. The method further includes generating, by the processing device, the unique identifier for the unique entity. The method additionally includes associating, by the processing device, the unique identifier with the unique entity and the DBA name associated with the unique entity and updating, by the processing device, the supply chain data to include the unique identifier for the unique entity and the DBA name associated with the unique entity.
1 . A computer-implemented method for entity resolution and consolidation using a unique identifier, the method comprising:
receiving, by a processing device, supply chain data from a plurality of data sources distributed across a network;
analyzing, by the processing device, the supply chain data to identify an entity and a doing business as (DBA) name associated with the entity;
generating, by the processing device, the unique identifier for the entity;
identifying, by the processing device, a second DBA name associated with a second entity in the supply chain data;
comparing, by the processing device, information about the DBA name and the second DBA name to determine whether the DBA name and the second DBA name are both associated with a same entity;
grouping, by the processing device, the second DBA name and the second entity with the DBA name and the entity based on determining that the DBA name and the second DBA name are both associated with the same entity as a unique entity;
associating, by the processing device, the unique identifier with the unique entity, the DBA name associated with the unique entity, and the second DBA name associated with the unique entity;
creating, by the processing device, a data structure with a plurality of unique identifiers including the unique identifier to link multiple source data sets from the data sources as normalized dynamic data streams;
flowing, by the processing device, updates from the source data sets based on data feeds of the data sources to the data structure through inheritance properties such that the updates are available at a predetermined interval;
updating, by the processing device, the data structure to include the DBA name associated with the unique entity, and the second DBA name associated with the unique entity, thereby providing entity resolution and consolidation using the unique identifier.
2 . The computer-implemented method of claim 1 , wherein the DBA name is one of a plurality of DBA names associated with the unique entity.
3 . The computer-implemented method of claim 1 , wherein the unique identifier is a hash and is determined as part of a multi-phase process comprising a plurality of extract, transform, and load (ETL) phases, and wherein separate processing of entities having a source system identifier and missing the source system identifier is performed in one of the ETL phases prior to processing the unique identifier in a subsequent one of the ETL phases.
4 . The computer-implemented method of claim 3 , wherein the hash is based at least in part on information about the unique entity.
5 . The computer-implemented method of claim 3 , wherein the hash is based at least in part on a uniform resource locator associated with the unique entity.
6 . The computer-implemented method of claim 1 , wherein the supply chain data comprises global trade data, cyber data, and government spending data.
7 . The computer-implemented method of claim 1 , further comprising performing a supply chain analysis using the supply chain data updated with the unique identifier, wherein the supply chain analysis identifies one or more potential risks associated with the unique entity.
8 . A system comprising:
a memory comprising computer readable instructions; and
a processing device for executing the computer readable instructions, the computer readable instructions controlling the processing device to perform operations comprising:
receiving supply chain data from a plurality of data sources distributed across a network;
analyzing the supply chain data to identify an entity and a doing business as (DBA) name associated with the entity;
generating a unique identifier for the entity;
identifying a second DBA name associated with a second entity in the supply chain data;
comparing information about the DBA name and the second DBA name to determine whether the DBA name and the second DBA name are both associated with a same entity;
grouping the second DBA name and the second entity with the DBA name and the entity based on determining that the DBA name and the second DBA name are both associated with the same entity as a unique entity;
associating the unique identifier with the unique entity, the DBA name associated with the unique entity, and the second DBA name associated with the unique entity;
creating a data structure with a plurality of unique identifiers including the unique identifier to link multiple source data sets from the data sources as normalized dynamic data streams;
flowing updates from the source data sets based on data feeds of the data sources to the data structure through inheritance properties such that the updates are available at a predetermined interval; and
updating the data structure to include the DBA name associated with the unique entity, and the second DBA name associated with the unique entity, thereby providing entity resolution and consolidation using the unique identifier.
9 . The system of claim 8 , wherein the DBA name is one of a plurality of DBA names associated with the unique entity.
10 . The system of claim 8 , wherein the unique identifier is a hash and is determined as part of a multi-phase process comprising a plurality of extract, transform, and load (ETL) phases, and wherein separate processing of entities having a source system identifier and missing the source system identifier is performed in one of the ETL phases prior to processing the unique identifier in a subsequent one of the ETL phases.
11 . The system of claim 10 , wherein the hash is based at least in part on information about the unique entity.
12 . The system of claim 10 , wherein the hash is based at least in part on a uniform resource locator associated with the unique entity.
13 . The system of claim 8 , wherein the supply chain data comprises global trade data, cyber data, and government spending data.
14 . The system of claim 8 , wherein the computer readable instructions controlling the processing device are further configured to perform operations comprising:
performing a supply chain analysis using the supply chain data updated with the unique identifier, wherein the supply chain analysis identifies one or more potential risks associated with the unique entity.
15 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:
receiving supply chain data from a plurality of data sources distributed across a network;
analyzing the supply chain data to identify an entity and a doing business as (DBA) name associated with the entity;
generating a unique identifier for the entity;
identifying a second DBA name associated with a second entity in the supply chain data;
comparing information about the DBA name and the second DBA name to determine whether the DBA name and the second DBA name are both associated with a same entity;
grouping the second DBA name and the second entity with the DBA name and the entity based on determining that the DBA name and the second DBA name are both associated with the same entity as a unique entity;
associating the unique identifier with the unique entity, the DBA name associated with the unique entity, and the second DBA name associated with the unique entity;
creating a data structure with a plurality of unique identifiers including the unique identifier to link multiple source data sets from the data sources as normalized dynamic data streams;
flowing updates from the source data sets based on data feeds of the data sources to the data structure through inheritance properties such that the updates are available at a predetermined interval; and
updating the data structure to include the DBA name associated with the unique entity, and the second DBA name associated with the unique entity, thereby providing entity resolution and consolidation using the unique identifier.
16 . The computer program product of claim 15 , wherein the DBA name is one of a plurality of DBA names associated with the unique entity, the unique identifier is a hash, and the hash is based at least in part on information about the unique entity and/or a uniform resource locator associated with the unique entity.
17 . The computer program product of claim 15 , wherein the supply chain data comprises global trade data, cyber data, and government spending data.
18 . The computer program product of claim 15 , wherein the program instructions executable by the processor to cause the processor to further perform operations comprising:
performing a supply chain analysis using the supply chain data updated with the unique identifier, wherein the supply chain analysis identifies one or more potential risks associated with the unique entity, and wherein the supply chain data is received from a plurality of sources.