Federated search of multiple sources with conflict resolution
View Patent ↗Methods and apparatuses related to federated search of multiple sources with conflict resolution are disclosed. A method may comprise obtaining a set of data ontologies (e.g., types, properties, and links) associated with a plurality of heterogeneous data sources; receiving a selection of a graph comprising a plurality of graph nodes connected by one or more graph edges; and transforming the graph into one or more search queries across the plurality of heterogeneous data sources. A method may comprise obtaining a first data object as a result of executing a first search query across a plurality of heterogeneous data sources; resolving, based on one or more resolution rules, at least the first data object with a repository data object; deduplicating data associated with at least the first data object and the repository data object prior to storing the deduplicated data in a repository that has a particular data model.
1. A method comprising:
obtaining a first data object as a result of executing a first search query against a first data source of a plurality of heterogeneous data sources;
obtaining a second data object as a result of executing a second search query against a second data source, that is not the first data source, of the plurality of heterogeneous data sources;
determining, based on one or more resolution rules, whether the first data object and the second data object represent similar objects or identical objects;
in response to determining that the first data object and the second data object represent similar objects or identical objects:
generating an intermediate data object based on grouping the first data object with the second data object;
generating a unique identifier for the intermediate data object based on hashing one or more data object properties that uniquely identify the intermediate data object;
determining whether a repository data object that shares the unique identifier is stored in a repository that has a particular data model;
in response to determining that the repository data object is not stored in the repository, generating a stub data object that is referenced by the unique identifier and that is stored in the repository;
resolving the intermediate data object with the stub data object;
deduplicating data associated with the intermediate data object and the stub data object;
storing the deduplicated data in the repository that has the particular data model;
wherein the method is performed by one or more computing devices.
2. The method of claim 1 , wherein the particular data model comprises an object-centric data model.
3. The method of claim 1 , wherein the one or more data object properties comprise a provenance identifier.
4. The method of claim 1 , wherein the second search query takes as input one or more results of the first search query.
5. The method of claim 1 , wherein a change to data in one of the plurality of heterogeneous data sources and a change to data in the repository are synchronized based on vector clocks, repository rankings, or data source rankings.
6. A system comprising:
one or more processors; and
one or more non-transitory storage media storing instructions which, when executed by the one or more processors, cause:
obtaining a first data object as a result of executing a first search query against a first data source of a plurality of heterogeneous data sources;
obtaining a second data object as a result of executing a second search query against a second data source, that is not the first data source, of the plurality of heterogeneous data sources;
determining, based on one or more resolution rules, whether the first data object and the second data object represent similar objects or identical objects;
in response to determining that the first data object and the second data object represent similar objects or identical objects:
generating an intermediate data object based on grouping the first data object with the second data object;
generating a unique identifier for the intermediate data object based on hashing one or more data object properties that uniquely identify the intermediate data object;
determining whether a repository data object that shares the unique identifier is stored in a repository that has a particular data model;
in response to determining that the repository data object is not stored in the repository, generating a stub data object that is referenced by the unique identifier and that is stored in the repository;
resolving the intermediate data object with the stub data object;
deduplicating data associated with the intermediate data object and the stub data object;
storing the deduplicated data in the repository that has the particular data model.
7. The system of claim 6 , wherein the particular data model comprises an object-centric data model.
8. The system of claim 6 , wherein the one or more data object properties comprise a provenance identifier.
9. The system of claim 6 , wherein the second search query takes as input one or more results of the first search query.
10. The system of claim 6 , wherein a change to data in one of the plurality of heterogeneous data sources and a change to data in the repository are synchronized based on vector clocks, repository rankings, or data source rankings.