Managing data objects for graph-based data structures
Various embodiments provide methods, systems, apparatus, computer program products, and/or the like for managing, ingesting, monitoring, updating, and/or extracting/retrieving information/data associated with an electronic record (ER) stored in an ER data store and/or accessing information/data from the ER data store, wherein the ERs are generated, updated/modified, and/or accessed via a graph-based domain ontology.
1. A method for identifying an electronic record in a database corresponding to a subject entity, the method comprising:
automatically receiving, by a computing entity, a message via one or more communications interfaces, wherein the message (a) comprises data corresponding to the subject entity, and (b) is received from a source system;
identifying, by the computing entity, a first attribute and a second attribute for the subject entity from the message;
generating, by the computing entity, an application programming interface (API) request to an identity matching service, wherein (a) the API request comprises the first attribute and the second attribute, and (b) responsive to the API request, the identity matching service:
accesses a rule set from a plurality of rule sets based at least in part on the source system,
generates a first query based at least in part on the rule set and the first attribute,
queries a database, using the first query, for an electronic record stored in the database corresponding to the first attribute,
responsive to an electronic record not being identified based at least in part on the first attribute, generates a second query based at least in part on the rule set and the second attribute,
queries the database, using the second query, for an electronic record stored in the database corresponding to the second attribute, wherein the electronic record comprises an entity identifier corresponding to the subject entity, and
responsive to an electronic record being identified based at least in part on the second attribute, generates an API response comprising the entity identifier corresponding to the subject entity;
receiving, by the computing entity, the API response comprising the entity identifier corresponding to the subject entity;
automatically generating, by the computing entity, a data artifact packet data object, wherein (a) the data artifact packet data object comprises the entity identifier, (b) the data artifact packet data object is generated based at least in part on an observable packet data object, and (c) the observable packet data object is an XML, document generated based at least in part on parsing a message received from a source system;
providing, by the computing entity, the data artifact packet data object corresponding to the first entity identifier to an ingestion processing module for processing;
automatically generating, by the computing entity, a container tree data structure comprising a data artifact packet container node as a root node based at least in part on the data artifact packet data object, wherein (a) the container tree data structure is generated based at least in part on the data artifact packet data object, (b) the container tree data structure comprises a plurality of container nodes that are descendants of the root node based at least in part on the data artifact packet data object, (c) each container node of the plurality of container nodes corresponds to one pair of the plurality of observable-value pairs, and (d) automatically generating the container tree data structure comprises:
(i) determining a type of container node for an observable-value pair of the plurality of observable-value pairs based at least in part on the graph-based domain ontology,
(ii) determining whether a container node having the determined type is present in the container tree data structure, and
(iii) responsive to determining that a container node having the determined type is not present in the container tree data structure:
(1) constructing the container node having the determined type,
(2) inserting the container node into an appropriate position in the container tree data structure, wherein the appropriate position in the container tree data structure is determined based at least in part on the graph-based domain ontology, and
(3) storing the observable-value pair in the container node;
automatically identifying, by the computing entity, an entity associated with the container tree data structure based at least in part on the entity identifier;
automatically traversing the container tree data structure in a depth-first traversal to generate a data transfer object for each of container node of the plurality of container nodes, wherein (a) each data transfer object corresponds to one pair of the plurality of observable-value pairs, and (b) the depth-first traversal of the container tree data structure comprises comparing a value of an observable-value pair of the plurality of observable-value pairs to at least one requirement corresponding to an observable of the observable-value pair to determine whether the value satisfies the at least one requirement; and
automatically providing, by the computing entity, at least one of the plurality of data transfer objects for use in performing a database update function.
2. The method of claim 1 further comprising:
automatically storing, by the computing entity, the data artifact packet data object in a database, wherein the data artifact packet data object is identifiable by the corresponding entity identifier;
automatically detecting, by the computing entity, a user interaction event corresponding to a first entity identifier; and
responsive to detecting the user interaction event corresponding to the first entity identifier, automatically retrieving, by the computing entity, the data artifact packet data object corresponding to the first entity identifier.
3. The method of claim 1 , wherein at least one container node comprises at least one of a source vocabulary description, source vocabulary code, or text description of an observable, and during the depth-first traversal of the container tree data structure, an aggregator method resolves the at least one of the source vocabulary description, the source vocabulary code, or the text description of the observable.
4. The method of claim 1 , wherein the electronic record is identified based at least in part on the second attribute when the electronic record comprises a record attribute that (a) corresponds to the second attribute and (b) has a same value as the second attribute.
5. A system comprising one or more processors, one or more memory storage areas comprising program code, the one or more memory storage areas and the program code configured to, with the one or more processors, cause the system to at least:
automatically receive a message via one or more communications interfaces, wherein the message (a) comprises data corresponding to the subject entity, and (b) is received from a source system;
identify a first attribute and a second attribute for the subject entity from the message;
generate an application programming interface (API) request to an identity matching service, wherein (a) the API request comprises the first attribute and the second attribute, and (b) responsive to the API request, the identity matching service:
accesses a rule set from a plurality of rule sets based at least in part on the source system,
generates a first query based at least in part on the rule set and the first attribute,
queries a database, using the first query, for an electronic record stored in the database corresponding to the first attribute,
responsive to an electronic record not being identified based at least in part on the first attribute, generates a second query based at least in part on the rule set and the second attribute,
queries the database, using the second query, for an electronic record stored in the database corresponding to the second attribute, wherein the electronic record comprises an entity identifier corresponding to the subject entity, and
responsive to an electronic record being identified based at least in part on the second attribute, generates an API response comprising the entity identifier corresponding to the subject entity;
receive the API response comprising the entity identifier corresponding to the subject entity; and
automatically generate a data artifact packet data object, wherein (a) the data artifact packet data object comprises the entity identifier, (b) the data artifact packet data object is generated based at least in part on an observable packet data object, and (c) the observable packet data object is an XML document generated based at least in part on parsing a message received from a source system;
provide the data artifact packet data object corresponding to the first entity identifier to an ingestion processing module for processing;
automatically generate a container tree data structure comprising a data artifact packet container node as a root node based at least in part on the data artifact packet data object, wherein (a) the container tree data structure is generated based at least in part on the data artifact packet data object, (b) the container tree data structure comprises a plurality of container nodes that are descendants of the root node based at least in part on the data artifact packet data object, (c) each container node of the plurality of container nodes corresponds to one pair of the plurality of observable-value pairs, and (d) automatically generating the container tree data structure comprises:
(i) determining a type of container node for an observable-value pair of the plurality of observable-value pairs based at least in part on the graph-based domain ontology,
(ii) determining whether a container node having the determined type is present in the container tree data structure, and
(iii) responsive to determining that a container node having the determined type is not present in the container tree data structure:
(1) constructing the container node having the determined type,
(2) inserting the container node into an appropriate position in the container tree data structure, wherein the appropriate position in the container tree data structure is determined based at least in part on the graph-based domain ontology, and
(3) storing the observable-value pair in the container node;
automatically identifying, by the computing entity, an entity associated with the container tree data structure based at least in part on the entity identifier;
automatically traverse the container tree data structure in a depth-first traversal to generate a data transfer object for each of container node of the plurality of container nodes, wherein (a) each data transfer object corresponds to one pair of the plurality of observable-value pairs, and (b) the depth-first traversal of the container tree data structure comprises comparing a value of an observable-value pair of the plurality of observable-value pairs to at least one requirement corresponding to an observable of the observable-value pair to determine whether the value satisfies the at least one requirement; and
automatically provide at least one of the plurality of data transfer objects for use in performing a database update function.
6. The system of claim 5 , wherein the one or more memory storage areas and the program code configured to, with the one or more processors, further cause the system to:
automatically store the data artifact packet data object in a database, wherein the data artifact packet data object is identifiable by the corresponding entity identifier; and
automatically detect a user interaction event corresponding to a first entity identifier; and
responsive to detecting the user interaction event corresponding to the first entity identifier, automatically retrieve the data artifact packet data object corresponding to the first entity identifier.
7. The system of claim 6 , wherein at least one container node comprises at least one of a source vocabulary description, source vocabulary code, or text description of an observable, and during the depth-first traversal of the container tree data structure, an aggregator method resolves the at least one of the source vocabulary description, the source vocabulary code, or the text description of the observable.
8. The system of claim 5 , wherein the electronic record is identified based at least in part on the second attribute when the electronic record comprises a record attribute that (a) corresponds to the second attribute and (b) has a same value as the second attribute.
9. A computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code stored therein, the computer-readable program code configured to at least:
automatically receive a message via one or more communications interfaces, wherein the message (a) comprises data corresponding to the subject entity, and (b) is received from a source system;
identify a first attribute and a second attribute for the subject entity from the message;
generate an application programming interface (API) request to an identity matching service, wherein (a) the API request comprises the first attribute and the second attribute, and (b) responsive to the API request, the identity matching service:
accesses a rule set from a plurality of rule sets based at least in part on the source system,
generates a first query based at least in part on the rule set and the first attribute,
queries a database, using the first query, for an electronic record stored in the database corresponding to the first attribute,
responsive to an electronic record not being identified based at least in part on the first attribute, generates a second query based at least in part on the rule set and the second attribute,
queries the database, using the second query, for an electronic record stored in the database corresponding to the second attribute, wherein the electronic record comprises an entity identifier corresponding to the subject entity, and
responsive to an electronic record being identified based at least in part on the second attribute, generates an API response comprising the entity identifier corresponding to the subject entity;
receive the API response comprising the entity identifier corresponding to the subject entity; and
automatically generate a data artifact packet data object, wherein (a) the data artifact packet data object comprises the entity identifier, (b) the data artifact packet data object is generated based at least in part on an observable packet data object, and (c) the observable packet data object is an XML document generated based at least in part on parsing a message received from a source system;
provide the data artifact packet data object corresponding to the first entity identifier to an ingestion processing module for processing;
automatically generate a container tree data structure comprising a data artifact packet container node as a root node based at least in part on the data artifact packet data object, wherein (a) the container tree data structure is generated based at least in part on the data artifact packet data object, (b) the container tree data structure comprises a plurality of container nodes that are descendants of the root node based at least in part on the data artifact packet data object, (c) each container node of the plurality of container nodes corresponds to one pair of the plurality of observable-value pairs, and (d) automatically generating the container tree data structure comprises:
(i) determining a type of container node for an observable-value pair of the plurality of observable-value pairs based at least in part on the graph-based domain ontology,
(ii) determining whether a container node having the determined type is present in the container tree data structure, and
(iii) responsive to determining that a container node having the determined type is not present in the container tree data structure:
(1) constructing the container node having the determined type,
(2) inserting the container node into an appropriate position in the container tree data structure, wherein the appropriate position in the container tree data structure is determined based at least in part on the graph-based domain ontology, and
(3) storing the observable-value pair in the container node;
automatically identifying, by the computing entity, an entity associated with the container tree data structure based at least in part on the entity identifier;
automatically traverse the container tree data structure in a depth-first traversal to generate a data transfer object for each of container node of the plurality of container nodes, wherein (a) each data transfer object corresponds to one pair of the plurality of observable-value pairs, and (b) the depth-first traversal of the container tree data structure comprises comparing a value of an observable-value pair of the plurality of observable-value pairs to at least one requirement corresponding to an observable of the observable-value pair to determine whether the value satisfies the at least one requirement; and
automatically provide at least one of the plurality of data transfer objects for use in performing a database update function.
10. The computer program product of claim 9 , wherein the computer-readable program code is further configured to:
automatically store the data artifact packet data object in a database, wherein the data artifact packet data object is identifiable by the corresponding entity identifier; and
automatically detect a user interaction event corresponding to a first entity identifier; and
responsive to detecting the user interaction event corresponding to the first entity identifier, automatically retrieve the data artifact packet data object corresponding to the first entity identifier.
11. The computer program product of claim 9 , wherein at least one container node comprises at least one of a source vocabulary description, source vocabulary code, or text description of an observable, and during the depth-first traversal of the container tree data structure, an aggregator method resolves the at least one of the source vocabulary description, the source vocabulary code, or the text description of the observable.
12. The computer program product of claim 9 , wherein the electronic record is identified based at least in part on the second attribute when the electronic record comprises a record attribute that (a) corresponds to the second attribute and (b) has a same value as the second attribute.