Methods and apparatus for a semantic multi-database data lake
Methods and apparatus are provided for integrating a plurality of different database types in a semantic multi-database data lake. An exemplary method comprises providing a plurality of databases having different database types; translating ontology definition language database commands obtained from a user into a plurality of data definition language and/or data manipulation language commands supported by the different database types in order to replicate data from the user to each of the different database types; obtaining a query specified in a query language of a given database; and delegating the query to the given database. A plurality of cluster gateways optionally manage a corresponding plurality of clusters of database instances and wherein queries are delegated to a given database instance by delegating the queries to the appropriate cluster gateway. Dark data that was not queried by any supported query language in a predefined period of time can be detected.
1. A method, comprising the steps of:
providing a plurality of databases having different database types;
obtaining one or more user-provided ontology definition language database commands from a user;
replicating data obtained from said user for storage in one or more of said databases of each of said different database types by translating said obtained user-provided ontology definition language database commands into a plurality of database type-specific ontology definition language database commands supported by each of said different database types, wherein the plurality of database type-specific ontology definition language database commands employ one or more of: (i) a database type-specific data definition language and a (ii) database type-specific data manipulation language supported by each of said different database types, wherein said one or more database type-specific ontology definition language database commands for a first one of the databases of a first database type comprise database type-specific ontology definition language database commands supported by said first database type;
obtaining, subsequent to the translating, one or more queries from the user specified in a query language of a given database of the plurality of databases; and
delegating the one or more queries obtained from the user to the given database of the plurality of databases having different database types, without translating the query language of the one or more queries, wherein the one or more queries are delegated to the given database having a given database type that supports the query language of the one or more queries, and wherein the data responsive to the one or more queries is retrieved from the given database.
2. The method of claim 1 , wherein a given database of a given database type comprises a cluster of one or more database instances of said database type.
3. The method of claim 1 , wherein said ontology definition language database commands comprise Resource Description Framework (RDF)-based commands.
4. The method of claim 1 , wherein said user-provided ontology definition language database commands and said database type-specific ontology definition language database commands comprise one or more of insertion, update and removal commands.
5. The method of claim 1 , wherein said query language comprises one of a plurality of supported query languages.
6. The method of claim 1 , further comprising the step of detecting data that was not queried by any supported query language for a predefined period of time.
7. The method of claim 1 , wherein said different database types comprise two or more of a relational database, a text database, an object-oriented database, a document-oriented database, a key-value database, a knowledge base, a real-time database, an XML database, a spatial database, a time series database, a temporal database and a graph database.
8. The method of claim 2 , further comprising the step of providing a plurality of cluster gateways to manage a plurality of said clusters of said one or more database instances of said database type and wherein said step of delegating the one or more queries to the given database further comprises delegating the one or more queries to the appropriate cluster gateway of the given database.
9. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
providing a plurality of databases having different database types;
obtaining one or more user-provided ontology definition language database commands from a user;
replicating data obtained from said user for storage in one or more of said databases of each of said different database types by translating said obtained user-provided ontology definition language database commands into a plurality of database type-specific ontology definition language database commands supported by each of said different database types, wherein the plurality of database type-specific ontology definition language database commands employ one or more of: (i) a database type-specific data definition language and a (ii) database type-specific data manipulation language supported by each of said different database types, wherein said one or more database type-specific ontology definition language database commands for a first one of the databases of a first database type comprise database type-specific ontology definition language database commands supported by said first database type;
obtaining, subsequent to the translating, one or more queries from the user specified in a query language of a given database of the plurality of databases; and
delegating the one or more queries obtained from the user to the given database of the plurality of databases having different database types, without translating the query language of the one or more queries, wherein the one or more queries are delegated to the given database having a given database type that supports the query language of the one or more queries, and wherein the data responsive to the one or more queries is retrieved from the given database.
10. The computer program product of claim 9 , wherein a given database of a given database type comprises a cluster of one or more database instances of said database type.
11. The computer program product of claim 9 , wherein said user-provided ontology definition language database commands and said database type-specific ontology definition language database commands comprise one or more of insertion, update and removal commands.
12. The computer program product of claim 9 , wherein said query language comprises one of a plurality of supported query languages.
13. The computer program product of claim 9 , further comprising the step of detecting data that was not queried by any supported query language for a predefined period of time.
14. The computer program product of claim 10 , further comprising the step of providing a plurality of cluster gateways to manage a plurality of said clusters of said one or more database instances of said database type and wherein said step of delegating the one or more queries to the given database further comprises delegating the one or more queries to the appropriate cluster gateway of the given database.
15. A system, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
providing a plurality of databases having different database types;
obtaining one or more user-provided ontology definition language database commands from a user;
replicating data obtained from said user for storage in one or more of said databases of each of said different database types by translating said obtained user-provided ontology definition language database commands into a plurality of database type-specific ontology definition language database commands supported by each of said different database types, wherein the plurality of database type-specific ontology definition language database commands employ one or more of: (i) a database type-specific data definition language and a (ii) database type-specific data manipulation language supported by each of said different database types, wherein said one or more database type-specific ontology definition language database commands for a first one of the databases of a first database type comprise database type-specific ontology definition language database commands supported by said first database type;
obtaining, subsequent to the translating, one or more queries from the user specified in a query language of a given database of the plurality of databases; and
delegating the one or more queries obtained from the user to the given database of the plurality of databases having different database types, without translating the query language of the one or more queries, wherein the one or more queries are delegated to the given database having a given database type that supports the query language of the one or more queries, and wherein the data responsive to the one or more queries is retrieved from the given database.
16. The system of claim 15 , wherein a given database of a given database type comprises a cluster of one or more database instances of said database type.
17. The system of claim 15 , wherein said user-provided ontology definition language database commands and said database type-specific ontology definition language database commands comprise one or more of insertion, update and removal commands.
18. The system of claim 15 , wherein said query language comprises one of a plurality of supported query languages.
19. The system of claim 15 , further comprising the step of detecting data that was not queried by any supported query language for a predefined period of time.
20. The system of claim 16 , further comprising the step of providing a plurality of cluster gateways to manage a plurality of said clusters of said one or more database instances of said database type and wherein said step of delegating the one or more queries to the given database further comprises delegating the one or more queries to the appropriate cluster gateway of the given database.