UNIVERSAL DATA INDEX FOR RAPID DATA EXPLORATION
Embodiments of the invention provide a novel and non-obvious method, system and computer program product for universal data index construction. In an embodiment of the invention, a universal data index construction method includes establishing a communicative coupling to a database by way of a database management system. The method additionally includes creating in an index in memory of a host computer, a union of field values in all columns of the database, with all meta-data for the columns of the database. In this regard, the index associates each of the values and each of the meta-data with a specific location in the database. The method further includes adding to the index, pair-wise field values as a co-occurrence list. Finally, the method includes issuing a query to the index without issuing a SQL WHERE statement to the database management system in order to produce a filtered query result.
1 . A universal data index generation and utilization method comprising:
establishing a communicative coupling to a database by way of a database management system;
creating in an index in memory of a host computer, a union of field values in all columns of the database, with all meta-data for the columns of the database, the index associating each of the values and each of the meta-data with a specific location in the database;
adding to the index, pair-wise field values as a co-occurrence list; and,
issuing a query to the index without issuing a structured query language (SQL) WHERE statement to the database management system in order to produce a filtered query result.
2 . The method of claim 1 , wherein the meta-data includes a name for each of the columns.
3 . The method of claim 2 , wherein the query includes a query term associated with a column name of one of the columns.
4 . The method of claim 2 , wherein the query includes a field value and produces a name of a column containing the field value as a reverse lookup of the name of the column based upon the field value.
5 . The method of claim 1 , wherein the query includes a first field value and the issuing of the query produces a second field value from a corresponding one of the pair-wise field values of the co-occurrence list so as to locate the second field value as co-occurring with the first field value.
6 . A data analytics data processing system configured for universal data index construction comprising:
a host computing system comprising one or more computers, each with memory and at least one processor;
a database index persistent in the memory of the host computing system; and, a universal data index construction module comprising computer program instructions executing in the memory of the host computing system, the program instructions performing:
establishing a communicative coupling to a database by way of a database management system;
creating in the database index in the memory of the host computing system, a union of field values in all columns of a database, with all meta-data for the columns of the database, the index associating each of the values and each of the meta-data with a specific location in the database;
adding to the database index, pair-wise field values as a co-occurrence list; and,
issuing a query to the database index without issuing a structured query language (SQL) WHERE statement to the database management system in order to produce a filtered query result.
7 . The system of claim 6 , wherein the meta-data includes a name for each of the columns.
8 . The system of claim 7 , wherein the query includes a query term associated with a column name of one of the columns.
9 . The system of claim 7 , wherein the query includes a field value and produces a name of a column containing the field value as a reverse lookup of the name of the column based upon the field value.
10 . The system of claim 6 , wherein the query includes a first field value and the issuing of the query produces a second field value from a corresponding one of the pair-wise field values of the co-occurrence list so as to locate the second field value as co-occurring with the first field value.
11 . A computer program product for universal data index construction, the computer program product including a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to perform a method including:
establishing a communicative coupling to a database by way of a database management system;
creating in an index in memory of a host computer, a union of field values in all columns of the database, with all meta-data for the columns of the database, the index associating each of the values and each of the meta-data with a specific location in the database;
adding to the index, pair-wise field values as a co-occurrence list; and,
issuing a query to the index without issuing a structured query language (SQL) WHERE statement to the database management system in order to produce a filtered query result.
12 . The computer program product of claim 11 , wherein the meta-data includes a name for each of the columns.
13 . The computer program product of claim 12 , wherein the query includes a query term associated with a column name of one of the columns.
14 . The computer program product of claim 12 , wherein the query includes a field value and produces a name of a column containing the field value as a reverse lookup of the name of the column based upon the field value.
15 . The computer program product of claim 11 , wherein the query includes a first field value and the issuing of the query produces a second field value from a corresponding one of the pair-wise field values of the co-occurrence list so as to locate the second field value as co-occurring with the first field value.