Identification of feature groups in feature graph databases
A processing system may apply a community detection process to a feature graph database to identify a plurality of communities of features, the feature graph database comprising: a plurality of objects, each associated with one of a feature or a concept, and a plurality of relationships between the plurality of objects. Next, the processing system may label a first plurality of features of the feature graph database with at least a first community label, where the first plurality of features comprises features of at least a first community of the plurality of communities. The processing system may then obtain a search associated with at least one feature of the feature graph database, where the at least one feature is a part of the at least the first plurality of features of the at least the first community, and provide the first plurality of features in response to the search.
1 . A method comprising:
applying, by a processing system including at least one processor, a community detection process to a feature graph database representing one or more data tables of telecommunication network operational data to identify a plurality of communities of features within the feature graph database, where each of the one or more data tables comprises a plurality of rows and a plurality of columns, wherein the feature graph database comprises:
a plurality of objects, each of the plurality of objects representing a feature of a plurality of features, wherein each of the plurality of features comprises a column of the one or more data tables, the column comprising a vector of values of a particular data type; and
a plurality of relationships between the plurality of objects, wherein the community detection process is applied only with respect to a plurality of feature-to-feature relationships of the plurality of relationships within the feature graph database, wherein each of the plurality of feature-to-feature relationships represents a column-to-column relationship between respective columns of the one or more data tables;
labeling, by the processing system, a first plurality of features of the feature graph database with at least a first community label, wherein the first plurality of features comprises features of at least a first community of the plurality of communities;
obtaining, by the processing system, a search associated with at least one feature of the feature graph database, wherein the at least one feature is a part of the at least the first plurality of features of the at least the first community; and
providing, by the processing system, the first plurality of features in response to the search for detecting duplicate features.
2 . The method of claim 1 , wherein at least a first portion of the plurality of feature-to-feature relationships is obtained from one or more users.
3 . The method of claim 2 , wherein at least a second portion of the plurality of feature-to-feature relationships is identified via at least one of: an ontological verification process or a fingerprint verification process.
4 . The method of claim 1 , wherein each of the feature-to-feature relationships is one of a plurality of feature-to-feature relationship types.
5 . The method 4 , wherein the plurality of feature-to-feature relationship types includes at least one of:
a related-to relationship type;
a subset-of-relationship type;
a derived-from relationship type;
a same-as-but-disjoint-from relationship type; or
a partially-overlapping relationship type.
6 . The method of claim 1 , wherein the labeling includes labeling a plurality of entries in a data dictionary associated with the feature graph database with the at least the first community label, wherein the plurality of entries is associated with the first plurality of features.
7 . The method of claim 6 , wherein the search is over the data dictionary.
8 . The method of claim 1 , wherein the providing comprises providing a list of the first plurality of features.
9 . The method of claim 1 , wherein the providing comprises providing a visualization of a sub-graph of the feature graph database, where the sub-graph comprises the first plurality of features.
10 . The method of claim 1 , further comprising:
assigning the at least the first community label to the least the first community of the plurality of communities.
11 . The method of claim 1 , wherein the community detection process is applied on a periodic basis.
12 . The method of claim 1 , wherein the community detection process is applied in response to an addition of at least one of: a threshold number of features or a threshold number of data sets to the feature graph database.
13 . The method of claim 1 , wherein the community detection process comprises a connected components process.
14 . The method of claim 13 , wherein the connected components process comprises:
a breadth-first search;
a depth-first search;
a union find; or
a distributed parallel union find.
15 . The method of claim 1 , wherein the community detection process comprises:
a label propagation community detection process;
a hierarchical clustering community detection process;
an eigenvector-based community detection process; or
a clique-based community detection process.
16 . The method of claim 1 , further comprising:
verifying the plurality of communities in accordance with a node similarity metric.
17 . The method of claim 16 , wherein the node similarity metric comprises:
a Jaccard measure; or
a vector embedding distance measure.
18 . A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
applying a community detection process to a feature graph database representing one or more data tables of telecommunication network operational data to identify a plurality of communities of features within the feature graph database, where each of the one or more data tables comprises a plurality of rows and a plurality of columns, wherein the feature graph database comprises:
a plurality of objects, each of the plurality of objects representing a feature of a plurality of features, wherein each of the plurality of features comprises a column of the one or more data tables, the column comprising a vector of values of a particular data type; and
a plurality of relationships between the plurality of objects, wherein the community detection process is applied only with respect to a plurality of feature-to-feature relationships of the plurality of relationships within the feature graph database, wherein each of the plurality of feature-to-feature relationships represents a column-to-column relationship between respective columns of the one or more data tables;
labeling a first plurality of features of the feature graph database with at least a first community label, wherein the first plurality of features comprises features of at least a first community of the plurality of communities;
obtaining a search associated with at least one feature of the feature graph database, wherein the at least one feature is a part of the at least the first plurality of features of the at least the first community; and
providing the first plurality of features in response to the search for detecting duplicate features.
19 . A device comprising:
a processor system including at least one processor; and
a computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
applying a community detection process to a feature graph database representing one or more data tables of telecommunication network operational data to identify a plurality of communities of features within the feature graph database, where each of the one or more data tables comprises a plurality of rows and a plurality of columns, wherein the feature graph database comprises:
a plurality of objects, each of the plurality of objects representing a feature of a plurality of features, wherein each of the plurality of features comprises a column of the one or more data tables, the column comprising a vector of values of a particular data type; and
a plurality of relationships between the plurality of objects, wherein the community detection process is applied only with respect to a plurality of feature-to-feature relationships of the plurality of relationships within the feature graph database, wherein each of the plurality of feature-to-feature relationships represents a column-to-column relationship between respective columns of the one or more data tables;
labeling a first plurality of features of the feature graph database with at least a first community label, wherein the first plurality of features comprises features of at least a first community of the plurality of communities;
obtaining a search associated with at least one feature of the feature graph database, wherein the at least one feature is a part of the at least the first plurality of features of the at least the first community; and
providing the first plurality of features in response to the search for detecting duplicate features.
20 . The device of claim 19 , wherein at least a first portion of the plurality of feature-to-feature relationships is obtained from one or more users.