System and method for building an evolving ontology from user-generated content

1. A method for constructing an evolving ontology database comprising:

receiving, by a computing device, a plurality of data entries;

calculating, by the computing device, semantic similarity scores between any two of the data entries based on feature sources and feature similarities of the data entries;

clustering, by the computing device, the data entries into a plurality of current themes based on the semantic similarity scores;

selecting, by the computing device, new concepts from the current themes by comparing the current themes with a plurality of previous themes prepared using previous data entries; and

updating, by the computing device, the evolving ontology database using the new concepts,

wherein the step of updating the evolving ontology database comprises:

detecting a parent concept by comparing at least one verified concept with concepts in the ontology;

computing similarity between the at least one verified concept and sibling concepts to obtain a most similar sibling concept from the sibling concepts, wherein the sibling concepts are child concepts of the parent concept, and the most similar sibling concept is the concept having a highest possibility among possibilities where the at least one verified concept belongs to the sibling concepts;

proposing ontology adjustments based on the parent concept and the most

similar sibling concept; and

using an optimal adjustment from proposed ontology adjustments to update the ontology, wherein, the optimal adjustment is an adjustment having a hierarchy as follows:

H

opt

=

arg

⁢

max

H

⁢

log

⁢

p

⁡

(

D

|

H

)

wherein H is a given ontology hierarchies, Q={H 1 , H 2 , . . . , H n } and D are a sequence of related adjusted hierarchies and related dataset used for training and testing.

2. The method of claim 1 , wherein the semantic similarity scores between any two of the data entries are calculated by:

score

=

∏

i

=

0

n

⁢

⁢

s

i

⁢

e

∑

j

=

0

k

⁢

w

j

×

f

j

,

semantic similarity, wherein s i is weight of the features sources, f j is one of the feature similarities between the two of the data entries, w j is a weight of f j , and j, k and n are positive integers.

3. The method of claim 2 , wherein the data entries are user generated feedbacks, and the step of calculating semantic similarity scores comprises:

predicting sentiment similarity values by a sentiment analyzer, the sentiment similarity values representing similarity between the two data entries in regard to positive feedback, negative feedback, neutral feedback, very negative feedback, and internet abuse;

predicting text similarity values by a similarity calculator, the text similarity values representing similarity between semantic meaning of text extracted from the two data entries; and

predicting syntactic similarity values by a neutral language parser, the syntactic similarity values representing syntactic complexity of the text of the two data entries.

4. The method of claim 3 , wherein the step of clustering the data entries further comprises:

calculating the semantic similarity score for the two data entries using the sentiment similarity values, the text similarity values, and the syntactic similarity values.

5. The method of claim 2 , wherein the step of selecting the new concepts from the current themes comprises:

retrieving the current themes and the previous themes;

identifying duplicate themes from the current themes and the previous themes;

removing the duplicated themes from the current themes to obtain non-duplicate themes;

comparing the non-duplicate themes to concepts in the ontology database to obtain novel concepts candidates, wherein the novel concepts candidates are the non-duplicate themes that have a similarity lower than a pre-determined threshold to any of the concepts in the ontology database; and

verifying the novel concepts candidates according to an instruction from a manager of the ontology database, to obtain the new concepts.

6. The method of claim 1 , wherein the proposed ontology adjustment comprises an insertion adjustment, and in the insertion adjustment, the new concept is defined as a child node of the parent concept.

7. The method of claim 1 , wherein the proposed ontology adjustment comprises a lift adjustment, and in the lift adjustment, the new concept is defined as a sibling node of the parent concept.

8. The method of claim 1 , wherein the proposed ontology adjustment comprises a shift adjustment, and in the shift adjustment, the new concept is defined as a child node of the most similar sibling concept.

9. The method of claim 1 , wherein the proposed ontology adjustment comprises a merge adjustment, and in the merge adjustment, the new concept is combined with the most similar sibling concept to form a combined concept, the combined concept is defined as a child node of the parent concept, and the new concept and the most similar sibling concept are defined as child nodes of the combined concept.

10. The method of claim 2 , wherein each concept in the ontology data base is defined by a classification model, and the classification model comprises a logistic regression model and a gradient boosting classifier.

11. The method of claim 10 , wherein the step of updating the evolving ontology database comprises updating labels of the plurality of data entries, and the method further comprises: tuning the classification model using the updated data entries.

12. A system for constructing an evolving ontology database, the system comprising a computing device, the computing device comprising a processor and a storage device storing computer executable code, wherein the computer executable code, when executed at the processor, is configured to:

receive a plurality of data entries;

calculate semantic similarity scores between any two of the data entries based on feature sources and feature similarities of the data entries;

cluster the data entries into a plurality of current themes based on the semantic similarity scores;

select new concepts from the current themes by comparing the current themes with a plurality of previous themes prepared using previous data entries; and

update the evolving ontology database using the new concepts,

wherein the computer executable code is configured to update the evolving ontology data base by:

detecting a parent concept by comparing at least one verified concept with concepts in the ontology;

computing similarity between the at least one verified concept and sibling concepts to obtain a most similar sibling concept from the sibling concepts, wherein the sibling concepts are child concepts of the parent concept, and the most similar sibling concept is the concept having a highest possibility among possibilities where the at least one verified concept belongs to the sibling concept;

proposing ontology adjustments based on the parent concept and the most similar sibling concept; and

using an optimal adjustment from proposed ontology adjustments to update the ontology, wherein, the optimal adjustment is an adjustment having a hierarchy as follows:

H

opt

=

arg

⁢

max

H

⁢

log

⁢

p

⁡

(

D

|

H

)

wherein His a given ontology hierarchies, Q={H 1 , H 2 , . . . , H n } and D are a sequence of related adjusted hierarchies and related dataset used for training and testing.

13. The system of claim 12 , wherein the semantic similarity scores between any two of the data entries are calculated by:

score

=

∏

i

=

0

n

⁢

⁢

s

i

⁢

e

∑

j

=

0

k

⁢

w

j

×

f

j

,

semantic similarity wherein s i is weight of features sources, f j is a feature similarity between the two of the data entries, w j is a weight of f j , and j, k and n are positive integers.

14. The system of claim 13 , wherein the data entries are user generated feedbacks, and the computer executable code is configured to calculate semantic similarity scores by:

predicting sentiment similarity values by a sentiment analyzer, the sentiment similarity values representing similarity between the two data entries in regard to positive feedback, negative feedback, neutral feedback, very negative feedback, and internet abuse;

predicting text similarity values by a similarity calculator, the text similarity values representing similarity between semantic meaning of text extracted from the two data entries; and

predicting syntactic similarity values by a neutral language parser, the syntactic similarity values representing syntactic complexity of the text of the two data entries.

15. The system of claim 13 , wherein the computer executable code is configured to select the new concepts from the current themes by:

retrieving the current themes and the previous themes;

identifying duplicate themes from the current themes and the previous themes;

removing the duplicated themes from the current themes to obtain non-duplicate themes;

comparing the non-duplicate themes to concepts in the ontology database to obtain novel concepts candidates, wherein the novel concepts candidates are the non-duplicate themes that have a similarity lower than a pre-determined threshold to any of the concepts in the ontology database; and

verifying the novel concepts candidates according to an instruction from a manager of the ontology database, to obtain the new concepts.

16. The system of claim 12 , wherein the proposed ontology adjustments comprises an insertion adjustment, a lift adjustment, a shift adjustment, and a merge adjustment, and the computer executable code is configured to:

in the insertion adjustment, define the new concept as a child node of the parent concept;

in the lift adjustment, define the new concept as a sibling node of the parent concept;

in the shift adjustment, defined the new concept as a child node of the most similar sibling concept; and

in the merging adjustment, combine the new concept with the most similar sibling concept to form a combined concept, define the combined concept as a child node of the parent concept, and define the new concept and the most similar sibling concept as child nodes of the combined concept.

17. A non-transitory computer readable medium storing computer executable code, wherein the computer executable code, when executed at a processor of a computing device, is configured to:

receive a plurality of data entries;

calculate semantic similarity scores between any two of the data entries;

cluster the data entries into a plurality of current themes based on the semantic similarity scores;

select new concepts from the current themes by comparing the current themes with a plurality of previous themes prepared using previous data entries; and

update the evolving ontology database using the new concepts,

wherein the computer executable code is configured to update the evolving ontology data base by:

detecting a parent concept by comparing at least one verified concept with concepts in the ontology;

computing similarity between the at least one verified concept and sibling concepts to obtain a most similar sibling concept from the sibling concepts, wherein the sibling concepts are child concepts of the parent concept, and the most similar sibling concept is the concept having a highest possibility among possibilities where the at least one verified concept belongs to the sibling concept;

proposing ontology adjustments based on the parent concept and the most similar sibling concept; and

using an optimal adjustment from proposed ontology adjustments to update the ontology, wherein, the optimal adjustment is an adjustment having a hierarchy as follows:

H

opt

=

arg

⁢

max

H

⁢

log

⁢

p

⁡

(

D

|

H

)

wherein H is a given ontology hierarchies, Q={H 1 , H 2 , . . . , H n } and D are a sequence of related adjusted hierarchies and related dataset used for training and testing.

18. The non-transitory computer readable medium of claim 17 , wherein the semantic similarity scores between any two of the data entries are calculated by:

score

=

∏

i

=

0

n

⁢

⁢

s

i

⁢

e

∑

j

=

0

k

⁢

w

j

×

f

j

,

semantic similarity, wherein s i is weight of features sources, f j is a feature similarity between the two of the data entries, w j is a weight of f j , and j, k and n are positive integers.