SYSTEM AND METHOD FOR MODIFYING A KNOWLEDGE REPRESENTATION BASED ON A MACHINE LEARNING CLASSIFIER

1 . A method of modifying a knowledge representation based on a machine-learning classifier, the method comprising:

receiving a knowledge representation encoded as a non-transitory computer-readable data structure, based on an object of interest, the knowledge representation comprising at least one concept and/or relationship between two or more concepts;

receiving validation data, the validation data comprising a first set of one or more labeled content items having a label that classifies each content item into one or more categories including a first category known to be relevant to the object of interest and a second category known to not be relevant to the object of interest;

predicting, with a machine-learning classifier that uses at least one attribute derived from the knowledge representation as a feature, each of the one or more labeled content items as one of: a) relevant to the object of interest or b) not relevant to the object of interest; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

2 . The method of claim 1 , further comprising synthesizing the knowledge representation based on contents of the object of interest.

3 . The method of claim 2 , wherein the synthesizing further comprises generating the at least one concept and/or relationship between two or more concepts, wherein the concepts and/or relationships are not recited in the object of interest.

4 . The method of claim 1 , wherein the knowledge representation includes weights associated with the at least one concept.

5 . The method of claim 1 , wherein the predicting is based on an intersection of the one or more labeled content items and the feature.

6 . The method of claim 1 , wherein the object of interest comprises a topic, a tweet, a webpage, a website, a document, a collection of documents, a document title, a message, an advertisement, and/or a search query.

7 . The method of claim 1 , further comprising:

after modifying the knowledge representation:

re-predicting each of the first set of one or more labeled content items using the modified knowledge representation; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

8 . The method of claim 7 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of labeled content items in the first category is equal to or exceeds a precision threshold.

9 . The method of claim 7 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is equal to or exceeds a recall threshold.

10 . The method of claim 1 , wherein modifying the knowledge representation comprises modifying weights associated with the at least one concept in the knowledge representation, and/or adding additional concepts to the knowledge representation.

11 . The method of claim 1 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items in the first category is less than a threshold precision value.

12 . The method of claim 1 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is less than a threshold recall value.

13 . The method of claim 1 , wherein the at least one attribute comprises at least one of:

a total number of concepts intersecting between the knowledge representation and the one or more labeled content items,

a number of broader concepts intersecting between the knowledge representation and the one or more labeled content items,

a sum of weights of concepts intersecting between the knowledge representation and the one or more labeled content items, and/or

a number of narrower concepts intersecting between the knowledge representation and the one or more labeled content items.

14 . A system for modifying a knowledge representation based on a machine-learning classifier, the system comprising:

at least one processor configured to perform a method comprising:

receiving a knowledge representation encoded as a non-transitory computer-readable data structure, based on an object of interest, the knowledge representation comprising at least one concept and/or relationship between two or more concepts;

receiving validation data, the validation data comprising a first set of one or more labeled content items having a label that classifies each content item into one or more categories including a first category known to be relevant to the object of interest and a second category known to not be relevant to the object of interest;

predicting, with a machine-learning classifier that uses at least one attribute derived from the knowledge representation as a feature, each of the one or more labeled content items as one of: a) relevant to the object of interest or b) not relevant to the object of interest; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

15 . The system of claim 14 , wherein the method further comprises synthesizing the knowledge representation based on contents of the object of interest.

16 . The system of claim 15 , wherein the synthesizing further comprises generating the at least one concept and/or relationship between two or more concepts, wherein the concepts and/or relationships are not recited in the object of interest.

17 . The system of claim 14 , wherein the knowledge representation includes weights associated with the at least one concept.

18 . The system of claim 14 , wherein the predicting is based on an intersection of the one or more labeled content items and the feature.

19 . The system of claim 14 , wherein the object of interest comprises a topic, a tweet, a webpage, a website, a document, a collection of documents, a document title, a message, an advertisement, and/or a search query.

20 . The system of claim 14 , wherein the method further comprises:

after modifying the knowledge representation:

re-predicting each of the first set of one or more labeled content items using the modified knowledge representation; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

21 . The system of claim 20 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of labeled content items in the first category is equal to or exceeds a precision threshold.

22 . The system of claim 20 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is equal to or exceeds a recall threshold.

23 . The system of claim 14 , wherein modifying the knowledge representation comprises modifying weights associated with the at least one concept in the knowledge representation, and/or adding additional concepts to the knowledge representation.

24 . The system of claim 14 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items in the first category is less than a threshold precision value.

25 . The system of claim 14 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is less than a threshold recall value.

26 . The system of claim 14 , wherein the at least one attribute comprises at least one of:

a total number of concepts intersecting between the knowledge representation and the one or more labeled content items,

a number of broader concepts intersecting between the knowledge representation and the one or more labeled content items,

a sum of weights of concepts intersecting between the knowledge representation and the one or more labeled content items, and/or

a number of narrower concepts intersecting between the knowledge representation and the one or more labeled content items.

27 . At least one non-transitory computer readable storage medium storing processor-executable instructions that, when executed by at least one processor, cause the at least one processor to perform a method of modifying a knowledge representation based on a machine-learning classifier, the method comprising:

receiving a knowledge representation encoded as a non-transitory computer-readable data structure, based on an object of interest, the knowledge representation comprising at least one concept and/or relationship between two or more concepts;

receiving validation data, the validation data comprising a first set of one or more labeled content items having a label that classifies each content item into one or more categories including a first category known to be relevant to the object of interest and a second category known to not be relevant to the object of interest;

predicting, with a machine-learning classifier that uses at least one attribute derived from the knowledge representation as a feature, each of the one or more labeled content items as one of: a) relevant to the object of interest or b) not relevant to the object of interest; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

28 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the method further comprises synthesizing the knowledge representation based on contents of the object of interest.

29 . The at least one non-transitory computer readable storage medium of claim 28 , wherein the synthesizing further comprises generating the at least one concept and/or relationship between two or more concepts, wherein the concepts and/or relationships are not recited in the object of interest.

30 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the knowledge representation includes weights associated with the at least one concept.

31 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the predicting is based on an intersection of the one or more labeled content items and the feature.

32 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the object of interest comprises a topic, a tweet, a webpage, a website, a document, a collection of documents, a document title, a message, an advertisement, and/or a search query.

33 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the method further comprises:

after modifying the knowledge representation:

re-predicting each of the first set of one or more labeled content items using the modified knowledge representation; and

modifying the knowledge representation based on a comparison of the prediction by the machine-learning classifier for each content item of the first set to the label of each respective content item.

34 . The at least one non-transitory computer readable storage medium of claim 33 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of labeled content items in the first category is equal to or exceeds a precision threshold.

35 . The at least one non-transitory computer readable storage medium of claim 33 , wherein the re-predicting and the modifying are repeated until a ratio of a number of the one or more labeled content items correctly predicted as being relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is equal to or exceeds a recall threshold.

36 . The at least one non-transitory computer readable storage medium of claim 27 , wherein modifying the knowledge representation comprises modifying weights associated with the at least one concept in the knowledge representation, and/or adding additional concepts to the knowledge representation.

37 . The at least one non-transitory computer readable storage medium of claim 27 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items in the first category is less than a threshold precision value.

38 . The at least one non-transitory computer readable storage medium of claim 27 , wherein modifying the knowledge representation based on the comparing comprises modifying the knowledge representation when a ratio of a number of the one or more labeled content items correctly predicted to be relevant to the object of interest to a total number of the one or more labeled content items predicted to be relevant to the object of interest is less than a threshold recall value.

39 . The at least one non-transitory computer readable storage medium of claim 27 , wherein the at least one attribute comprises at least one of:

a total number of concepts intersecting between the knowledge representation and the one or more labeled content items,

a number of broader concepts intersecting between the knowledge representation and the one or more labeled content items,

a sum of weights of concepts intersecting between the knowledge representation and the one or more labeled content items, and/or

a number of narrower concepts intersecting between the knowledge representation and the one or more labeled content items.