Feature completion in computer-human interactive learning
View Patent ↗A collection of data that is extremely large can be difficult to search and/or analyze. Relevance may be dramatically improved by automatically classifying queries and web pages in useful categories, and using these classification scores as relevance features. A thorough approach may require building a large number of classifiers, corresponding to the various types of information, activities, and products. Creation of classifiers and schematizers is provided on large data sets. Exercising the classifiers and schematizers on hundreds of millions of items may expose value that is inherent to the data by adding usable meta-data. Some aspects include active labeling exploration, automatic regularization and cold start, scaling with the number of items and the number of classifiers, active featuring, and segmentation and schematization.
1. One or more hardware computer-storage media having embodied thereon computer-usable instructions that, when executed, facilitate a method of feature completion for machine learning, the method comprising:
storing a first set of data items, wherein each data item includes a text stream of words;
accessing a dictionary, wherein the dictionary includes a list of words that define a concept usable as an input feature for training a machine-learning model to score data items with a probability of being a positive example or a negative example of a particular class of data item;
providing a feature that is already trained to determine a probability that a word at a given word position corresponds semantically to the concept defined by the words in the dictionary; and
training the machine-learning model with the dictionary as an input feature, wherein the training includes
A) for the given word position in a text stream within a data item, utilizing the provided feature to calculate a first probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary,
B) examining a context of the given word position, wherein the context includes a number of words preceding the given word position and a number of words following the given word position, and wherein the context does not include the word at the given word position,
C) calculating a second probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary, based on a function of the words in the context of the given word position, wherein calculating the second probability comprises one or more of:
1) determining whether any words from a given list appear at a center of a window of text around the given word position in which center words in the window of text have been removed,
2) determining a presence or absence of a verb in the window,
3) determining a presence or absence of a noun followed by an adjective, or
4) determining a number of occurrences of a given word in the window, and
D) modifying the function to adjust the calculated second probability, based on the calculated first probability.
2. The media of claim 1 , wherein modifying the function to adjust the calculated second probability includes
A) modifying the function to increase the calculated second probability when the word at the given word position is in the dictionary, and
B) modifying the function to decrease the calculated second probability when the word at the given word position is not in the dictionary.
3. The media of claim 1 , wherein the machine-learning model includes at least one of a classifier and a schematizer.
4. The media of claim 1 , wherein the context is a sliding window that includes a number of words immediately preceding the given word position and a number of words immediately following the given word position.
5. The media of claim 1 , wherein the calculated first probability is an estimate of the first probability.
6. A method of feature completion for machine learning, comprising:
storing a first set of data items, wherein each data item includes a text stream of words;
accessing a dictionary, wherein the dictionary includes a list of words that define a concept usable as an input feature for training a machine-learning model to score data items with a probability of being a positive example or a negative example of a particular class of data item;
providing a feature that is already trained to determine a probability that a word at a given word position corresponds semantically to the concept defined by the words in the dictionary; and
training the machine-learning model with the dictionary as an input feature, wherein the training includes
A) for the given word position in a text stream within a data item, utilizing the provided feature to calculate a first probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary,
B) examining a context of the given word position, wherein the context includes a number of words preceding the given word position and a number of words following the given word position, and wherein the context does not include the word at the given word position,
C) calculating a second probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary, based on a function of the words in the context of the given word position, wherein calculating the second probability comprises one or more of:
1) determining whether any words from a given list appear at a center of a window of text around the given word position in which center words in the window of text have been removed,
2) determining a presence or absence of a verb in the window,
3) determining a presence or absence of a noun followed by an adjective, or
4) determining a number of occurrences of a given word in the window, and
D) modifying the function to adjust the calculated second probability, based on the calculated first probability.
7. The method of claim 6 , wherein modifying the function to adjust the calculated second probability includes
A) modifying the function to increase the calculated second probability when the word at the given word position is in the dictionary, and
B) modifying the function to decrease the calculated second probability when the word at the given word position is not in the dictionary.
8. The method of claim 6 , wherein the machine-learning model includes at least one of a classifier and a schematizer.
9. The method of claim 6 , wherein the context is a sliding window that includes a number of words immediately preceding the given word position and a number of words immediately following the given word position.
10. The method of claim 6 , wherein the calculated first probability is an estimate of the first probability.
11. The method of claim 6 , wherein the feature is a regular expression operating over strings to predict semantically matching positions in text within a string at each considered position.
12. A system for feature completion for machine learning, comprising:
one or more computer-storage media configured to store a first set of data items, wherein each data item includes a text stream of words;
one or more computer-storage media configured to store a dictionary; and
one or more computing devices configured to
A) access the dictionary, wherein the dictionary includes a list of words that define a concept usable as an input feature for training a machine-learning model to score data items with a probability of being a positive example or a negative example of a particular class of data item;
B) utilize a feature that is already trained to determine a probability that a word at a given word position corresponds semantically to the concept defined by the words in the dictionary; and
C) train the machine-learning model with the dictionary as an input feature, wherein the training includes
1) for the given word position in a text stream within a data item, utilize the provided feature to calculate a first probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary,
2) examine a context of the given word position, wherein the context includes a number of words preceding the given word position and a number of words following the given word position, and wherein the context does not include the word at the given word position,
3) calculate a second probability that the word at the given word position corresponds semantically to the concept defined by the words in the dictionary, based on a function of the words in the context of the given word position, wherein calculate the second probability comprises one or more of:
i) determine whether any words from a given list appear at a center of a window of text around the given word position in which center words in the window of text have been removed,
ii) determine a presence or absence of a verb in the window,
iii) determine a presence or absence of a noun followed by an adjective, or
iv) determine a number of occurrences of a given word in the window, and
4) modify the function to adjust the calculated second probability, based on the calculated first probability.
13. The system of claim 12 , wherein modify the function to adjust the calculated second probability includes
A) modify the function to increase the calculated second probability when the word at the given word position is in the dictionary, and
B) modify the function to decrease the calculated second probability when the word at the given word position is not in the dictionary.
14. The system of claim 12 , wherein the machine-learning model includes at least one of a classifier and a schematizer.
15. The system of claim 12 , wherein the context is a sliding window that includes a number of words immediately preceding the given word position and a number of words immediately following the given word position.
16. The system of claim 12 , wherein the calculated first probability is an estimate of the first probability.
17. The system of claim 12 , wherein the feature is a regular expression operating over strings to predict semantically matching positions in text within a string at each considered position.