Optimization filters for user generated content searches
Embodiments are directed towards filtering from a user generated content (UGC) search result those documents determined to have insufficient subject matter specificity as defined by a training of a classification filter. The training comprises selecting a set of UGC that is definable as having sufficient subject matter specificity (a good set), and another set of UGC that is definable as having insufficient subject matter specificity (a bad set). The trained UGC classifier may examine search documents, and based on the documents having values above a defined threshold categorize the document as having sufficient subject matter specificity (or not). Those documents having insufficient subject matter specificity based on their determined thresholds may be filtered out of the submitted UGC search results. The documents remaining within the UGC search results may then be provided to a searcher for display at a client device.
1. A network device, comprising:
a transceiver to send and receive data over a network; and
a processor that is operative to perform actions, including:
determining a plurality of document features for each document in a plurality of documents, wherein the plurality of documents include at least one document defined as having sufficient subject matter specificity, and at least one other document having insufficient subject matter specificity;
training a document classifier based at least on the plurality of document features including at least a frequency of visual line breaks and a frequency of images for the at least one document having sufficient subject matter specificity;
providing to the trained document classifier, the at least one other document, wherein the trained document classifier determines a first quality value for the at least one other document;
if the first determined quality value of the at least one other document is above a quality threshold value, identifying the at least one other document to have sufficient subject matter specificity, and providing the at least one other document to a client device for display;
if the first determined quality value of the at least one other document is at an unacceptable level, re-training the document classifier based on at least the plurality of features for the at least one other document; and
if a second quality value of the at least one other document is determined by the retrained document classifier to be above the quality threshold, providing the at least one other document to the client device for display.
2. The network device of claim 1 , wherein the plurality of document features further comprises at least one of a word count, a number of phonetics, a frequency of special characters, a number of special characters, and a frequency of function words, wherein the frequencies are determined based in part on the word count.
3. The network device of claim 1 , wherein the document classifier comprises a decision tree that is trained based on the plurality of document features for each document in the plurality of documents.
4. The network device of claim 1 , wherein identifying the at least one other document to have sufficient subject matter specificity further comprises traversing a decision tree, and combining feature scores as the decision tree is traversed.
5. The network device of claim 1 , wherein the at least one other document comprises user generated content documents obtained during a search query.
6. The network device of claim 1 , wherein the processor that is operative to perform actions, including:
re-training the document classifier based at least in part on user feedback.
7. A system that is operative to manage a search query over a network, comprising:
one or more processors having a search tool component configured to receive a search query request and to provide a plurality of documents in response; and
one or more memory devices that store a document classifier that is configured to detect a document's subject matter specificity based on a plurality of document features about the document, wherein the document classifier performs actions, including:
receiving a plurality of documents from the search tool; and
for each document in the plurality of documents:
extracting from each respective document, a plurality of document features including at least a frequency of visual line breaks and a frequency of images;
employing a machine learning algorithm to generate a document quality feature score for the respective document based on the extracted document features; and
if the document quality feature score for the respective document is below a threshold value, removing the respective document from the received plurality of documents as indicating that the document has insufficient subject matter specificity; and
providing a resulting set of at least one document to a search requestor based on each document quality feature score, such that the at least one provided document is determined to have sufficient subject matter specificity; and
if the first determined quality value of the at least one document is at an unacceptable level, re-training the document classifier based on at least the plurality of features for the at least one document; and
if a second quality value of the at least one other document is determined by the retrained document classifier to be above the quality threshold, providing the at least one document to the client device for display.
8. The system of claim 7 , wherein the document classifier is generated based on a training process that comprises:
selecting a first set of documents that are identified as having sufficient subject matter specificity based at least on one feature characteristic of each document;
selecting a second set of documents that are identified as having insufficient subject matter specificity based at least on one feature characteristic of each document;
employing the first and second sets of documents to generate the machine learning algorithm.
9. The system of claim 7 , wherein receiving a plurality of documents from the search tool further comprises receiving a plurality of documents associated with user generated content, wherein at least one document is associated with a blog site.
10. The system of claim 7 , wherein the machine learning algorithm comprises at least one of a decision tree, naive Bayes algorithm, Support-Vector machine, logistic regression algorithm, a perceptron, a Markovian discrimination based algorithm, or a neural network.
11. The system of claim 7 , wherein the document features further comprise at least one of a word count, a number of phonetics within a document, a frequency of special characters, a number of non-word symbols, and a frequency of function words.
12. A non-transitory computer-readable storage medium having computer-executable instructions, the computer-executable instructions when installed onto a computing device enable the computing device to perform actions, comprising:
receiving a plurality of documents in response to a search query; and
for each document in the plurality of documents:
extracting from each respective document, a plurality of document features including at least a frequency of visual line breaks and a frequency of images;
employing a document classifier to generate a document quality feature score for the respective document based on the extracted document features;
if the document quality feature score for the respective document is below a threshold value, removing the respective document from the received plurality of documents as indicating that the document has insufficient subject matter specificity; and
providing a resulting set of at least one document to a search requestor based on each document quality feature score, such that the at least one provided document is determined to have sufficient subject matter specificity; and
if the first determined quality value of the at least one document is at an unacceptable level, re-training the document classifier based on at least the plurality of features for the at least one document; and
if a second quality value of the at least one other document is determined by the retrained document classifier to be above the quality threshold, providing the at least one document to the client device for display.
13. The non-transitory computer-readable storage medium of claim 12 , wherein employing the document classifier further comprises:
traversing a decision tree that is generated based on a training process using a plurality of training documents, wherein each of training documents are labeled as to a respective quality for subject matter specificity; and
generating the overall document quality feature score based combining individual document feature scores determined as the decision tree is traversed.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the training process further comprises:
extracting from each of the plurality of training documents at least one document feature;
labeling each training document based on a quality for subject matter specificity; and
generating the decision tree, including a decision flow, and feature scores using the labeled training documents and extracted document features.
15. The non-transitory computer-readable storage medium of claim 12 , wherein the document features further comprise at least one of a word count, a number of phonetics within a document, a frequency of special characters, a number of non-word symbols, and a frequency of function words.
16. The non-transitory computer-readable storage medium of claim 12 , wherein the search query is performed exclusively over user generated content sites.
17. A method of performing a search query, comprising:
receiving a plurality of documents in response to the search query over user generated content (UGC); and
for each document in the plurality of documents:
extracting document features including at least a frequency of visual line breaks and a frequency of images from each respective document useable to identify subject matter specificity of a UGC document;
employing a document classifier to generate a first document quality feature score for the respective document based on the extracted document features; and
if the first document quality feature score for the respective document is below a threshold value, removing the respective document from the received plurality of documents as indicating that the document has insufficient subject matter specificity; and
providing a resulting set of at least one document to a search requestor based on each document quality feature score, such that the at least one provided document is determined to have sufficient subject matter specificity; and
if the first determined quality value of the at least one document is at an unacceptable level, re-training the document classifier based on at least the plurality of features for the at least one document; and
if a second quality value of the at least one document is determined by the retrained document classifier to be above the quality threshold, providing the at least other document to the client device for display.
18. The method of claim 17 , wherein the UGC further comprises content from at least one blog site.
19. The method of claim 17 , wherein the document classifier comprises a machine learning algorithm that is trained based on another set of UGC documents, wherein each of UGC document in the other set are labeled with a subject matter specificity quality value.
20. The method of claim 17 , further comprising:
receiving user feedback regarding a subject matter specificity of the result set of documents; and employing the user feedback to modify the document classifier.