Log-aided automatic query expansion approach based on topic modeling

1. A method comprising:

obtaining a base query having a plurality of base query terms;

accessing a plurality of problem log files comprising error messages;

extracting words, contained in a corpus vocabulary, from the error messages of the plurality of problem log files;

based on the words extracted from the error messages of the plurality of problem log files, generating a first expanded query from the base query; and

querying the corpus, via a query engine and a corpus index, with a second expanded query related to the first expanded query.

2. The method of claim 1 , further comprising determining that none of the query terms is in the corpus vocabulary; wherein the generating comprises, responsive to the determining that none of the query terms is in the corpus vocabulary:

picking one or more most recent ones of the words extracted from the error messages of the plurality of problem log files, having highest relevance to a single topic cluster in the log files, based on a topic model of said corpus; and

replacing the base query with at least one of the words having said highest relevance, to obtain the first expanded query; and

wherein the picking comprises:

selecting given ones of the words from a most recent selection time window;

identifying topics with highest relevance to the given ones of the words;

identifying topic clusters which contain the topics;

selecting a most frequent one of the topic clusters; and

selecting those given ones of the words with the highest relevance to the topics in the topic cluster.

3. The method of claim 2 , wherein the topic clustering comprises:

analyzing the log files to extract the words belonging to the corpus vocabulary;

grouping the words belonging to the corpus vocabulary by a measurement time window;

mapping each word in the group of the words to a most relevant topic in the topic model of the corpus; and

forming an optimal set of clusters from the set of topics appearing in each group.

4. The method of claim 3 , wherein the second expanded query is identical to the first expanded query.

5. The method of claim 3 , wherein the first expanded query has a plurality of first expanded query terms and wherein the second expanded query is derived from the first expanded query by:

using the topic model of said corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, the generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a second threshold.

6. The method of claim 1 , wherein the first expanded query has a plurality of first expanded query terms, and wherein the second expanded query is derived from the first expanded query by:

using a topic model of the corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the corpus vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, the generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a threshold.

7. The method of claim 1 , wherein the plurality of problem log files comprises an operating system diagnostic log.

8. A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform operations comprising:

obtaining a base query having a plurality of base query terms;

accessing a plurality of problem log files comprising error messages;

extracting words, contained in a corpus vocabulary, from the error messages of the plurality of problem log files;

based on the words extracted from the error messages of the plurality of problem log files, generating a first expanded query from the base query; and

querying the corpus, via a query engine and a corpus index, with a second expanded query related to the first expanded query.

9. The non-transitory computer readable medium of claim 8 , the operations further comprising determining that none of the query terms is in the corpus vocabulary; wherein the generating comprises, responsive to the determining that none of the query terms is in the corpus vocabulary:

picking one or more most recent ones of the words extracted from the error messages of the plurality of problem log files, having highest relevance to a single topic cluster in the log files, based on a topic model of said corpus; and

replacing the base query with at least one of the words having said highest relevance, to obtain the first expanded query; and

wherein the picking comprises:

selecting given ones of the words from a most recent selection time window;

identifying topics with highest relevance to the given ones of the words;

identifying topic clusters which contain the topics;

selecting a most frequent one of the topic clusters; and

selecting those given ones of the words with the highest relevance to the topics in the topic cluster.

10. The non-transitory computer readable medium of claim 9 , wherein the topic clustering comprises:

analyzing the log files to extract the words belonging to the corpus vocabulary;

grouping the words belonging to the corpus vocabulary by a measurement time window;

mapping each word in the group of the words to a most relevant topic in the topic model of the corpus; and

forming an optimal set of clusters from the set of topics appearing in each group.

11. The non-transitory computer readable medium of claim 10 , wherein the second expanded query is identical to the first expanded query.

12. The non-transitory computer readable medium of claim 10 , wherein the first expanded query has a plurality of first expanded query terms and wherein said second expanded query is derived from the first expanded query by:

using the topic model of said corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, said generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a second threshold.

13. The non-transitory computer readable medium of claim 8 , wherein the first expanded query has a plurality of first expanded query terms, and wherein the second expanded query is derived from the first expanded query by:

using a topic model of the corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the corpus vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, the generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a threshold.

14. The non-transitory computer readable medium of claim 8 , wherein the plurality of problem log files comprises an operating system diagnostic log.

15. An apparatus comprising:

a memory;

at least one processor, coupled to said memory; and

a non-transitory computer readable medium comprising computer executable instructions which when loaded into the memory configure the at least one processor to perform operations comprising:

obtaining a base query having a plurality of base query terms;

accessing a plurality of problem log files comprising error messages;

extracting words, contained in a corpus vocabulary, from the error messages of the plurality of problem log files;

based on the words extracted from the error messages of the plurality of problem log files, generating a first expanded query from the base query; and

querying the corpus, via a query engine and a corpus index, with a second expanded query related to the first expanded query.

16. The apparatus of claim 15 , the operations further comprising determining that none of the query terms is in the corpus vocabulary; wherein the generating comprises, responsive to the determining that none of the query terms is in the corpus vocabulary:

picking one or more most recent ones of the words extracted from the error messages of the plurality of problem log files, having highest relevance to a single topic cluster in the log files, based on a topic model of said corpus; and

replacing the base query with at least one of the words having said highest relevance, to obtain the first expanded query; and

wherein the picking comprises:

selecting given ones of the words from a most recent selection time window;

identifying topics with highest relevance to the given ones of the words;

identifying topic clusters which contain the topics;

selecting a most frequent one of the topic clusters; and

selecting those given ones of the words with the highest relevance to the topics in the topic cluster.

17. The apparatus of claim 16 , wherein the topic clustering comprises:

analyzing the log files to extract the words belonging to the corpus vocabulary;

grouping the words belonging to the corpus vocabulary by a measurement time window;

mapping each word in the group of the words to a most relevant topic in the topic model of the corpus; and

forming an optimal set of clusters from the set of topics appearing in each group.

18. The apparatus of claim 17 , wherein the second expanded query is identical to the first expanded query.

19. The apparatus of claim 17 , wherein the first expanded query has a plurality of first expanded query terms and wherein the second expanded query is derived from the first expanded query by:

using the topic model of said corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, the generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a second threshold.

20. The apparatus of claim 15 , wherein the first expanded query has a plurality of first expanded query terms, and wherein the second expanded query is derived from the first expanded query by:

using a topic model of the corpus, generating a topic for each of the first expanded query terms to obtain a plurality of generated topics;

for each given one of the plurality of generated topics, generating a candidate word from the corpus vocabulary; and

adding one or more of the generated candidate words to the first expanded query to obtain the second expanded query, the generated candidate words to be added to the first expanded query having a highest value of conditional probability, given the first expanded query, and exceeding a threshold.