Clustering a set of natural language queries based on significant events

1. An improved method for searching a selected corpus by clustering a set of natural language queries (NLQ) based on a set of significant events retrieved from a corpus stored in a computer system comprising:

using a set of NLQs by a search engine for searching a selected corpus to retrieve respective sets of significant events from the selected corpus;

for each NLQ in the set of NLQs, extracting a first set of entities from the NLQ and using the first set of entities to search for a first set of significant events in the selected corpus in a first search pass;

extracting a second set of entities from the first set of significant events and using the second set of entities to search for a second set of significant events in the selected corpus in a second search pass;

producing a distribution profile for each NLQ based on a threshold number of significant events retrieved in a respective first search pass and a respective second search pass;

clustering the set of NLQs into NLQ clusters according to the distribution profiles;

and

using a respective NLQ cluster in a query expansion of an initial user NLQ in the search engine.

2. The method as recited in claim 1 , wherein the clustering is also based in part on common linguistic and semantic features of respective NLQs.

3. The method as recited in claim 2 , further comprising:

from user input, receiving a threshold number of significant events as a clustering criterion; and

from user input, receiving a threshold number of common linguistic and semantic features in an NLQ as a clustering criterion.

4. The method as recited in claim 1 , further comprising:

building a knowledge graph based on a selected corpus stored in the computer system, the knowledge graph having a set of co-occurrence scores on edges of the knowledge graph between respective events in the selected corpus placed at the nodes of the knowledge graph, wherein the co-occurrence scores indicate co-occurrence of entities within respective pairs of events in the selected corpus; and

using the knowledge graph to extract the second set of entities.

5. The method as recited in claim 1 , further comprising:

extracting a third set of entities from the second set of significant events and using the third set of entities to search for a third set of significant events in the selected corpus in a third search pass; and

producing a distribution profile for each NLQ based on a number of significant events retrieved in the first search pass, the second search pass and the third search pass.

6. The method as recited in claim 1 , further comprising:

determining a significance score for respective events retrieved by the search system according to a metric of mutual information (MMI); and

filtering the retrieved events according to respective significance scores to produce the first set of significant events.

7. The method as recited in claim 1 , wherein the first and second sets of entities have no common members.

8. Apparatus, comprising:

a processor;

computer memory holding computer program instructions executed by the processor for improved searching of a selected corpus by clustering a set of natural language queries (NLQ), the computer program instructions comprising:

program code, operative to use a set of NLQs by for searching a selected corpus to retrieve respective sets of significant events from the selected corpus;

program code, operative to extract a first set of entities from the NLQ and using the first set of entities to search for a first set of significant events in the selected corpus in a first search pass;

program code, operative to extract a second set of entities from the first set of significant events and using the second set of entities to search for a second set of significant events in the selected corpus in a second search pass;

program code, operative to producing a distribution profile for each NLQ based on a threshold number of significant events retrieved in a respective first search pass and a respective second search pass;

program code, operative to cluster the set of NLQs into NLQ clusters according to the distribution profiles;

and

program code, operative to use a respective NLQ cluster in a query expansion of an initial user NLQ in the search engine.

9. The apparatus as recited in claim 8 , wherein the clustering is also based in part on common linguistic and semantic features of respective NLQs.

10. The apparatus as recited in claim 9 , further comprising:

program code, operative to receive a threshold number of significant events as a clustering criterion; and

program code, operative to receive a threshold number of common linguistic and semantic features in an NLQ as a clustering criterion.

11. The apparatus as recited in claim 8 , further comprising:

program code, operative to build a knowledge graph based on a selected corpus stored in the computer system, the knowledge graph having a set of co-occurrence scores on edges of the knowledge graph between in the selected corpus placed at the nodes of the knowledge graph, wherein the co-occurrence scores indicate co-occurrence of entities within respective pairs of events in the selected corpus; and

program code, operative to use the knowledge graph to extract the second set of entities.

12. The apparatus as recited in claim 8 , further comprising:

program code, operative to extract a third set of entities from the second set of significant events and using the third set of entities to search for a third set of significant events in the selected corpus in a third search pass; and

program code, operative to produce a distribution profile for each NLQ based on a number of significant events retrieved in the first search pass, the second search pass and the third search pass.

13. A computer program product in a non-transitory computer readable medium for use in a data processing system, the computer program product holding computer program instructions executed by the data processing system for improved searching of a selected corpus by performing clustering of natural language queries (NLQ), the computer program instructions comprising:

program code, operative to use a set of NLQs by for searching a selected corpus to retrieve respective sets of significant events from the selected corpus;

program code, operative to extract a first set of entities from the NLQ and using the first set of entities to search for a first set of significant events in the selected corpus in a first search pass;

program code, operative to extract a second set of entities from the first set of significant events and using the second set of entities to search for a second set of significant events in the selected corpus in a second search pass;

program code, operative to producing a distribution profile for each NLQ based on a threshold number of significant events retrieved in a respective first search pass and a respective second search pass;

program code, operative to cluster the set of NLQs into NLQ clusters according to the distribution profiles;

and

program code, operative to use a respective NLQ cluster in a query expansion of an initial user NLQ in the search engine.

14. The computer program product as recited in claim 13 , further comprising:

program code, operative to cluster based on common linguistic and semantic features of respective NLQs.

15. The computer program product as recited in claim 14 , further comprising:

program code, operative to receive a threshold number of significant events as a clustering criterion; and

program code, operative to receive a threshold number of common linguistic and semantic features in an NLQ as a clustering criterion.

16. The computer program product as recited in claim 13 , further comprising:

program code, operative to build a knowledge graph based on a selected corpus stored in the computer system, the knowledge graph having a set of co-occurrence scores on edges of the knowledge graph between respective events in the selected corpus placed at the nodes of the knowledge graph, wherein the co-occurrence scores indicate the co-occurrence of entities within respective pairs of events in the selected corpus; and

program code, operative to use the knowledge graph to extract the second set of entities.