Semantic clustering based retrieval for candidate set expansion

1. A system comprising:

a non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to:

in a training phase:

obtain training data pertaining to sample job posting search queries, the training data comprising sample job posting search results and indications as to which of the sample job posting search results were selected by members performing corresponding job posting search queries;

for each of the sample job posting search queries, feed the corresponding training data into a first machine learning algorithm to train a query-based deep semantic similarity neural network to output a query context vector for an input job posting search query and into a second machine learning algorithm to train a document-based deep semantic similarity neural network to output a document context vector for an input job posting;

map each query context vector and each document context vector into a cluster using a clustering algorithm;

in a retrieval phase:

obtain a first job posting search query;

pass the first job posting search query into the query-based deep semantic similarity neural network to output a first query context vector;

identify a cluster in which to map the first document query vector using the clustering algorithm; and

retrieve documents contained in a cluster corresponding to the identified cluster.

2. The system of claim 1 , wherein the instructions further cause the system to:

add an entry for each cluster identification to an inverted index, each entry further indicating a document mapped into a corresponding cluster; and

the retrieving documents further comprising locating a first set of one or more entries, in the inverted index, corresponding to the identification of the cluster in which to map the first query context vector and retrieving any document indicated by the first set of one or more entries.

3. The system of claim 1 , wherein the clustering algorithm is a k-means clustering algorithm.

4. The system of claim 1 , wherein the clustering algorithm is a k-nearest neighbor clustering algorithm.

5. The system of claim 1 , wherein the retrieved documents are interleaved with documents retrieved via a keyword-based retrieval process.

6. The system of claim 1 , wherein the query-based deep semantic similarity neural network includes a hashing layer, an embedding layer, and a plurality of non-linear activation layers.

7. The system of claim 6 , wherein the plurality of non-linear activation layers have an activation unit of tan h.

8. A computerized method, comprising

in a training phase:

obtaining training data pertaining to sample job posting search queries, the training data comprising sample job posting search results and indications as to which of the sample job posting search results were selected by members performing corresponding job posting search queries;

for each of the sample job posting search queries, feeding the corresponding training data into a first machine learning algorithm to train a query-based deep semantic similarity neural network to output a query context vector for an input job posting search query and into a second machine learning algorithm to train a document-based deep semantic similarity neural network to output a document context vector for an input job posting;

mapping each query context vector and each document context vector into a cluster using a clustering algorithm;

in a retrieval phase:

obtaining a first job posting search query;

passing the first job posting search query into the query-based deep semantic similarity neural network to output a first query context vector;

identify a cluster in which to map the first document query vector using the clustering algorithm; and

retrieving documents contained in the identified cluster.

9. The method of claim 8 , further comprising:

adding an entry for each cluster identification to an inverted index, each entry further indicating a document mapped into a corresponding cluster; and

the retrieving documents further comprising locating a first set of one or more entries, in the inverted index, corresponding to the identification of the cluster in which to map the first query context vector and retrieving any document indicated by the first set of one or more entries.

10. The method of claim 8 , wherein the clustering algorithm is a k-means clustering algorithm.

11. The method of claim 8 , wherein the clustering algorithm is a k-nearest neighbor clustering algorithm.

12. The method of claim 8 , wherein the retrieved documents are interleaved with documents retrieved via a keyword-based retrieval process.

13. The method of claim 8 , wherein the query-based deep semantic similarity neural network includes a hashing layer, an embedding layer, and a plurality of non-linear activation layers.

14. The method of claim 13 , wherein the plurality of non-linear activation layers have an activation unit of tan h.

15. A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations comprising:

in a training phase:

obtaining training data pertaining to sample job posting search queries, the training data comprising sample job posting search results and indications as to which of the sample job posting search results were selected by members performing corresponding job posting search queries;

for each of the sample job posting search queries, feeding the corresponding training data into a first machine learning algorithm to train a query-based deep semantic similarity neural network to output a query context vector for an input job posting search query and into a second machine learning algorithm to train a document-based deep semantic similarity neural network to output a document context vector for an input job posting;

mapping each query context vector and each document context vector into a cluster using a clustering algorithm;

in a retrieval phase:

obtaining a first job posting search query;

passing the first job posting search query into the query-based deep semantic similarity neural network to output a first query context vector;

identify a cluster in which to map the first document query vector using the clustering algorithm; and

retrieving documents contained in the identified cluster.

16. The non-transitory machine-readable storage medium of claim 15 , further comprising:

adding an entry for each cluster identification to an inverted index, each entry further indicating a document mapped into a corresponding cluster; and

the retrieving documents further comprising locating a first set of one or more entries, in the inverted index, corresponding to the identification of the cluster in which to map the first query context vector and retrieving any document indicated by the first set of one or more entries.

17. The non-transitory machine-readable storage medium of claim 15 , wherein the clustering algorithm is a k-means clustering algorithm.

18. The non-transitory machine-readable storage medium of claim 15 , wherein the clustering algorithm is a k-nearest neighbor clustering algorithm.

19. The non-transitory machine-readable storage medium of claim 15 , wherein the retrieved documents are interleaved with documents retrieved via a keyword-based retrieval process.

20. The non-transitory machine-readable storage medium of claim 15 , wherein the query-based deep semantic similarity neural network includes a hashing layer, an embedding layer, and a plurality of non-linear activation layers.