Creating and using triplet representations to assess similarity between job description documents
A method, system and computer program product for assessing similarity between two job description documents. Job description documents consist of sentences framed in a particular manner, where the sentences are represented as a set of actions, an object corresponding to each action and a set of attributes corresponding to the object. The two job description documents are parsed to generate a first and a second set of an action-object-attribute triplet representation, where the first set of the action-object-attribute triplet representation is associated with the first job description document and the second set of the action-object-attribute triplet representation is associated with the second job description document. A similarity score between the first and second sets of action-object-attribute triplet representations is then calculated by hierarchically matching the first and second sets of action-object-attribute triplet representations across the job description documents. In this manner, similar job positions/job descriptions may be more accurately identified.
1 . A computer-implemented method for assessing similarity between two job description documents, the method comprising:
receiving, by a job description analyzer, a first and a second job description document, wherein each of said first and second job description documents comprises sentences represented as a set of actions, an object corresponding to each action and a set of attributes corresponding to said object;
parsing, by said job description analyzer, said first and second job description documents to generate a first and a second set of an action-object-attribute triplet representation, wherein a sentence tokenizer of said job description analyzer identifies a list of sentences in each of said first and second job description documents using natural language processing, wherein a word tokenizer of said job description analyzer identifies an action-object-attribute triplet representation for each sentence using dictionaries, language taxonomies and natural language processing, wherein said first set of action-object-attribute triplet representation is associated with said first job description document and said second set of action-object-attribute triplet representation is associated with said second job description document;
constructing, by said job description analyzer, an action similarity matrix containing semantic similarity scores between action terms represented in a triplet representation of a first document and action terms represented in a triplet representation of a second document, wherein said semantic similarity scores indicate how similar in meaning two terms are;
identifying, by said job description analyzer, assignments of actions with a highest semantic similarity score among job description documents based on said action similarity matrix;
calculating, by said job description analyzer, semantic similarity scores among objects and attributes of said action-object-attribute triplet representations corresponding to matched actions amongst said job description documents;
identifying, by said job description analyzer, assignments of objects with highest semantic similarity scores amongst said job description documents corresponding to matching pairs of actions using an object similarity matrix for action assignments which identifies a semantic similarity between object terms;
identifying, by said job description analyzer, assignments of attributes amongst said job description documents corresponding to matching pairs of objects using an attribute similarity matrix containing semantic similarity scores between attribute terms associated with object terms of said triplet representations of said job description documents;
hierarchically combining, by said job description analyzer, semantic similarity scores of said identified assignments of objects and said identified assignments of attributes to generate one matching score per pair of actions in said job description documents;
combining, by said job description analyzer, scores corresponding to all matching action pairs to create one document similarity score for said job description documents, wherein said document similarity score for said first and second job description documents is ((V A1 (1+N A1 (1+A A1 )/3)+(V A2 (1+(N A2 +N A3 +N A4 )/3))/2)/2), where V A1 is a semantic similarity score between action terms represented in triplet representations of said first document, where V A2 is a semantic similarity score between action terms represented in triplet representations of said second document, where N A1 is a highest similarity score in a first row of said object similarity matrix, where N A2 is a highest similarity score in a second row of said object similarity matrix, where N A3 is a highest similarity score in a third row of said object similarity matrix, where N A4 is a highest similarity score in a fourth row of said object similarity matrix, where A A1 is a semantic similarity score for said attribute terms associated with said object terms of said triplet representations of said first and second job description documents; and
determining, by said job description analyzer, a degree of similarity between said first and said second job description documents based on said document similarity score for said first and second description documents, wherein the higher a value of said document similarity score, the greater the degree of similarity between said first and second job description documents.
2 . The method as recited in claim 1 further comprising:
calculating semantic similarity scores among actions of said first and second sets of action-object-attribute triplet representations amongst said first and second job description documents.
3 . The method as recited in claim 1 further comprising:
identifying assignments of actions with highest semantic similarity scores amongst said first and second job description documents.
4 . The method as recited in claim 1 , wherein said sentence tokenizer identifies a beginning and an ending of sentences from said received first and second job descriptions using natural language processing or sentence boundary disambiguation, wherein after identifying said sentences from said received first and second job descriptions, said word tokenizer is used to identify a list of words in strings and to tag parts of speech, wherein said word tokenizer identifies actions, objects and attributes using said dictionaries and said language taxonomies.
5 . A computer program product for assessing similarity between two job description documents, the computer program product comprising one or more computer readable storage mediums having program code embodied therewith, the program code comprising programming instructions for:
receiving, by a job description analyzer, a first and a second job description document, wherein each of said first and second job description documents comprises sentences represented as a set of actions, an object corresponding to each action and a set of attributes corresponding to said object;
parsing, by said job description analyzer, said first and second job description documents to generate a first and a second set of an action-object-attribute triplet representation, wherein a sentence tokenizer of said job description analyzer identifies a list of sentences in each of said first and second job description documents using natural language processing, wherein a word tokenizer of said job description analyzer identifies an action-object-attribute triplet representation for each sentence using dictionaries, language taxonomies and natural language processing, wherein said first set of action-object-attribute triplet representation is associated with said first job description document and said second set of action-object-attribute triplet representation is associated with said second job description document;
constructing, by said job description analyzer, an action similarity matrix containing semantic similarity scores between action terms represented in a triplet representation of a first document and action terms represented in a triplet representation of a second document, wherein said semantic similarity scores indicate how similar in meaning two terms are;
identifying, by said job description analyzer, assignments of actions with a highest semantic similarity score among job description documents based on said action similarity matrix;
calculating, by said job description analyzer, semantic similarity scores among objects and attributes of said action-object-attribute triplet representations corresponding to matched actions amongst said job description documents;
identifying, by said job description analyzer, assignments of objects with highest semantic similarity scores amongst said job description documents corresponding to matching pairs of actions using an object similarity matrix for action assignments which identifies a semantic similarity between object terms;
identifying, by said job description analyzer, assignments of attributes amongst said job description documents corresponding to matching pairs of objects using an attribute similarity matrix containing semantic similarity scores between attribute terms associated with object terms of said triplet representations of said job description documents;
hierarchically combining, by said job description analyzer, semantic similarity scores of said identified assignments of objects and said identified assignments of attributes to generate one matching score per pair of actions in said job description documents;
combining, by said job description analyzer, scores corresponding to all matching action pairs to create one document similarity score for said job description documents, wherein said document similarity score for said first and second job description documents is ((V A1 (1+N A1 (1+A A1 )/3)+ (V A2 (1+ (N A2 +N A3 +N A4 )/3))/2)/2), where V A1 is a semantic similarity score between action terms represented in triplet representations of said first document, where V A2 is a semantic similarity score between action terms represented in triplet representations of said second document, where N A1 is a highest similarity score in a first row of said object similarity matrix, where N A2 is a highest similarity score in a second row of said object similarity matrix, where N A3 is a highest similarity score in a third row of said object similarity matrix, where N A4 is a highest similarity score in a fourth row of said object similarity matrix, where A A1 is a semantic similarity score for said attribute terms associated with said object terms of said triplet representations of said first and second job description documents; and
determining, by said job description analyzer, a degree of similarity between said first and said second job description documents based on said document similarity score for said first and second description documents, wherein the higher a value of said document similarity score, the greater the degree of similarity between said first and second job description documents.
6 . The computer program product as recited in claim 5 , wherein the program code further comprises the programming instructions for:
calculating semantic similarity scores among actions of said first and second sets of action-object-attribute triplet representations amongst said first and second job description documents.
7 . The computer program product as recited in claim 5 , wherein the program code further comprises the programming instructions for:
identifying assignments of actions with highest semantic similarity scores amongst said first and second job description documents.
8 . The computer program product as recited in claim 5 , wherein said sentence tokenizer identifies a beginning and an ending of sentences from said received first and second job descriptions using natural language processing or sentence boundary disambiguation, wherein after identifying said sentences from said received first and second job descriptions, said word tokenizer is used to identify a list of words in strings and to tag parts of speech, wherein said word tokenizer identifies actions, objects and attributes using said dictionaries and said language taxonomies.
9 . A job description analyzer, comprising:
a memory for storing a computer program for assessing similarity between two job description documents; and
a processor connected to said memory, wherein said processor is configured to execute program instructions of the computer program comprising:
receiving a first and a second job description document, wherein each of said first and second job description documents comprises sentences represented as a set of actions, an object corresponding to each action and a set of attributes corresponding to said object;
parsing said first and second job description documents to generate a first and a second set of an action-object-attribute triplet representation, wherein a sentence tokenizer of said job description analyzer identifies a list of sentences in each of said first and second job description documents using natural language processing, wherein a word tokenizer of said job description analyzer identifies an action-object-attribute triplet representation for each sentence using dictionaries, language taxonomies and natural language processing, wherein said first set of action-object-attribute triplet representation is associated with said first job description document and said second set of action-object-attribute triplet representation is associated with said second job description document;
constructing an action similarity matrix containing semantic similarity scores between action terms represented in a triplet representation of a first document and action terms represented in a triplet representation of a second document, wherein said semantic similarity scores indicate how similar in meaning two terms are;
identifying assignments of actions with a highest semantic similarity score among job description documents based on said action similarity matrix;
calculating semantic similarity scores among objects and attributes of said action-object-attribute triplet representations corresponding to matched actions amongst said job description documents;
identifying assignments of objects with highest semantic similarity scores amongst said job description documents corresponding to matching pairs of actions using an object similarity matrix for action assignments which identifies a semantic similarity between object terms;
identifying assignments of attributes amongst said job description documents corresponding to matching pairs of objects using an attribute similarity matrix containing semantic similarity scores between attribute terms associated with object terms of said triplet representations of said job description documents;
hierarchically combining semantic similarity scores of said identified assignments of objects and said identified assignments of attributes to generate one matching score per pair of actions in said job description documents;
combining scores corresponding to all matching action pairs to create one document similarity score for said job description documents, wherein said document similarity score for said first and second job description documents is ((V A1 (1+N A1 (1+A A1 )/3)+ (V A2 (1+ (N A2 +N A3 +N A4 )/3))/2)/2), where V A1 is a semantic similarity score between action terms represented in triplet representations of said first document, where V A2 is a semantic similarity score between action terms represented in triplet representations of said second document, where N A1 is a highest similarity score in a first row of said object similarity matrix, where N A2 is a highest similarity score in a second row of said object similarity matrix, where N A3 is a highest similarity score in a third row of said object similarity matrix, where N A4 is a highest similarity score in a fourth row of said object similarity matrix, where A A1 is a semantic similarity score for said attribute terms associated with said object terms of said triplet representations of said first and second job description documents; and
determining a degree of similarity between said first and said second job description documents based on said document similarity score for said first and second description documents, wherein the higher a value of said document similarity score, the greater the degree of similarity between said first and second job description documents.
10 . The job description analyzer as recited in claim 9 , wherein the program instructions of the computer program further comprise:
calculating semantic similarity scores among actions of said first and second sets of action-object-attribute triplet representations amongst said first and second job description documents.
11 . The job description analyzer as recited in claim 9 , wherein the program instructions of the computer program further comprise: identifying assignments of actions with highest semantic similarity scores amongst said first and second job description documents.
12 . The job description analyzer as recited in claim 9 , wherein said sentence tokenizer identifies a beginning and an ending of sentences from said received first and second job descriptions using natural language processing or sentence boundary disambiguation, wherein after identifying said sentences from said received first and second job descriptions, said word tokenizer is used to identify a list of words in strings and to tag parts of speech, wherein said word tokenizer identifies actions, objects and attributes using said dictionaries and said language taxonomies.