Title standardization
View Patent ↗A title standardization system is may be configured to detect an edit operation associated with the job title field of a member profile stored by an on-line social network system and, in response, perform operations to derive a canonical title that represents a raw title string found in the job title field. The derived canonical title may be then associated with the member profile, in which the originally-obtained subject title string was found. This association may be stored in a database for future use, e.g., for targeting job recommendations, recruiting, making professional contacts, as well as for other purposes.
1. A computer-implemented method comprising:
detecting a raw title string in a subject profile, the subject profile being from member profiles maintained by an on-line social network system;
representing the raw title string as a triplet comprising a prefix, a core, and a suffix, the core including a core string, the prefix including a non-empty or an empty string, the suffix including a non-empty or an empty string;
deriving, using at least one processor, from the core string, a canonical title, wherein the deriving comprises:
extracting one or more phrases from the core string,
for each extracted phrase calculating frequency of occurrence of a phrase in respective job title fields in a subject set of member profiles from the member profiles and calculating one or more conditional probability values, the one or more conditional probability values indicative of probability of a phrase being a complete stand-alone job title, and from the extracted phrases designating a phrase as the canonical title based on frequency of occurrence calculated for that phrase and the one or more conditional probability values as compared to frequency of occurrence and one or more conditional probability values calculated for other phrases from the extracted phrases;
generating an indication of association of the canonical title with the subject profile; and
storing the indication of association of the canonical title with the subject profile in a database.
2. The method of claim 1 , wherein the one or more conditional probability values for a phrase extracted from the core string comprises probability of the last word in the phrase being the last word in a job title, with respect to job titles stored in the subject set of member profiles.
3. The method of claim 1 , wherein the one or more conditional probability values for a phrase extracted from the core string comprises probability of the first word in the phrase being the first word in a job title and the last word in that phrase being the last word in a job title, with respect to job titles stored in the subject set of member profiles.
4. The method of claim 1 , wherein the subject set of member profiles is associated with a particular industry.
5. The method of claim 1 , wherein the calculating of the one or more conditional probability values for an extracted phrase comprises:
calculating a branching factor for a first word in the extracted phrase, the branching factor reflecting a number of unique words that occur after the first word in job titles stored in the subject set of member profiles;
calculating probability of co-occurrence of the first word and a second word in the extracted phrase; and
comparing the branching factor and the probability of co-occurrence of the first word and the second word.
6. The method of claim 5 , wherein the calculating of the probability of co-occurrence of the first word and the second word in the extracted phrase comprises:
determining a first value comprising a number of instances of the second word occurring subsequent to the first word in the subject set of member profiles;
determining a second value comprising a number of instances of any other word occurring subsequent to the first word in the subject set of member profiles;
dividing the first value by the second value.
7. The method of claim 1 , comprising storing the canonical title in a dictionary of canonical titles.
8. The method of claim 7 , wherein an entry in the dictionary of canonical titles comprises a canonical title string and a count indicative of a number of raw title strings in the member profiles that have been mapped to that canonical title wing.
9. The method of claim 8 , comprising updating, periodically, the dictionary of canonical titles.
10. A computer-implemented system comprising:
a subject string detector, implemented using at least one processor, to detect a raw title string in a subject profile, the subject profile being from member profiles maintained by an on-line social network system;
a core extractor, implemented using at least one processor, to represent the raw title string as a triplet comprising a prefix, a core, and a suffix, the core including a core string, the prefix including a non-empty or an empty string, the suffix including a non-empty or an empty string;
a subject string processing module, implemented using at least one processor, to derive, from the core string, a canonical title, wherein the deriving comprises:
extracting one or more phrases from the core string,
for each extracted phrase calculating frequency of occurrence of a phrase in respective job title fields in a subject set of member profiles from the member profiles and calculating one or more conditional probability values, the one or more conditional probability values indicative of probability of a phrase being a complete stand-alone job title, and from the extracted phrases designating a phrase as the canonical title based on frequency of occurrence calculated for that phrase and the one or more conditional probability values as compared to frequency of occurrence and one or more conditional probability values calculated for other phrases from the extracted phrases;
an association module, implemented using at least one processor, to generate an indication of association of the canonical title with the subject profile; and
a storing module, implemented using at least one processor, to store the indication of association of the canonical title with the subject profile in a database.
11. The system of claim 10 , wherein the one or more conditional probability values for a phrase extracted from the core string comprises probability of the last word in the phrase being the last word in a job title, with respect to job titles stored in the subject set of member profiles.
12. The system of claim 10 , wherein the one or more conditional probability values for a phrase extracted from the core string comprises probability of the first word in the phrase being the first word in a job title and the last word in that phrase being the last word in a job title, with respect to job titles stored in the subject set of member profiles.
13. The system of claim 10 , wherein the subject set of member profiles is associated with a particular industry.
14. The system of claim 10 , wherein the calculating of the one or more conditional probability values for an extracted phrase comprises:
calculating a branching factor for a first word in the extracted phrase, the branching factor reflecting a number of unique words that occur after the first word in job titles stored in the subject set of member profiles;
calculating probability of co-occurrence of the first word and a second word in the extracted phrase; and
comparing the branching factor and the probability of co-occurrence of the first word and the second word.
15. The system of claim 14 , wherein the calculating of the probability of co-occurrence of the first word and the second word in the extracted phrase comprises:
determining a first value comprising a number of instances of the second word occurring subsequent to the first word in the subject set of member profiles;
determining a second value comprising a number of instances of any other word occurring subsequent to the first word in the subject set of member profiles;
dividing the first value by the second value.
16. The system of claim 11 , wherein the storing module is to store the canonical title in a dictionary of canonical titles.
17. The system of claim 16 , wherein an entry in the dictionary of canonical titles comprises a canonical title string and a count indicative of a number of raw title strings in the member profiles that have been mapped to that canonical title string.
18. A machine-readable non-transitory storage medium having instruction data executable by a machine to cause the machine to perform operations comprising:
detecting a raw title string in a subject profile, the subject profile being from member profiles maintained by an on-line social network system;
representing the raw title string as a triplet comprising a prefix, a core, and a suffix, the core including a core string, the prefix including a non-empty or an empty string, the suffix including a non-empty or an empty string;
deriving, from the core string, a canonical title, wherein the deriving comprises:
extracting one or more phrases from the core string,
for each extracted phrase calculating frequency of occurrence of a phrase in respective job title fields in a subject set of member profiles from the member profiles and calculating one or more conditional probability values, the one or more conditional probability values indicative of probability of a phrase being a complete stand-alone job title, and from the extracted phrases designating a phrase as the canonical title based on frequency of occurrence calculated for that phrase and the one or more conditional probability values as compared to frequency of occurrence and one or more conditional probability values calculated for other phrases from the extracted phrases;
generating an indication of association of the canonical title with the subject profile; and
storing the indication of association of the canonical title with the subject profile in a database.