JOB POSTING STANDARDIZATION AND DEDUPLICATION
Techniques for standardizing and deduplicating unpaid job postings obtained from third-party systems are described. An unpaid job posting is obtained by a social networking service from a third-party system. The title and description of the unpaid job posting are standardized and combined into a standardized unpaid job posting. A deduplication process is performed to prevent the standardized unpaid job posting from replacing a paid job posting within the social networking service, and to prevent the standardized unpaid job posting from replacing a more authoritative, unpaid job posting within the social networking service.
1 . A method comprising:
obtaining, by a first entity, data representing a job posting on a third-party employment system, the data including a job title and a job description;
standardizing the job title to match at least one of a plurality of pre-defined job titles recognized by the first entity;
standardizing the job description to conform to a data format recognized by the first entity;
combining the standardized job title and the standardized job description into a standardized job posting; and
integrating the standardized job posting into an employment system of the first entity.
2 . The method of claim 1 , wherein standardizing the job title includes removing an occurrence of an undesired character, the removing performed using at least one regular expression.
3 . The method of claim 1 , wherein standardizing the job title includes at least one of:
determining a geographical location within the job title and removing the determined geographical location from the job title; or
determining an employer name within the job title and removing the determined employer name from the job title.
4 . The method of claim 1 , wherein standardizing the job title includes replacing an abbreviation within the job title with a word or a phrase that is recognized by the first entity as representing the abbreviation.
5 . The method of claim 4 , wherein replacing includes disambiguating the abbreviation using at least one of context within the job title and context within the job description.
6 . The method of claim 1 , wherein standardizing the job title includes:
dividing the job title, comprising an ordered plurality of words, into a list of words;
generating a plurality of permutations of words from the list of words; and
choosing, from the plurality of permutations of words, a permutation of words that most closely matches at least one of the plurality of pre-defined job titles recognized by the first entity.
7 . The method of claim 6 , wherein standardizing the job title further includes determining a job title number and a job seniority level corresponding to the standardized job title, and wherein the job title number and the job seniority level are included in the standardized job posting.
8 . The method of claim 1 , wherein the standardized job posting includes at least one of an identification of a geographical location, an employer name, an employment industry, and a job skill.
9 . A system comprising:
a machine including a memory and at least one processor;
a job capture module, executable by the machine, configured to obtain, by a first entity, data representing a job posting on a third-party employment system, the data including a job title and a job description; and
a job standardization module, executable by the machine, configured to:
standardize the job title to match at least one of a plurality of pre-defined job titles recognized by the first entity;
standardize the job description to conform to a data format recognized by the first entity;
combine the standardized job title and the standardized job description into a standardized job posting; and
integrate the standardized job posting into an employment system of the first entity.
10 . The system of claim 9 , wherein standardizing the job title includes removing an occurrence of an undesired character, the removing performed using at least one regular expression.
11 . The system of claim 9 , wherein standardizing the job title includes at least one of:
determining a geographical location within the job title and removing the determined geographical location from the job title; or
determining an employer name within the job title and removing the determined employer name from the job title.
12 . The system of claim 9 , wherein standardizing the job title includes replacing an abbreviation within the job title with a word or a phrase that is recognized by the first entity as representing the abbreviation.
13 . The system of claim 12 , wherein replacing includes disambiguating the abbreviation using at least one of context within the job title and context within the job description.
14 . The system of claim 9 , wherein standardizing the job title includes:
dividing the job title, comprising an ordered plurality of words, into a list of words;
generating a plurality of permutations of words from the list of words; and
choosing, from the plurality of permutations of words, a permutation of words that most closely matches at least one of the plurality of pre-defined job titles recognized by the first entity.
15 . The system of claim 14 , wherein standardizing the job title further includes determining a job title number and a job seniority level corresponding to the standardized job title, and wherein the job title number and the job seniority level are included in the standardized job posting.
16 . The system of claim 9 , wherein the standardized job posting includes at least one of an identification of a geographical location, an employer name, an employment industry, and a job skill.
17 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
obtaining, by a first entity, data representing a job posting on a third-party employment system, the data including a job title and a job description;
standardizing the job title to match at least one of a plurality of pre-defined job titles recognized by the first entity;
standardizing the job description to conform to a data format recognized by the first entity;
combining the standardized job title and the standardized job description into a standardized job posting; and
integrating the standardized job posting into an employment system of the first entity.
18 . The non-transitory machine-readable storage medium of claim 17 , wherein standardizing the job title includes removing an occurrence of an undesired character, the removing performed using at least one regular expression.
19 . The non-transitory machine-readable storage medium of claim 17 , wherein standardizing the job title includes at least one of:
determining a geographical location within the job title and removing the determined geographical location from the job title; or
determining an employer name within the job title and removing the determined employer name from the job title.
20 . The non-transitory machine-readable storage medium of claim 17 , wherein standardizing the job title includes replacing an abbreviation within the job title with a word or a phrase that is recognized by the first entity as representing the abbreviation.
21 . The non-transitory machine-readable storage medium of claim 20 , wherein replacing includes disambiguating the abbreviation using at least one of context within the job title and context within the job description.
22 . The non-transitory machine-readable storage medium of claim 17 , wherein standardizing the job title includes:
dividing the job title, comprising an ordered plurality of words, into a list of words;
generating a plurality of permutations of words from the list of words; and
choosing, from the plurality of permutations of words, a permutation of words that most closely matches at least one of the plurality of pre-defined job titles recognized by the first entity.
23 . The non-transitory machine-readable storage medium of claim 22 , wherein standardizing the job title further includes determining a job title number and a job seniority level corresponding to the standardized job title, and wherein the job title number and the job seniority level are included in the standardized job posting.
24 . The non-transitory machine-readable storage medium of claim 17 , wherein the standardized job posting includes at least one of an identification of a geographical location, an employer name, an employment industry, and a job skill.