Systems and methods for skills inference using a datastore and models
A system comprising: a skills data store; an employee action data store; at least one hardware processor; and one or more software modules that are configured to, when executed by the at least one hardware processor, retrieve skills data and employee action data from the skills data store and employee action data store, train a classification model, wherein training a classification model comprises performing feature preprocessing, generating an LDA topic vector and TF/IDF Word2Vec similarity scoring, and use AutoML to train ML models, and infer employee skills and levels based on the classification model and employee action data.
1 . A system comprising:
a skills data store comprising skill descriptions for a plurality of employees associated with a plurality of companies;
an employee action data store comprising employee action data for the plurality of employees;
at least one hardware processor; and
one or more software modules are configured to, when executed by the at least one hardware processor, for each of the plurality of companies and the plurality of employees associated therewith:
retrieve the skill descriptors and employee action data from the skills data store and employee action data store,
convert the skill descriptors and employee action data to topic vectors,
compare the employee action data to the skill descriptors using the topic vectors,
generate a maximum and average semantic similarity scores based on topic vector differences between the skill descriptors and employee action data;
based on the maximum and average semantic similarity scores associated with each of the plurality of companies, generate a classification model based on the maximum and average semantic similarity scores;
train the classification model based on known accurate employee to skill mappings, wherein training a classification model comprises performing acronym/synonym expansion, stemming/lemmatization, vectorizing text in order to generate an LDA topic vector model and TF/IDF Word2Vec similarity scoring model, and cosine similarity feature preprocessing;
infer employee skills and levels, for the plurality of employees, based on the classification model and employee action data; and
track the accuracy of the classification model over time as new employee action data is received; and
automatically retrain the classification model when the tracking indicates that the classification model accuracy has degraded below a threshold.
2 . The system of claim 1 , wherein acronym expansion comprises given a dictionary with acronyms definition, augmenting the data with full statements for each acronym, and wherein the original acronym is preserved for use with a TF-IDF model.
3 . The system of claim 1 , wherein synonym expansion comprises using a WordNet language model to augment nouns with synonyms to increase the intersection of words between entities, which helps to improve description similarity.
4 . The system of claim 1 , wherein stemming/lemmatization comprises normalizing words for TF-IDF and LDA models to make them a 1-to-1 match.
5 . The system of claim 1 , wherein vectorizing text comprises training models TF-IDF and LDA models on a text corpus and transforming text description into numeric vectors to feed them into ML algorithms.
6 . The system of claim 5 , wherein the TF-IDF model is a baseline, which roughly counts the number of intersecting words and weights them by relevance to a particular document, and wherein the LDA model is configured to extract topics from text to allow matching relevant documents with a low number of intersecting words via topics, and wherein the Word2vec model allows matching documents even without common words by employing semantic similarity.
7 . The system of claim 1 , wherein cosine similarity feature preprocessing comparing numeric vectors for similarity on average.