MACHINE LEARNED CANDIDATE SELECTION ON INVERTED INDICES
Methods and systems for developing a behavioral index model are described. According to various embodiments, the system accesses a set of behavioral data stored in association with a set of member profiles. The system generates a set of training data from the set of behavioral data and generates a set of feature vectors from the set of training data. The system generates a behavioral index model from the set of training data and the set of feature vectors. The system then generates a tuned behavioral index model from the behavioral index model by tuning a threshold of the behavioral index model using a precision recall curve.
1 . A method, comprising:
accessing, by one or more hardware processors, a set of behavioral data stored on a social networking database, the set of behavioral data associated with a set of corresponding user profiles;
generating a set of training data from the set of behavioral data, the set of training data generated by identifying positive data and negative data within the set of behavioral data;
generating a set of feature vectors from the set of training data, the set of feature vectors identifying fields within two distinct portions of the set of training data;
generating a behavioral index model from the set of training data and the set of feature vectors;
generating a tuned behavioral index model from the behavioral index model, the tuned behavioral index model generated by tuning a threshold of the behavioral index model using a precision recall curve; and
causing display of an ordered list via a graphical user interface, the ordered list comprising a set of documents retrieved from a plurality of inverted indices, the ordered list generated by searching the plurality of inverted indices in accordance with one or more of the behavioral index model and the tuned behavioral index model.
2 . The method of claim 1 further comprising collecting the set of behavioral data by accessing interaction data representing interaction selecting data stored on a social networking database, the interaction data generated for a user profile of the set of user profiles after receiving a login indication for the user profile.
3 . The method of claim 1 further comprising collecting the set of behavioral data by accessing impression data representing display of data at a client device associated with a user profile of the set of user profiles after receiving a login indication for the user profile, the data displayed being data stored on the social networking database.
4 . The method of claim 1 , wherein identifying positive data comprises:
parsing the set of behavioral data to identify a linked portion of data within the social networking database associated with interaction data included within a portion of the behavioral data of the set of behavioral data;
extracting a representation of the interaction data and the portion of data; and
modifying the representation to include a positive data indicator.
5 . The method of claim 1 , wherein identifying negative data comprises:
parsing the set of behavioral data to determine a free portion of data, within the social networking database, presented at a client device associated with a user profile of the set of user profiles, the free portion of data being unassociated with interaction data included within a portion of the behavioral data of the set of behavioral data;
modifying the free portion of data to include a negative data indicator; and
discarding the free portion of data from inclusion in the behavioral index model.
6 . The method of claim 1 , wherein the behavioral index is generated for a first user profile of a set of user profiles, the first user profile determined to be similar to one or more second user profiles of the set of user profiles; and wherein identifying negative data comprises:
parsing the set of behavioral data associated with the one or more second user profiles to identify one or more portions of data presented at one or more client devices associated with the one or more second user profiles;
parsing the set of behavioral data associated with the first user profile to identify unpresented portions of data among the one or more portions of data, the unpresented portions of data being presented at the one or more client devices associated with the one or more second user profiles and unpresented at a client device associated with the first profile;
writing the one or more portions of data into the set of behavioral data associated with the first user profile, the one or more portions of data including a negative data indicator; and
discarding the one or more portions of data from inclusion in the behavioral index model.
7 . The method of claim 1 , wherein generating a behavioral index model further comprises:
generating a first logistic regression model based on the set of feature vectors;
identifying one or more feature vectors from the logistic regression model, each feature vector of the one or more feature vectors resulting in a negative coefficient in the logistic regression model;
removing the one or more feature vectors resulting in the negative coefficients; and
regenerating the first logistic regression model to generate a second logistic regression model, the second logistic regression model being generated without the one or more feature vectors.
8 . The method of claim 7 further comprising:
regenerating a subsequent logistical regression model without one or more feature vectors associated with negative coefficients of a previous logistical regression model until each final feature vector of a set of final feature vectors of the subsequent logistical regression model is associated with a positive coefficient.
9 . The method of claim 1 , wherein generating the tuned behavioral index model further comprises:
subdividing the set of training data into three training data subsets;
generating the precision recall curve from a validation data subset for one or more values of a threshold parameter, the validation data subset being one of the three training data subsets; and
selecting a value of the threshold parameter associated with a specified recall value.
10 . A system, comprising:
one or more hardware processors; and
a processor-readable storage device comprising processor executable instructions that, when executed by the one or more hardware processors, causes the one or more hardware processors to perform operations comprising:
accessing, by one or more processors of a machine, a set of behavioral data associated with a set of corresponding user profiles;
generating a set of training data from the set of behavioral data, the set of training data generated by identifying positive data and negative data within the set of behavioral data;
generating a set of feature vectors from the set of training data, the set of feature vectors identifying fields within two distinct portions of the set of training data;
generating a behavioral index model from the set of training data and the set of feature vectors;
generating a tuned behavioral index model from the behavioral index model, the tuned behavioral index model generated by tuning a threshold of the behavioral index model using a precision recall curve; and
causing display of an ordered list via a graphical user interface, the ordered list comprising a set of documents retrieved from a plurality of inverted indices, the ordered list generated by searching the plurality of inverted indices in accordance with one or more of the behavioral index model and the tuned behavioral index model.
11 . The system of claim 10 further comprising collecting the set of behavioral data by accessing interaction data and impression data, the interaction data representing interaction selecting data stored on a social networking database, the interaction data generated for a user profile of the set of user profiles after receiving a login indication for the user profile, the impression data representing display of data at a client device associated with a user profile of the set of user profiles after receiving a login indication for the user profile, the data displayed being data stored on the social networking database.
12 . The system of claim 10 , wherein identifying positive data comprises:
parsing the set of behavioral data to identify a linked portion of data within the social networking database associated with interaction data included within a portion of the behavioral data of the set of behavioral data;
extracting a representation of the interaction data and the portion of data; and
modifying the representation to include a positive data indicator.
13 . The system of claim 10 , wherein identifying negative data comprises:
parsing the set of behavioral data to determine a free portion of data, within the social networking database, presented at a client device associated with a user profile of the set of user profiles, the free portion of data being unassociated with interaction data included within a portion of the behavioral data of the set of behavioral data;
modifying the free portion of data to include a negative data indicator; and
discarding the free portion of data from inclusion in the behavioral index model.
14 . The system of claim 10 , wherein generating a behavioral index model further comprises:
generating a first logistic regression model based on the set of feature vectors;
identifying one or more feature vectors from the logistic regression model, each feature vector of the one or more feature vectors resulting in a negative coefficient in the logistic regression model;
removing the one or more feature vectors resulting in the negative coefficients: and
regenerating the first logistic regression model to generate a second logistic regression model, the second logistic regression model being generated without the one or more feature vectors.
15 . The system of claim 10 , wherein generating the tuned behavioral index model further comprises:
subdividing the set of training data into three training data subsets;
generating the precision recall curve from a validation training data subset for one or more values of a threshold parameter, the validation training data subset being one of the three training data subsets; and
selecting a value of the threshold parameter associated with a specified recall value.
16 . A processor-readable storage device comprising processor executable instructions that, when executed by one or more processors of a machine, causes the machine to perform operations comprising:
accessing, by one or more processors of a machine, a set of behavioral data associated with a set of corresponding user profiles;
generating a set of training data from the set of behavioral data, the set of training data generated by identifying positive data and negative data within the set of behavioral data;
generating a set of feature vectors from the set of training data, the set of feature vectors identifying fields within two distinct portions of the set of training data;
generating a behavioral index model from the set of training data and the set of feature vectors;
generating a tuned behavioral index model from the behavioral index model, the tuned behavioral index model generated by tuning a threshold of the behavioral index model using a precision recall curve; and
causing display of an ordered list via a graphical user interface, the ordered list comprising a set of documents retrieved from a plurality of inverted indices, the ordered list generated by searching the plurality of inverted indices in accordance with one or more of the behavioral index model and the tuned behavioral index model.
17 . The processor-readable storage device of claim 16 further comprising collecting the set of behavioral data by accessing interaction data and impression data, the interaction data representing interaction selecting data stored on the social networking database, the interaction data generated for a user profile of the set of user profiles after receiving a login indication for the user profile, the impression data representing display of data at a client device associated with a user profile of the set of user profiles after receiving a login indication for the user profile, the data displayed being data stored on the social networking database.
18 . The processor-readable storage device of claim 16 , wherein identifying positive data comprises:
parsing the set of behavioral data to identify a linked portion of data within the social networking database associated with interaction data included within a portion of the behavioral data of the set of behavioral data;
extracting a representation of the interaction data and the portion of data; and
modifying the representation to include a positive data indicator.
19 . The processor-readable storage device of claim 16 , wherein generating a behavioral index model further comprises:
generating a first logistic regression model based on the set of feature vectors;
identifying one or more feature vectors from the logistic regression model, each feature vector of the one or more feature vectors resulting in a negative coefficient in the logistic regression model;
removing the one or more feature vectors resulting in the negative coefficients; and
regenerating the first logistic regression model to generate a second logistic regression model, the second logistic regression model being generated without the one or more feature vectors.
20 . The processor-readable storage device of claim 16 , wherein generating the tuned behavioral index model further comprises:
subdividing the set of training data into three training data subsets;
generating the precision recall curve from a validation training data subset for one or more values of a threshold parameter, the validation training data subset being one of the three training data subsets; and
selecting a value of the threshold parameter associated with a specified recall value.