Boosting algorithm for ranking model adaptation
Model adaptation may be performed to take a general model trained with a set of training data (possibly large), and adapt the model using a set of domain-specific training data (possibly small). The parameters, structure, or configuration of a model trained in one domain (called the background domain) may be adapted to a different domain (called the adaptation domain), for which there may be a limited amount of training data. The adaption may be performed using the Boosting Algorithm to select an optimal basis function that optimizes a measure of error of the model as it is being iteratively refined, i.e., adapted.
1. A server device comprising a computer hosting a search engine that uses an adaption model to rank documents in a search domain corresponding to a search domain of a query received by the computer server, the adaptation model being stored on the server device or another server device communicating with the server device, the adaptation model having been derived by a model adaptation process comprising:
receiving a trained background model that was trained before performance of the model adaptation process with labeled training data, the background model having been used to rank documents searched and found to match queries;
when starting a process for iteratively refining a base model, using the background model as the initial base model; and
iteratively refining the base model by repeatedly computing a residual error of the base model as applied to labeled training data, repeatedly computing a basis function that minimizes a difference between the residual error and the basis function as applied to the labeled training data, and repeatedly updating the base model according to the computed basis function, wherein a tree-based boosting algorithm is used as a tree feature generator, and wherein the basis function is combined with the base model in a way that reduces a magnitude of the basis function that is combined with the base model.
2. A server device according to claim 1 , wherein the basis function comprises a decision tree.
3. A server device according to claim 2 , wherein the process further comprises taking outputs of the decision tree as new features, and then retraining a ranking model, which comprises a new neural net model, on adaptation data using both the original features and newly generated tree features.
4. A server device according to claim 1 , wherein a tree-is used as a tree feature generator.
5. A server device comprising a computer hosting a search engine that uses an adaption model to rank documents in a search domain corresponding to a search domain of a query received by the computer server, the adaptation model being stored on the server device or another server device communicating with the server device, the adaptation model having been derived by a model adaptation process comprising:
receiving a trained background model that was trained before performance of the process with labeled training data, the background model having been used to rank documents searched and found to match queries;
when starting a process for iteratively refining a base model, using the background model as the initial base model;
iteratively refining the base model by repeatedly computing a residual error of the base model as applied to labeled training data, repeatedly computing a basis function that minimizes a difference between the residual error and the basis function as applied to the training data, and repeatedly updating the base model according to the computed basis function, wherein the basis function is combined with the base model in a way that reduces a magnitude of the basis function that is combined with the base model, wherein local linear model fitting is used to increase the speed of the iterative refining, the basis function comprises a decision tree and a tree-based boosting algorithm is used as a feature generator; and
taking outputs of the tree-based boosting algorithm on the decision tree as new features, and then retraining a ranking model on adaptation data using both original tree features and the new features.