Evaluating personalized recommendation models

1. A system for evaluating personalized recommendation models, the system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:

generate, using a personalized recommendation model, an object score for each object in a test set of objects, the test set of objects includes an interaction set of objects with which a user interacted and a random set of objects with which the user lacks a known interaction, the test set of objects configured for sorting the interaction set of objects with the random set of objects;

sort each scored object based on a decreasing order of each corresponding object score;

identify a high scoring set of the sorted objects with a number of highest corresponding object scores, wherein the number equals a total number of objects in the interaction set of objects;

aggregate a corresponding order value for each object in the high scoring set that is also in the interaction set of objects, wherein the corresponding order value for an object is based on a corresponding order for the object in the high scoring set; and

evaluate the personalized recommendation model for the user by dividing the aggregated order value by an aggregation of a corresponding order value for each object in the high scoring set.

2. The system of claim 1 , comprising further instructions, which when executed, cause the one or more processors to train the personalized recommendation model on a training set of objects, wherein the training set of objects comprise objects with which the user interacted, and excludes objects in the interaction set of objects.

3. The system of claim 1 , wherein the random set of objects with which the user lacks the known interaction comprises objects randomly selected from a plurality of objects which are available for and capable of scoring by the personalized recommendation engine.

4. The system of claim 1 , wherein aggregating the corresponding order value for each object in the high scoring set that is also in the interaction set of objects comprises weighting an order value for an object by an interaction value that corresponds to an interaction of the user with the object.

5. The system of claim 4 , wherein the aggregation of the corresponding order value for each object in the high scoring set is based on resorting each scored object based on a decreasing order of a corresponding interaction value and weighting an order value for each object by a corresponding interaction value.

6. The system of claim 1 , comprising further instructions, which when executed, cause the one or more processors to evaluate the personalized recommendation model for each of a plurality of users, which comprise the user.

7. The system of claim 6 , comprising further instructions, which when executed, cause the one or more processors to comprehensively evaluate a recommendation model upon which the personalized recommendation model is based by averaging the evaluation of the personalized recommendation model for each of the plurality of users.

8. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:

generate, using a personalized recommendation model, an object score for each object in a test set of objects, the test set of objects includes_an interaction set objects with which a user interacted and a random set of objects with which the user lacks a known interaction, the test set of objects configured for sorting the interaction set of objects with the random set of objects;

sort each scored object based on a decreasing order of each corresponding object score;

identify a high scoring set of the sorted objects with a number of highest corresponding object scores, wherein the number equals a total number of objects in the interaction set of objects;

aggregate a corresponding order value for each object in the high scoring set that is also in the interaction set of objects, wherein the corresponding order value for an object is based on a corresponding order for the object in the high scoring set; and

evaluate the personalized recommendation model for the user by dividing the aggregated order value by an aggregation of a corresponding order value for each object in the high scoring set.

9. The computer program product of claim 8 , wherein the program code comprises further instructions to train the personalized recommendation model on a training set of objects, wherein the training set of objects comprise objects with which the user interacted, and excludes objects in the interaction set of objects.

10. The computer program product of claim 8 , wherein the random set of objects with which the user lacks the known interaction comprise objects randomly selected from a plurality of objects which are available for and capable of scoring by the personalized recommendation engine.

11. The computer program product of claim 8 , wherein aggregating the corresponding order value for each object in the high scoring set that is also in the interaction set of objects comprises weighting an order value for an object by an interaction value that corresponds to an interaction of the user with the object.

12. The computer program product of claim 11 , wherein the aggregation of the corresponding order value for each object in the high scoring set is based on resorting each scored object based on a decreasing order of a corresponding interaction value and weighting an order value for each object by a corresponding interaction value.

13. The computer program product of claim 8 , wherein the program code comprises further instructions to evaluate the personalized recommendation model for each of a plurality of users, which comprise the user.

14. The computer program product of claim 13 , wherein the program code comprises further instructions to comprehensively evaluate a recommendation model upon which the personalized recommendation model is based by averaging the evaluation of the personalized recommendation model for each of the plurality of users.

15. A method for evaluating personalized recommendation models, the method comprising:

generating, using a personalized recommendation model, an object score for each object in a test set of objects, the test set of objects includes an interaction set of objects with which a user interacted and a random set of objects with which the user lacks a known interaction, the test set of objects configured for sorting the interaction set of objects with the random set of objects;

sorting each scored object based on a decreasing order of each corresponding object score;

identifying a high scoring set of the sorted objects with a number of highest corresponding object scores, wherein the number equals a total number of objects in the interaction set of objects;

aggregating a corresponding order value for each object in the high scoring set that is also in the interaction set of objects, wherein the corresponding order value for an object is based on a corresponding order for the object in the high scoring set; and

evaluating the personalized recommendation model for the user by dividing the aggregated order value by an aggregation of a corresponding order value for each object in the high scoring set.

16. The method of claim 15 , wherein the method further comprises training the personalized recommendation model on a training set of objects, wherein the training set of objects comprise objects with which the user interacted, and excludes objects in the interaction set of objects.

17. The method of claim 15 , wherein the random set of objects with which the user lacks the known interaction comprise objects randomly selected from a plurality of objects which are available for and capable of scoring by the personalized recommendation engine.

18. The method of claim 15 , wherein aggregating the corresponding order value for each object in the high scoring set that is also in the interaction set of objects comprises weighting an order value for an object by an interaction value that corresponds to an interaction of the user with the object.

19. The method of claim 18 , wherein the aggregation of the corresponding order value for each object in the high scoring set is based on resorting each scored object based on a decreasing order of a corresponding interaction value and weighting an order value for each object by a corresponding interaction value.

20. The method of claim 15 , wherein the method further comprises:

evaluating the personalized recommendation model for each of a plurality of users, which comprise the user; and

comprehensively evaluating a recommendation model upon which the personalized recommendation model is based by averaging the evaluation of the personalized recommendation model for each of the plurality of users.