Unlearning of recommendation models
The present disclosure describes techniques of performing machine unlearning in a recommendation model. An unlearning process of the recommendation model may be initiated in response to receiving a request for deleting a fraction of user data from any particular user. The recommendation model may be pre-trained to recommend content to users based at least in part on user data. Values of entries in a matrix corresponding to the fraction of user data may be configured as zero. The matrix may comprise entries denoting preferences of users with respect to content items. Confidence values associated with the fraction of user data may be configured as zero to block influence of the fraction of user data on performance of the recommendation model. The unlearning process may be implemented by performing a number of iterations until the recommendation model has converged.
1 . A method of performing machine unlearning in a recommendation model, the method implemented by a computer, and the method comprising:
initiating a process of unlearning associated with the recommendation model in response to receiving a request for deleting a fraction of user data associated with a user, wherein the recommendation model is a pre-trained machine learning model that recommends content to users based at least in part on user data, wherein initiating the process of unlearning associated with the recommendation model comprises:
configuring values of entries in a matrix corresponding to the fraction of the user data as zero to delete the fraction of the user data, wherein the matrix comprises entries denoting preferences of users with respect to content items; and
configuring confidence values associated with the deleted fraction of the user data as zero; and
implementing the process of unlearning after deleting the fraction of the user data and configuring the confidence values associated with the deleted fraction of the user data as zero, wherein the process of unlearning comprises performing a number of iterations to re-fit parameters of the recommendation model to a remainder of the user data using an algorithm without retraining the recommendation model from scratch on the remainder of the user data, wherein configuring the confidence values associated with the deleted fraction of the user data as zero blocks influence of the deleted fraction of the user data on the algorithm during the iterative re-fitting process.
2 . The method of claim 1 , wherein the unlearning process enables the recommendation model to perform with the same privacy as a retrained recommendation model.
3 . The method of claim 1 , wherein the algorithm comprises an Alternating Least Squares (ALS) algorithm.
4 . The method of claim 1 , wherein the recommendation model is a bi-linear model.
5 . The method of claim 1 , further comprising:
configuring the unlearning process to enable the recommendation model to have the same loss function as a retrained recommendation model.
6 . The method of claim 1 , further comprising:
determining that the recommendation model after the process of unlearning has approximately the same privacy as a retrained recommendation model using a membership inference attack.
7 . The method of claim 1 , further comprising:
determining the number of iterations based on an amount of the deleted fraction of the user data, wherein the number of iterations is inversely proportional to the amount of the deleted fraction of the user data.
8 . The method of claim 1 , wherein the number of iterations is equal to or greater than 10 and equal to or less than 45.
9 . A system, comprising:
at least one processor; and
at least one memory comprising computer-readable instructions that upon execution by the at least one processor cause the computing device to perform operations comprising:
initiating a process of unlearning associated with the recommendation model in response to receiving a request for deleting a fraction of user data associated with a user, wherein the recommendation model is a pre-trained machine learning model that recommends content to users based at least in part on user data, wherein initiating the process of unlearning associated with the recommendation model comprises:
configuring values of entries in a matrix corresponding to the fraction of the user data as zero to delete the fraction of the user data, wherein the matrix comprises entries denoting preferences of users with respect to content items; and
configuring confidence values associated with the deleted fraction of the user data as zero; and
implementing the process of unlearning after deleting the fraction of the user data and configuring the confidence values associated with the deleted fraction of the user data as zero, wherein the process of unlearning comprises performing a number of iterations to re-fit parameters of the recommendation model to a remainder of the user data using an algorithm without retraining the recommendation model from scratch on the remainder of the user data, wherein configuring the confidence values associated with the deleted fraction of the user data as zero blocks influence of the deleted fraction of the user data on the algorithm during the iterative re-fitting process.
10 . The system of claim 9 , wherein the unlearning process enables the recommendation model to perform with the same privacy as a retrained recommendation model.
11 . The system of claim 9 , wherein the algorithm comprises an Alternating Least Squares (ALS) algorithm.
12 . The system of claim 9 , the operations further comprising:
configuring the unlearning process to enable the recommendation model to have the same loss function as a retrained recommendation model.
13 . A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations, the operation comprising:
initiating a process of unlearning associated with the recommendation model in response to receiving a request for deleting a fraction of user data associated with a user, wherein the recommendation model is a pre-trained machine learning model that recommends content to users based at least in part on user data, wherein initiating the process of unlearning associated with the recommendation model comprises:
configuring values of entries in a matrix corresponding to the fraction of the user data as zero to delete the fraction of the user data, wherein the matrix comprises entries denoting preferences of users with respect to content items; and
configuring confidence values associated with the deleted fraction of the user data as zero; and
implementing the process of unlearning after deleting the fraction of the user data and configuring the confidence values associated with the deleted fraction of the user data as zero, wherein the process of unlearning comprises performing a number of iterations to re-fit parameters of the recommendation model to a remainder of the user data using an algorithm without retraining the recommendation model from scratch on the remainder of the user data, wherein configuring the confidence values associated with the deleted fraction of the user data as zero blocks influence of the deleted fraction of the user data on the algorithm during the iterative re-fitting process.
14 . The non-transitory computer-readable storage medium of claim 13 , wherein the unlearning process enables the recommendation model to perform with the same privacy as a retrained recommendation model.
15 . The non-transitory computer-readable storage medium of claim 13 , wherein the algorithm comprises an Alternating Least Squares (ALS) algorithm.
16 . The non-transitory computer-readable storage medium of claim 13 , the operations further comprising:
configuring the unlearning process to enable the recommendation model to have the same loss function as a retrained recommendation model.
17 . The non-transitory computer-readable storage medium of claim 13 , the operations further comprising:
determining the number of iterations based on an amount of the deleted fraction of the user data, wherein the number of iterations is inversely proportional to the amount of the deleted fraction of the user data.
18 . The non-transitory computer-readable storage medium of claim 13 , wherein the number of iterations is equal to or greater than 10 and equal to or less than 45.