Efficient and fault-tolerant distributed algorithm for learning latent factor models through matrix factorization
A method for estimating model parameters. The method comprises receiving a data set related to a plurality of users and associated content, partitioning the data set into a plurality of sub data sets in accordance with the users so that data associated with each user are not partitioned into more than one sub data set, storing each of the sub data sets in a separate one of a plurality of user data storages, each of said data storages being coupled with a separate one of a plurality of estimators, storing content associated with the plurality of users in a content storage, where the content storage is coupled to the plurality of estimators so that the content in the content storage is shared by the estimators, and estimating, asynchronously by each estimator, one or more parameters associated with a model based on data from one of the sub data sets.
1. A method implemented on a computer having at least one processor, a storage, and a communication platform for providing personalized content, comprising:
receiving a data set including information on interactions between a plurality of users and a plurality of pieces of content, wherein the plurality of users include multiple sets of users;
partitioning the data set into a plurality of sub data sets each of which corresponds to a separate set of the multiple sets of users;
associating each of the plurality of sub data sets with a corresponding one of a plurality of estimators;
storing one or more model parameters in a shared storage, wherein the shared storage is coupled to the plurality of estimators so that the plurality of estimators can asynchronously access the one or more model parameters;
estimating, asynchronously by each of the plurality of estimators, the one or more model parameters based on corresponding one of the plurality of sub data sets associated with the estimator and the stored one or more model parameters;
updating, asynchronously by each of the plurality of estimators, the one or more model parameters stored in the shared storage based on the asynchronously estimated one or more model parameters;
obtaining information related to a user; and
providing content personalized with respect to the user based on the information related to the user and the one or more model parameters stored in the shared storage.
2. The method of claim 1 , wherein the one or more of the model parameters include:
a first parameter indicating vectors of users in the corresponding sub data set, and
a second parameter indicating increments associated with at least a portion of the plurality of pieces of content in the corresponding sub data set.
3. The method of claim 2 , wherein the step of estimating further includes:
periodically updating the second parameter.
4. The method of claim 2 , wherein the step of estimating further comprises:
upon completion of the estimation, storing the first parameter to a respective one of a plurality of user data storages;
synchronizing the second parameters estimated by each of the plurality of estimators; and
storing the synchronized second parameters to the shared storage.
5. The method of claim 1 , wherein each of the plurality of estimators is implemented on a respective processor of a distributed computing system.
6. The method of claim 1 , wherein the one or more of the model parameters are estimated using a batch gradient descent approach.
7. A system having at least one processor for providing personalized content, the system comprising:
a modeling unit implemented on the at least one processor and configured to
receive a data set including information on interactions between a plurality of users and a plurality of pieces of content, wherein the plurality of users include multiple sets of users,
partition the data set into a plurality of sub data sets each of which corresponds to a separate set of the multiple sets of users, and
associating each of the plurality of sub data sets with a corresponding one of a plurality of estimators;
a content storage configured to store one or more model parameters in a shared storage, wherein the shared storage is coupled to the plurality of estimators so that the plurality of estimators can asynchronously access the one or more model parameters; and
the plurality of estimators, wherein each of the plurality of estimators is configured to
asynchronously estimate the one or more model parameters based on corresponding one of the plurality of sub data sets associated with the estimator and the stored one or more model parameters, and
asynchronously update the one or more model parameters stored in the shared storage based on the asynchronously estimated one or more model parameters, wherein the system is further configured to:
obtain information related to a user; and
provide content personalized with respect to the user based on the information related to the user and the one or more model parameters stored in the shared storage.
8. The system of claim 7 , wherein the one or more of the model parameters include:
a first parameter indicating vectors of users in the corresponding sub data set, and
a second parameter indicating increments associated with at least a portion of the plurality of pieces of content in the corresponding sub data set.
9. The system of claim 8 , wherein each estimator is further configured to periodically update the second parameter.
10. The system of claim 8 , wherein the estimators are configured to:
upon completion of the estimation, store the first parameter to a respective one of a plurality of user data storages;
synchronize the second parameters estimated by each of the plurality of estimators; and
store the synchronized second parameters to the shared storage.
11. The system of claim 7 , wherein the one or more of the model parameters are estimated using a batch gradient descent approach.
12. A non-transitory computer readable medium having recorded thereon information for providing personalized content, wherein the information, when read by a computer, causes the computer to perform the steps of:
receiving a data set including information on interactions between a plurality of users and a plurality of pieces of content, wherein the plurality of users include multiple sets of users;
partitioning the data set into a plurality of sub data sets each of which corresponds to a separate set of the multiple sets of users;
associating each of the plurality of sub data sets with a corresponding one of a plurality of estimators;
storing one or more model parameters in a shared storage, wherein the shared storage is coupled to the plurality of estimators so that the plurality of estimators can asynchronously access the one or more model parameters;
estimating, asynchronously by each of the plurality of estimators, the one or more model parameters based on corresponding one of the plurality of sub data sets associated with the estimator and the stored one or more model parameters;
updating, asynchronously by each of the plurality of estimators, the one or more model parameters stored in the shared storage based on the asynchronously estimated one or more model parameters;
obtaining information related to a user; and
providing content personalized with respect to the user based on the information related to the user and the one or more model parameters stored in the shared storage.
13. The medium of claim 12 , wherein the one or more of the model parameters include:
a first parameter indicating vectors of users in the corresponding sub data set, and
a second parameter indicating increments associated with at least a portion of the plurality of pieces of content in the corresponding sub data set.
14. The medium of claim 13 , wherein the step of estimating further includes:
periodically updating the second parameter.
15. The medium of claim 13 , wherein the step of estimating further comprises:
upon completion of the estimation, storing the first parameter to a respective one of a plurality of user data storages;
synchronizing the second parameters estimated by each of the plurality of estimators; and
storing the synchronized second parameters to the shared storage.
16. The medium of claim 12 , wherein each of the plurality of estimators is implemented on a respective processor of a distributed computing system.
17. The medium of claim 12 , wherein the one or more of the model parameters are estimated using a batch gradient descent approach.