Methods and apparatus for automatic attribute extraction for training machine learning models
This application relates to apparatus and methods for training machine learning models using supervised, or semi-supervised, learning. In some examples, a computing device obtains training data that includes labelled, and unlabeled, data for training a machine learning model. The computing device applies the machine learning model to the training data to generate output data. The machine learning model executes with a plurality of coefficients applied to a plurality of hyperparameters. The computing device further applies a loss model to the training data and the output data to generate a loss value. Based on the loss values, the computing device determines updated values for the plurality of coefficients of the machine learning model. The computing device may continue to determine updated values for the plurality of coefficients until one or more conditions are satisfied. The computing device may then store the final coefficient values in a data repository.
1. A system, comprising:
a computing device configured to:
obtain training data for training a machine learning model comprising a plurality of hyperparameters and a plurality of coefficients;
generate a first training dataset based on the obtained training data;
train the machine learning model by applying the machine learning model to the first training dataset to generate first output data;
apply a loss algorithm to the first training dataset and the first output data to generate a first loss value;
determine a first updated value for each of the plurality of coefficients based on the first loss value and a corresponding previous value of each of the plurality of coefficients;
in accordance with a determination that the first updated value for each of the plurality of coefficients is outside of a threshold of a last value:
generate a second training dataset based on the obtained training data, wherein the second training dataset is different from the first training dataset;
retrain the machine learning model by applying the machine learning model with the first updated values of the plurality of coefficients to the second training dataset to generate second output data;
apply the loss algorithm to the second training dataset and the second output data to generate a second loss value;
determine a second updated value for each of the plurality of coefficients based on the second loss value and the first updated value of each of the plurality of coefficients;
generate a comparison value based on the first updated value for each of the plurality of coefficients of the machine learning model and the second updated value for each of the plurality of coefficients for each of the plurality of coefficients of the machine learning model;
determine that the comparison value is less than or equal a predetermined threshold; and
store, in a database associated with the computing device, the first updated value and the second updated value for each of the plurality of coefficients in a data repository; and
apply the first updated value and the second update value for each of the plurality of coefficient in a data repository from a source external to the computing device to generate predicted outputs.
2. The system of claim 1 , wherein the computing device is further configured to determine that updating of the plurality of coefficients is not complete based on the first updated value and the corresponding previous value for at least one of the plurality of coefficients.
3. The system of claim 2 , wherein determining that updating of the plurality of coefficients is not complete comprises comparing the first updated value and the corresponding previous value for the at least one of the plurality of coefficients, and determining that a threshold is not met.
4. The system of claim 1 , wherein the loss algorithm is based on logistics regression.
5. The system of claim 1 , wherein the computing device is configured to:
receive the machine learning model from an item recommendation system; and
transmit the first updated value for each of the plurality of coefficients to the item recommendation system.
6. The system of claim 1 , wherein the training data comprises labelled and unlabeled training data.
7. The system of claim 1 , wherein the computing device is further configured to validate the machine learning model by:
applying the machine learning model to validation data to generate second output data; and
determining that a metric is satisfied based on the second output data.
8. The system of claim 1 , wherein the computing device is configured to randomly determine an initial value for each of the plurality of coefficients.
9. The system of claim 1 , wherein the computing device is configured to train the machine learning model by applying the machine learning model to the first training dataset to generate the first output data by using manifold learning-based techniques that leverage a framework for data-dependent regularization, and a geometry of a probability distribution.
10. The system of claim 1 , wherein the predetermined threshold is preconfigurable by a user via a user interface.
11. The system of claim 10 , wherein the user interface enables the user to select labelled and/or unlabeled data to be used for the training the machine learning model.
12. The system of claim 10 , wherein the user interface enables the user to select whether to train the machine learning model using supervised or semi-supervised learning.
13. The system of claim 12 , wherein the computing device comprises a processor and a non-transitory memory storing instructions, that when executed, cause the processor to apply a manifold learning-based algorithm that uses logistics regression as a classifier for labelled training data and/or unlabeled training data.
14. A method comprising:
obtaining training data for training a machine learning model comprising a plurality of hyperparameters and a plurality of coefficients;
generating a first training dataset based on the obtained training data;
training the machine learning model by applying the machine learning model to the first training dataset to generate first output data;
applying a loss algorithm to the first training dataset and the first output data to generate a first loss value;
determining a first updated value for each of the plurality of coefficients based on the first loss value and a corresponding previous value of each of the plurality of coefficients;
in accordance with a determination that the first updated value for each of the plurality of coefficients is outside of a threshold of a last value:
generating a second training dataset based on the obtained training data, wherein the second training dataset is different from the first training dataset;
retraining the machine learning model by applying the machine learning model with the first updated values of the plurality of coefficients to the second training dataset to generate second output data;
applying the loss algorithm to the second training dataset and the second output data to generate a second loss value;
determining a second updated value for each of the plurality of coefficients based on the second loss value and the first updated value of each of the plurality of coefficients;
generating a comparison value based on the first updated value for each of the plurality of coefficients of the machine learning model and the second updated value for each of the plurality of coefficients of the machine learning model;
determining that the comparison value is less than or equal a predetermined threshold; and
storing, in a database associated with a computing device, the first updated value and the second updated value for each of the plurality of coefficients in a data repository; and
applying the first updated value and the second update value for each of the plurality of coefficient in a data repository on data received from a source external to the computing device to generate predicted outputs.
15. The method of claim 14 wherein the training data comprises labelled and unlabeled training data.
16. The method of claim 14 , further comprising validating the machine learning model by:
applying the machine learning model to validation data to generate second output data; and
determining that a metric is satisfied based on the second output data.
17. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:
obtaining training data for training a machine learning model comprising a plurality of hyperparameters and a plurality of coefficients;
generating a first training dataset based on the obtained training data;
training the machine learning model by applying the machine learning model to the first training dataset to generate first output data;
applying a loss algorithm to the first training dataset and the first output data to generate a first loss value;
determining a first updated value for each of the plurality of coefficients based on the first loss value and a corresponding previous value of each of the plurality of coefficients;
in accordance with a determination that the first updated value for each of the plurality of coefficients is outside of a threshold of a last value:
generating a second training dataset based on the obtained training data, wherein the second training dataset is different from the first training dataset;
retraining the machine learning model by applying the machine learning model with the first updated values of the plurality of coefficients to the second training dataset to generate second output data;
applying the loss algorithm to the second training dataset and the second output data to generate a second loss value;
determining a second updated value for each of the plurality of coefficients based on the second loss value and the first updated value of each of the plurality of coefficients;
generating a comparison value based on the first updated value for each of the plurality of coefficients of the machine learning model and the second updated value for each of the plurality of coefficients of the machine learning model;
determining that the comparison value is less than or equal a predetermined threshold; and
storing, in a database associated with the device, the first updated value and the second updated value for each of the plurality of coefficients in a data repository; and
applying the first updated value and the second update value for each of the plurality of coefficient in a data repository on data received from a source external to the device to generate predicted outputs.
18. The non-transitory computer readable medium of claim 17 , wherein the training data comprises labelled and unlabeled training data.