Domain adaptation
View Patent ↗A computer implement method for domain adaptation. According to the method, training data from a target domain may be classified by using a group of source models of a source domain, where each of the source models is trained to classify data from the source domain. A first pseudo label indicating a category of the training data may be generated by aggregating classification results of the source models based on respective weights of the source models on predetermined data categories. A target model of the target domain may be trained based on the training data and the first pseudo label and the second pseudo label, where the target model is trained to classify data from the target domain.
1. A computer-implemented method for domain adaptation comprising:
classifying, by one or more processors, training data from a target domain by using a group of source models of a source domain, wherein each of the source models is trained to classify data from the source domain;
generating, by one or more processors, a first pseudo label indicating a category of the training data by aggregating classification results of the source models based on respective weights of the source models on predetermined data categories;
generating, by one or more processors, a second pseudo label indicating the category of the training data by comparing the training data with a plurality of groups of data samples from the target domain; and
training, by one or more processors, a target model of the target domain based on the training data and the first pseudo label and the second pseudo label, wherein the target model is trained to classify data from the target domain, and the training further comprises:
generating at least one label for the training data by classifying the training data with the target model;
determining a loss of the target model based on the at least one label, the first pseudo label and the second pseudo label; and
updating at least one parameter of the target model such that the loss is minimized.
2. The method of claim 1 , wherein
each group of data samples relates to a corresponding one of the predetermined data categories.
3. The method of claim 2 , further comprising:
initializing, by one or more processors, the respective weights of the source models on the predetermined data categories; and
adjusting, by one or more processors, the respective weights of the source models on the predetermined data categories based on the plurality of groups of data samples.
4. The method of claim 3 , wherein adjusting the respective weights of the source models on the predetermined data categories comprises:
classifying, by one or more processors, a group of data samples relating to each of the predetermined data categories by using both the source models and the target model;
determining, by one or more processors, differences among classification results of the source models and classification results of the target model; and
adjusting, by one or more processors, the respective weights based on the differences.
5. The method of claim 2 , wherein training the target model of the target domain based on the training data, the first pseudo label and the second pseudo label comprises:
generating, by one or more processors, at least one label for the training data by classifying the training data with the target model;
determining, by one or more processors, a loss of the target model based on the at least one label, the first pseudo label and the second pseudo label; and
updating, by one or more processors, at least one parameter of the target model such that the loss is minimized.
6. The method of claim 5 , wherein generating the at least one label comprises:
obtaining, by one or more processors, a feature of the training data from a feature extraction layer of the target model; and
inputting, by one or more processors, the feature to at least one output layer of the target model for classifying the training data.
7. The method of claim 6 , wherein the at least one label comprises a first label and a second label generated with different output layers, and determining the loss of the target model comprises:
determining, by one or more processors, a first loss based on the first label and the first pseudo label;
determining, by one or more processors, a second loss based on the second label and the second pseudo label; and
determining, by one or more processors, the loss of the target model based on the first loss and the second loss.
8. The method of claim 2 , further comprising:
classifying, by one or more processors, each data sample in a set of data samples from the target domain by using the target model;
determining, by one or more processors, a confidence level of a classification result for each data sample in the set of data samples; and
determining, by one or more processors, the plurality of groups of data samples by selecting a group of data samples for each of the predetermined data categories from the set of data samples, wherein each of the selected data samples is associated with a confidence level exceeding a predetermined threshold.
9. The method of claim 8 , further comprising:
in response to the target model being updated during the training, updating, by one or more processors, the plurality of groups of data samples by using the updated target model.
10. The method of claim 1 , further comprising:
obtaining, by one or more processors, target data to be classified from the target domain; and
generating, by one or more processors, a label indicating a category of the target data by using the trained target model.
11. A system for domain adaptation comprising:
a processing unit; and
a memory coupled to the processing unit and storing instructions thereon, the instructions, when executed by the processing unit, are configured to cause the processing unit to:
classify training data from a target domain by using a group of source models of a source domain, wherein each of the source models is trained to classify data from the source domain;
generate a first pseudo label indicating a category of the training data by aggregating classification results of the source models based on respective weights of the source models on predetermined data categories;
generate, by one or more processors, a second pseudo label indicating the category of the training data by comparing the training data with a plurality of groups of data samples from the target domain;
train a target model of the target domain based on the training data and the first pseudo label, wherein the target model is trained to classify data from the target domain;
classify each data sample in a set of data samples from the target domain using the target model;
determine a confidence level of a classification result for each data sample in the set of data samples; and
determine the plurality of groups of data samples by selecting a group of data samples for each of the predetermined data categories from the set of data samples, wherein each of the selected data samples is associated with a confidence level exceeding a predetermined threshold.
12. The system of claim 11 , wherein
each group of data samples relates to a corresponding one of the predetermined data categories; and
wherein the training the target model of the target domain is further based on the second pseudo label.
13. The system of claim 12 , wherein the instruction are further configured to cause the processing unit to:
initialize the respective weights of the source models on the predetermined data categories; and
adjust the respective weights of the source models on the predetermined data categories based on the plurality of groups of data samples.
14. The system of claim 12 , wherein training the target model of the target domain based on the training data, the first pseudo label and the second pseudo label comprises:
generate at least one label for the training data by classifying the training data with the target model;
determine a loss of the target model based on the at least one label, the first pseudo label and the second pseudo label; and
update at least one parameter of the target model such that the loss is minimized.
15. The system of claim 11 , wherein the instruction are further configured to cause the processing unit to:
obtain target data to be classified from the target domain; and
generate a label indicating a category of the target data by using the trained target model.
16. A computer program product being tangibly stored on a non-transitory machine-readable medium and comprising machine-executable instructions, the instructions, when executed on a device, causing the device to perform actions comprising:
classifying training data from a target domain by using a group of source models of a source domain, wherein each of the source models is trained to classify data from the source domain;
generating a first pseudo label indicating a category of the training data by aggregating classification results of the source models based on respective weights of the source models on predetermined data categories; and
training a target model of the target domain based on the training data and the first pseudo label, wherein the target model is trained to classify data from the target domain
adjust, by one or more processors, the respective weights of the source models on the predetermined data categories based on a plurality of groups of data samples, wherein the adjustment further comprises:
classifying, a group of data samples relating to each of the predetermined data categories by using both the source models and the target model;
determining differences among classification results of the source models and classification results of the target model; and
adjusting the respective weights based on the differences.
17. The computer program product of claim 16 , wherein training the target model of the target domain based on the training data and the first pseudo label comprises:
generating a second pseudo label indicating a category of the training data by comparing the training data with the plurality of groups of data samples from the target domain, wherein each group of data samples relates to a corresponding one of the predetermined data categories; and
wherein the training the target model of the target domain is further based on the second pseudo label.
18. The computer program product of claim 17 , wherein the actions further comprise:
initializing the respective weights of the source models on the predetermined data categories.
19. The computer program product of claim 17 , wherein training the target model of the target domain based on the training data, the first pseudo label and the second pseudo label comprises:
generating at least one label for the training data by classifying the training data with the target model;
determining a loss of the target model based on the at least one label, the first pseudo label and the second pseudo label; and
updating at least one parameter of the target model such that the loss is minimized.
20. The computer program product of claim 16 , wherein the actions further comprise:
obtaining target data to be classified from the target domain; and
generating a label indicating a category of the target data by using the trained target model.