Exploiting local inter-task relationships in adaptive multi-task learning
Methods, systems, and computer-readable storage media for receiving a dataset, the dataset including a plurality of data values, clustering data values of the plurality of data values into a plurality of input feature clusters in input feature space, training a local multi-task Gaussian process (MTGP) for each input feature cluster to provide optimized hyper-parameters in hyper-parameter space, an optimized hyper-parameter being provided for each input feature cluster, merging data values based on the optimized hyper-parameters, and distances between hyper-parameter clusters in the hyper-parameter space to provide a plurality of merged data values, and providing a LL-MTGP model based on the merged data values.
1. A computer-implemented method for adaptive multi-task learning (MTL), the method being executed by one or more processors and comprising:
receiving, by the one or more processors, a dataset, the dataset comprising a plurality of data values;
clustering, by the one or more processors, data values of the plurality of data values into a plurality of input feature clusters in input feature space;
training, by the one or more processors, a local multi-task Gaussian process (MTGP) for each input feature cluster to provide a set of local MTGPs, each local MTGP in the set of local MTGPs being associated with a respective input feature cluster and having optimized hyper-parameters in hyper-parameter space, an optimized hyper parameter being provided for each input feature cluster;
iteratively training a local learning MTGP (LL-MTGP) by:
merging hyper-parameters of two or more local MTGPs to provide a merged cluster representative of merged data values based on the optimized hyper-parameters,
initializing hyper-parameters of a local MTGP of the merged cluster as the hyper-parameters of one of the two or more local MTGPs, and
optimizing the hyper-parameters of the local MTGP of the merged cluster,
wherein iteratively training selectively ceases based on one or more distances between hyper-parameter clusters of the set of hyper-parameter clusters in the hyper-parameter space; and
providing, by the one or more processors, the LL-MTGP model to generate predictions.
2. The method of claim 1 , wherein the clustering is performed using k-means clustering.
3. The method of claim 1 , wherein merging comprises:
merging clusters having a minimum distance in the hyper-parameter space to provide the merged cluster.
4. The method of claim 3 , wherein iteratively training is performed until the minimum distance of the one or more distances is greater than a threshold distance.
5. The method of claim 1 , wherein each distance is determined as an averaged absolute distance between means of locally-learned inter-task similarities.
6. The method of claim 1 , wherein the merged data values are representative of local inter-task relationships in the adaptive MTL.
7. The method of claim 1 , further comprising providing the LL-MTGP model to an information retrieval system.
8. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for adaptive multi-task learning (MTL), the operations comprising:
receiving, by the one or more processors, a dataset, the dataset comprising a plurality of data values;
clustering, by the one or more processors, data values of the plurality of data values into a plurality of input feature clusters in input feature space;
training, by the one or more processors, a local multi-task Gaussian process (MTGP) for each input feature cluster to provide a set of local MTGPs, each local MTGP in the set of local MTGPs being associated with a respective input feature cluster and having optimized hyper-parameters in hyper-parameter space;
iteratively training a local learning MTGP (LL-MTGP) by:
merging hyper-parameters of two or more local MTGPs to provide a merged cluster representative of merged data values based on the optimized hyper-parameters,
initializing hyper-parameters of a local MTGP of the merged cluster as the hyper-parameters of one of the two or more local MTGPs, and
optimizing the hyper-parameters of the local MTGP of the merged cluster,
wherein iteratively training selectively ceases based on one or more distances between hyper-parameter clusters of the set of hyper-parameter clusters in the hyper-parameter space; and
providing, by the one or more processors, the LL-MTGP model to generate predictions.
9. The computer-readable storage medium of claim 8 , wherein the clustering is performed using k-means clustering.
10. The computer-readable storage medium of claim 8 , wherein merging comprises:
merging clusters having a minimum distance in the hyper-parameter space to provide the merged cluster.
11. The computer-readable storage medium of claim 10 , wherein iteratively training is performed until the minimum distance of the one or more distances is greater than a threshold distance.
12. The computer-readable storage medium of claim 8 , wherein each distance is determined as an averaged absolute distance between means of locally-learned inter-task similarities.
13. The computer-readable storage medium of claim 8 , wherein the merged data values are representative of local inter-task relationships in the adaptive MTL.
14. The computer-readable storage medium of claim 8 , wherein actions further comprise providing the LL-MTGP model to an information retrieval system.
15. A system, comprising:
a computing device; and
a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for adaptive multi-task learning (MTL), the operations comprising:
receiving, by the one or more processors, a dataset, the dataset comprising a plurality of data values;
clustering, by the one or more processors, data values of the plurality of data values into a plurality of input feature clusters in input feature space;
training, by the one or more processors, a local multi-task Gaussian process (MTGP) for each input feature cluster to provide a set of local MTGPs, each local MTGP in the set of local MTGPs being associated with a respective input feature cluster and having optimized hyper-parameters in hyper-parameter space;
iteratively training a local learning MTGP (LL-MTGP) by:
merging hyper-parameters of two or more local MTGPs to provide a merged cluster representative of merged data values based on the optimized hyper-parameters,
initializing hyper-parameters of a local MTGP of the merged cluster as the hyper-parameters of one of the two or more local MTGPs, and
optimizing the hyper-parameters of the local MTGP of the merged cluster,
wherein iteratively training selectively ceases based on one or more distances between hyper-parameter clusters of the set of hyper-parameter clusters in the hyper-parameter space; and
providing, by the one or more processors, the LL-MTGP model to generate predictions.
16. The system of claim 15 , wherein the clustering is performed using k-means clustering.
17. The system of claim 15 , wherein merging comprises:
merging clusters having a minimum distance in the hyper-parameter space to provide the merged cluster.
18. The system of claim 17 , wherein iteratively training is performed until the minimum distance of the one or more distances is greater than a threshold distance.
19. The system of claim 15 , wherein each distance is determined as an averaged absolute distance between means of locally-learned inter-task similarities.
20. The system of claim 15 , wherein the merged data values are representative of local inter-task relationships in the adaptive MTL.