IP Library › Granted Patent US 12,294,529
Granted Patent B2
US 12,294,529 · App. 18/342,516 · Granted May 6, 2025

Transferable clustering of contextual bandits for cloud service resource allocation

Inventors: Kanak Mahadik (San Jose, CA); Tong Yu (San Jose, CA); Junda Wu (San Jose, CA)
Assignee: Adobe Inc.
H04L47/781H04L41/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,294,529
App. No.
18/342,516
Granted
May 6, 2025
Kind
B2
Abstract

Methods for determining optimal cloud service resource include determining a reward function for a set of resource configurations identifying cloud service resource parameters. The cloud service resource parameters include a source parameter and a target parameter of services to provide a client computing device. A source parameter dataset for the source parameter and a target parameter dataset is generated using the reward function and historical source parameter data. The matrices are then subject to SVD and clustering. A target parameter reward dataset is learned from output of the SVD and clustering. The target parameter dataset is used to determine the parameters for the target parameter for providing corresponding cloud service resources.

Claims (32)

1. A system for cloud service resource allocation, the system comprising:

at least one processor; and

one or more computer storage media storing computer readable instructions thereon that when executed by the at least one processor cause the at least one processor to perform operations comprising:

determining a reward function for a set of resource configurations identifying cloud service resource parameters, wherein the cloud service resource parameters include a source parameter and a target parameter of services to provide a client computing device;

generating a source parameter dataset for the source parameter using the reward function and historical source parameter data, the source parameter dataset corresponding to a source domain;

learning, from the source parameter dataset, a target parameter reward dataset for the target parameter, the target parameter reward dataset corresponding to a target domain different from the source domain, and each of the source domain and the target domain corresponding to different resource device types, whereby knowledge is transferred from the source domain to the different target domain by the learning; and

providing a cloud service resource corresponding to the target parameter derived from the target parameter reward dataset.

2. The system of claim 1 , wherein the source parameter dataset is a source parameter matrix that comprises source parameter vectors determined for the resource configurations in the set of resource configurations using the reward function.

3. The system of claim 1 , wherein the learning operation comprises employing a machine learning technique on the source parameter dataset.

4. The system of claim 3 , wherein the machine learning technique is singular value decomposition (SVD).

5. The system of claim 1 , further comprising clustering a set of pseudo service feature vectors in the source domain corresponding to the source parameter to determine clustering centroids as source domain clustering representations.

6. The system of claim 1 , wherein the source parameter is a central processing unit (CPU).

7. The system of claim 1 , wherein the target parameter is a graphics processing unit (GPU).

8. A method performed by one or more processors for cloud service resource allocation, the method comprising:

generating a source parameter dataset for a source parameter using a reward function and historical source parameter data, the reward function corresponding to a set of resource configurations that identify cloud service resource parameters, wherein the cloud service resource parameters include the source parameter and a target parameter of services to provide a client computing device;

learning, from the source parameter dataset, a target parameter reward dataset for the target parameter, the source parameter dataset corresponding to a source domain and the target parameter reward dataset corresponding to a target domain different from the source domain, and each of the source domain and the target domain corresponding to different resource device types, whereby knowledge is transferred from the source domain to the different target domain by the learning; and

providing a cloud service resource corresponding to the target parameter derived from the target parameter reward dataset.

9. The method of claim 8 , wherein the source parameter dataset is a source parameter matrix that comprises source parameter vectors determined for the resource configurations in the set of resource configurations using the reward function.

10. The method of claim 8 , wherein the learning comprises employing a machine learning technique on the source parameter dataset.

11. The method of claim 10 , wherein the machine learning technique is singular value decomposition (SVD).

12. The method of claim 8 , further comprising clustering a set of pseudo service feature vectors in the source domain corresponding to the source parameter to determine clustering centroids as source domain clustering representations.

13. The method of claim 8 , wherein the source parameter is a central processing unit (CPU).

14. The method of claim 8 , wherein the target parameter is a graphics processing unit (GPU).

15. One or more computer storage media storing computer readable instructions thereon that, when executed by a processor, cause the processor to perform a method for cloud service resource allocation, the method comprising:

accessing a source parameter dataset for a source parameter using a reward function and historical source parameter data, the reward function corresponding to a set of resource configurations that identify cloud service resource parameters, wherein the cloud service resource parameters include the source parameter and a target parameter of services to provide a client computing device;

learning, from the source parameter dataset, a target parameter reward dataset for the target parameter, the source parameter dataset corresponding to a source domain and the target parameter reward dataset corresponding to a target domain different from the source domain, and each of the source domain and the target domain corresponding to different resource device types, whereby knowledge is transferred from the source domain to the different target domain by the learning; and

providing a cloud service resource corresponding to the target parameter derived from the target parameter reward dataset.

16. The media of claim 15 , wherein the source parameter dataset is a source parameter matrix that comprises source parameter vectors determined for the resource configurations in the set of resource configurations using the reward function.

17. The media of claim 15 , wherein the learning comprises employing a machine learning technique on the source parameter dataset.

18. The media of claim 17 , wherein the machine learning technique is singular value decomposition (SVD).

19. The media of claim 15 , further comprising clustering a set of pseudo service feature vectors in the source domain corresponding to the source parameter to determine clustering centroids as source domain clustering representations.

20. The media of claim 15 , wherein the source parameter is a central processing unit (CPU), and the target parameter is a graphics processing unit (GPU).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2023
From: MAHADIK, KANAK; YU, TONG; WU, JUNDA
To: ADOBE INC.
Reel/Frame 064203/0479 →
Continuity (1)
Related Publication 20250007858A1 · Jan 2, 2025
References Cited (47)
US 10089575B1 · Redmon · 2018 [cited by examiner]
US 10877863B2 · Choi · 2020 [cited by examiner]
US 11776542B1 · Bueche · 2023 [cited by examiner]
US 12106180B2 · Nelson · 2024 [cited by examiner]
US 20100121850A1 · Moitra · 2010 [cited by examiner]
US 20180018590A1 · Szeto · 2018 [cited by examiner]
US 20190242608A1 · Laftchiev · 2019 [cited by examiner]
US 20190370927A1 · Frenkel · 2019 [cited by examiner]
US 20200150622A1 · Cai · 2020 [cited by examiner]
US 20210191759A1 · Fong · 2021 [cited by examiner]
US 20210357255A1 · Mahadik · 2021 [cited by examiner]
US 20220114225A1 · Murugesan · 2022 [cited by examiner]
US 20230032853A1 · Mahadik · 2023 [cited by examiner]
US 20230195723A1 · Kaku · 2023 [cited by examiner]
US 20240320513A1 · Sun · 2024 [cited by examiner]
Abdelbaky, M., et al., “Docker containers across multiple clouds and data centers”, In 2015 IEEE/ACM 8th International Conference on Utility and Cloud computing (UCC), pp. 1-4 (Dec. 2015). [cited by applicant]
Auer, P., et al., “The non-stochastic multi-armed bandit problem”, SIAM journal on computing, vol. 32, No. 1, pp. 1-33 (2002). [cited by applicant]
Burns, B., et al., “Borg, Omega, and Kubernetes: Lessons learned from three container-management systems over a decade”, ACM Queue, vol. 14, No. (1), pp. 1-24 (2016). [cited by applicant]
Calheiros, R. N., et al., “Workload prediction using ARIMA model and its impact on cloud applications' QoS”, IEEE transactions on cloud computing, vol. 3, No. 4, pp. 1-11 (2014). [cited by applicant]
Burns, B., et al., “Kubernetes: up and running: dive into the future of infrastructure”, O'Reilly Media, Inc., pp. 1-277 (2017). [cited by applicant]
Hindman, B., et al., “Mesos: A platform for fine-grained resource sharing in the data center”, In NSDI'11: Proceedings of the 8th USENIX conference on Networked systems design and implementation, vol. 11, pp. 1-14 (Mar.… [cited by applicant]
Lattimore, T., and Szepesvari, C., “Bandit algorithms”, Cambridge University Press, pp. 1-596 (2020). [cited by applicant]
Lorido-Botran, T., et al., “A review of auto-scaling techniques for elastic applications in cloud environments”, Journal of Grid Computing, vol. 12, No. 4, pp. 1-26 (2014). [cited by applicant]
Mauro, T., “Adopting microservices at netflix: Lessons for architectural design”, DZone Integration, published on Apr. 7, 2015, Retrieved from Internet URL: https://dzone.com/articles/adopting-microservices-netflix, acc… [cited by applicant]
Park, Y., et al., “Linear quadratic regulator for resource-efficient cloud services”, In Proceedings of the ACM Symposium on Cloud Computing, p. 1 (Nov. 2019). [cited by applicant]
White, J.M., “Bandit algorithms for website optimization”, O'Reilly Media, Inc., pp. 1-87 (2012). [cited by applicant]
Auer, P., et al., “Finite-time analysis of the multiarmed bandit problem”, Machine Learning, vol. 47, pp. 235-256 (2002). [cited by applicant]
Russo, D. J., et al., “A tutorial on thompson sampling”, Foundations and Trends® in Machine Learning, vol. 11, No. 1, pp. 1-39 (2018). [cited by applicant]
Sutton, R. S., and Barto, A.G., “Reinforcement learning: An introduction”, MIT press, pp. 1-398 (2018). [cited by applicant]
Rossi, F., et al., “Horizontal and vertical scaling of container-based applications using reinforcement learning,” in 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 329-338 (2019). [cited by applicant]
Tesauro, G., et al., “A hybrid reinforcement learning approach to autonomic resource allocation,” in 2006 IEEE International Conference on Autonomic Computing, pp. 1-9 (2006). [cited by applicant]
Delande, D., et al., “Horizontal scaling in cloud using contextual bandits”, in European Conference on Parallel Processing, Springer, pp. 285-300 (2021). [cited by applicant]
Lu, T., et al., “Contextual multi-armed bandits,” Appearing in Proceedings of the Thirteenth international conference on Artificial Intelligence and Statistics, pp. 485-492 (2010). [cited by applicant]
Li, L., et al., “A contextual-bandit approach to personalized news article recommendation”, in Proceedings of the 19th international conference on World wide web, pp. 1-10 (2010). [cited by applicant]
Li, S., et al., “Improved algorithm on online clustering of bandits”, in Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 1-13 (2019). [cited by applicant]
Gentile, C., et al., “Online clustering of bandits”, Proceedings of the 31st International Conference on Machine Learning, PMLR, pp. 1-9 (2014). [cited by applicant]
Gentile, C., et al., “On context-dependent clustering of bandits”, Proceedings of the 34th International Conference on Machine Learning, ICML, pp. 1-10 (2017). [cited by applicant]
Nguyen, T.T., and Lauw, H.W., “Dynamic clustering of contextual multi-armed bandits”, in Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1-5 (2014). [cited by applicant]
Zhu, J-Y., et al., “Unpaired image-to-image translation using cycle-consistent adversarial networks”, in Proceedings of the IEEE international conference on computer vision (ICCV), pp. 2223-2232 (2017). [cited by applicant]
Ganin, Y., et al., “Domain-adversarial training of neural networks”, The journal of machine learning research, vol. 17, No. 1, pp. 1-35 (2016). [cited by applicant]
Eysenbach, B., et al., “Off-dynamics reinforcement learning: Training for transfer with domain classifiers”, arXiv preprint arXiv:2006.13916v1, pp. 1-20 (Jun. 24, 2020). [cited by applicant]
Wu, J., et al., “Dynamics-aware adaptation for reinforcement learning based cross-domain interactive recommendation”, in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Informat… [cited by applicant]
Liu, B., et al., “Transferable contextual bandit for cross-domain recommendation”, in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, pp. 3619-3626 (2018). [cited by applicant]
Lloyd, S.P., “Least squares quantization in PCM”, IEEE transactions on information theory, vol. 28, No. 2, pp. 129-137 (1982). [cited by applicant]
Harper, F.M., and Konstan, J.A., “The movielens datasets: History and context”, ACM Transactions on Interactive Intelligent Systems (TIIS), vol. 5, No. 4, pp. 1-20 (2015). [cited by applicant]
Tong Yu, et al., U.S. Appl. No. 18/120,088, filed Mar. 10, 2023, title “Resource Provisioning by a Reinforcement Learning Framework Using Guided Ordering of Services”. [cited by applicant]
Kanak Vivek Mahadik, et al., U.S. Appl. No. 17/749,577, filed May 20, 2022, title “Efficient Adaptive Allocation of Resoures for Computational Systems via Statistically Derived Linear Models”. [cited by applicant]