IP Library › Granted Patent US 12,332,745
Granted Patent B2
US 12,332,745 · App. 18/423,125 · Granted Jun 17, 2025

Distributed package management using meta-scheduling

Inventors: Connor Patric Doyle (San Francisco, CA); Thomas Rampelberg (San Francisco, CA); Cody Maloney (San Francisco, CA); José Armando García Sancio (San Francisco, CA)
Assignee: Nutanix, Inc.
G06F11/1438G06F8/61G06F9/48G06F11/0793H04L67/34G06F9/44505G06F11/3409
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,332,745
App. No.
18/423,125
Filed
Jan 25, 2024
Granted
Jun 17, 2025
Kind
B2
Art Unit
2113
USPC
714/15
Abstract

A system for package management includes an interface and a processor. The interface is to receive an indication to install a package. The processor is to determine a configured package using a set local configuration properties and using the package and to launch, using a metascheduler, a set of subschedulers to install a plurality of applications of the configured package.

Claims (40)

1. A method, comprising:

receiving, at a cluster manager, an indication to install a package;

determining, using a processor, a configured package using a local configuration property for the package;

providing instructions from the cluster manager to a meta-scheduler to schedule installation of the configured package;

performing the installation by a sub-scheduler after the meta-scheduler schedules the installation, the sub-scheduler located at each node of a cluster and providing instructions to an agent;

performing, by an agent located on a node, for an installation task to install at least a portion of the configured package at the node to provision software services provided by the package, wherein the agent reports success or failure to the sub-scheduler; and

providing, by the sub-scheduler, a report regarding the installation, wherein the meta-scheduler takes a remedial action regarding the installation based upon the report in order to take the remedial action.

2. The method of claim 1 , wherein the sub-scheduler installs an application of a plurality of applications of the configured package on one or more nodes of the cluster.

3. The method of claim 1 , wherein the agent performs the installation task and generates a task result that is provided to the sub-scheduler, the meta-scheduler, or a cluster manager.

4. The method of claim 1 , wherein the remedial action kills an existing sub-scheduler process for the installation and re-starts a sub-scheduler process.

5. The method of claim 1 , wherein the remedial action re-starts the meta-scheduler.

6. The method of claim 1 , wherein the sub-scheduler restarts the installation of an application.

7. The method of claim 1 , wherein both the meta-scheduler and the sub-scheduler are hierarchically higher than the agent for the installation in that the meta-scheduler is integrated with the sub-scheduler on the node for execution of the installation task by the agent as directed by the meta-scheduler or the sub-scheduler.

8. A system, comprising:

a processor;

a memory for holding programmable code; and

wherein the programmable code includes instructions which, when executed by the processor, causes the processor to execute a set of acts, the set of acts comprising:

receiving, at a cluster manager, an indication to install a package;

determining, using a processor, a configured package using a local configuration property for the package;

providing instructions from the cluster manager to a meta-scheduler to schedule installation of the configured package;

performing the installation by a sub-scheduler after the meta-scheduler reschedules the installation, the sub-scheduler located at each node of a cluster and providing instructions to an agent;

performing, by the agent located on a node, an installation task to install at least a portion of the configured package at the node to provision software services provided by the package, wherein the agent reports success or failure to the sub-scheduler; and

providing, the sub-scheduler, a report regarding the installation, wherein the meta-scheduler takes a remedial action regarding the installation based upon the report in order to take the remedial action.

9. The system of claim 8 , wherein the sub-scheduler installs an application of a plurality of applications of the configured package on one or more nodes of the cluster.

10. The system of claim 8 , wherein the agent performs the installation task and generates a task result that is provided to the sub-scheduler, the meta-scheduler, or a cluster manager.

11. The system of claim 8 , wherein the remedial action kills an existing sub-scheduler process for the installation and re-starts a sub-scheduler process.

12. The system of claim 8 , wherein the remedial action re-starts the meta-scheduler.

13. The system of claim 8 , wherein the sub-scheduler restarts the installation of an application.

14. A computer program product comprising a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor, executes at least:

receiving, at a cluster manager, an indication to install a package;

determining, using a processor, a configured package using a local configuration property for the package;

providing instructions from the cluster manager to a meta-scheduler to schedule installation of the configured package;

performing the installation by the sub-scheduler the meta-scheduler schedules the installation, the sub-scheduler located at each node of c cluster and providing instructions to an agent;

performing, by the agent located on a node, an installation task to install at least a portion of the configured package at the node to provision software services provided by the package, wherein the agent reports success or failure to the sub-scheduler; and

providing, by the sub-scheduler, a report regarding the installation, wherein the meta-scheduler takes a remedial action regarding the installation based upon the report in order to take the remedial action.

15. The computer program product of claim 14 , wherein the sub-scheduler installs an application of a plurality of applications of the configured package on one or more nodes of the cluster.

16. The computer program product of claim 14 , wherein the agent performs the task and generates a task result that is provided to the sub-scheduler, the meta-scheduler, or a cluster manager.

17. The computer program product of claim 14 , wherein the remedial action kills an existing sub-scheduler process for the installation and re-starts a sub-scheduler process.

18. The computer program product of claim 14 , wherein the remedial action re-starts the meta-scheduler.

19. The computer program product of claim 14 , wherein the sub-scheduler restarts installation of an application.

Assignments (1)
SECURITY INTEREST Recorded Feb 13, 2025
From: NUTANIX, INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 070206/0463 →
Continuity (6)
Continuation 17957803 · Sep 30, 2022
Continuation 16679771 · Nov 11, 2019
Continuation 15829740 · Dec 1, 2017
Continuation 14931714 · Nov 3, 2015
Provisional Application 62202047 · Aug 6, 2015
Related Publication 20240311252A1 · Sep 19, 2024
References Cited (91)
US 5666501A · Jones · 1997 [cited by examiner]
US 7984268B2 · Hass et al. · 2011 [cited by applicant]
US 8347337B2 · Barton et al. · 2013 [cited by applicant]
US 8549518B1 · Aron et al. · 2013 [cited by applicant]
US 8601473B1 · Aron et al. · 2013 [cited by applicant]
US 8819266B2 · Salonikios · 2014 [cited by applicant]
US 8850130B1 · Aron et al. · 2014 [cited by applicant]
US 9058233B1 · Zeng · 2015 [cited by applicant]
US 9058251B1 · Dorwin et al. · 2015 [cited by applicant]
US 9304803B2 · Nelson et al. · 2016 [cited by applicant]
US 9311066B1 · Garman · 2016 [cited by examiner]
US 9772866B1 · Aron et al. · 2017 [cited by applicant]
US 10162619B1 · Lanner et al. · 2018 [cited by applicant]
US 10684840B1 · Hussain · 2020 [cited by examiner]
US 20020078186A1 · Engel et al. · 2002 [cited by applicant]
US 20040243995A1 · Sheehy · 2004 [cited by applicant]
US 20060080656A1 · Cain et al. · 2006 [cited by applicant]
US 20080072230A1 · Jackson · 2008 [cited by applicant]
US 20130031527A1 · Granholm et al. · 2013 [cited by applicant]
US 20140006620A1 · Assuncao et al. · 2014 [cited by applicant]
US 20140298324A1 · Voronkov · 2014 [cited by examiner]
Schwarzkopf, M., et al., “Omega: flexible, scalable schedulers for large compute clusters,” Google Research, dated 2013. [cited by applicant]
“Introduction to Kubernetes,” Slideshare, URL: https://www.slideshare.net/rajdeep/introduction-to-kubernetes, dated Sep. 8, 2014. [cited by applicant]
“Can't create pods from yaml template,” Github, URL: https://github.com/kubernetes/kubernetes/issues/2763, dated Dec. 5, 2014. [cited by applicant]
“Release Kubernetes v0.2,” Github, URL: https://github.com/kubernetes/kubernetes/commit/a0abb3815755d6a77eed2d07bb0aa7d255e4e769, dated Sep. 8, 2014. [cited by applicant]
“Product Details,” AWS, URL: https://web.archive.org/web/20141115184108/http:/aws.amazon.com/ecs/details, date found via Internet Archive as Nov. 15, 2014. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/957,803 dated Feb. 13, 2023. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/957,803 dated Feb. 22, 2024. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/957,803 dated Oct. 10, 2023. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/679,771 dated Mar. 2, 2022. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/679,771 dated Jul. 27, 2022. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/829,740 dated Aug. 12, 2019. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 14/931,714 dated Apr. 27, 2017. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 14/931,714 dated Oct. 6, 2017. [cited by applicant]
“Packages”, VMWare Docs, URL: https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/2/about-tkg/packages-index.html, dated Apr. 19, 2024. [cited by applicant]
“Package Management,” kapp-controller, URL: https://carvel.dev/kapp-controller/docs/v0.48.x/packaging/#package, date found via Google as May 5, 2021. [cited by applicant]
“Tanzu CLI Package Commands,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-Tanzu-Packages/2023.11.21/tanzu-packages/ref.html, dated Apr. 8, 2024. [cited by applicant]
“About ytt,” Carvel, URL: https://carvel.dev/ytt/docs/v0.46.x/, date found via Google as Feb. 24, 2022. [cited by applicant]
Denneman, F., “Initial Placement of a vSphere Pod,” frankdenneman.nl, dated Mar. 6, 2020. [cited by applicant]
“Product Details,” Amazon Web Services, URL: https://web.archive.org/web/20141115184108/http:/aws.amazon.com/ecs/details, date found via Internet Archive as Nov. 15, 2014. [cited by applicant]
“Deploying Workloads to vSphere Pods,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-with-tanzu-services-workloads/GUID-8574A735-C02D-48BA-A266-FBA7803F7C3A.html, dated Oct. 11, 2023. [cited by applicant]
“What Is a vSphere Pod?,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-276F809D-2015-4FC6-92D8-8539D491815E.html, dated Dec. 7, 2022. [cited by applicant]
Ranjithbp, “vSphere 7 with Kubernetes—vSphere Pods and/vs Tanzu Kubernetes Grid Cluster,” Vconnect-It, URL: https://vconnectit.wordpress.com/2020/10/01/vsphere-7-with-kubernetes-vsphere-pods-and-vs-tanzu-kubernetes-grid… [cited by applicant]
Obeythefist Answer Post, Exam 5V0-23.20 Topic 1 Question 40 Discussion, URL: https://www.examtopics.com/discussions/vmware/view/90098-exam-5v0-2320-topic-1-question-40-discussion/, dated Apr. 15, 2023. [cited by applicant]
West, M., “vSphere with Tanzu—Highly Available Kubernetes,” VMWare, URL: https://core.vmware.com/resource/vsphere-tanzu-highly-available-kubernetes#section6, dated Dec. 3, 2020. [cited by applicant]
“Pinniped post deploy job stuck in Error status and fails with BackoffLimitExceeded,” Broadcom, URL: https://knowledge.broadcom.com/external/article?legacyId=85211, dated Aug. 23, 2023. [cited by applicant]
“ReplicationController,” Kubernetes, URL: https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller/, date found via Google as Mar. 14, 2024. [cited by applicant]
“Performance Best Practices for Kubernetes with VMware Tanzu,” Performance Study, dated Mar. 15, 2021. [cited by applicant]
“Kubernetes Scheduler,” Kubernetes, URL: https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/, date found via Google as Feb. 16, 2024. [cited by applicant]
“vSphere with Tanzu,” Penso Blog Space, URL: https://penso.io/2020/08/19/vsphere-7-0-with-kubernetes/, dated Aug. 19, 2020. [cited by applicant]
“Networking and Security in VMware vSphere with Kubernetes”, White Paper, dated Mar. 2020. [cited by applicant]
Larsen, D., “What is VMware HA and DRS?,” Liquid Web, URL: https://www.liquidweb.com/kb/what-is-vmware-ha-and-drs/, dated Oct. 20, 2021. [cited by applicant]
“vSphere with Tanzu Architecture,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-3E4E6039-BD24-4C40-8575-5AA0EECBBBEC.html, dated Dec. 7, 2022. [cited by applicant]
“Supervisor Architecture,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-with-tanzu-concepts-planning/GUID-3E4E6039-BD24-4C40-8575-5AA0EECBBBEC.html, dated May 5, 2023. [cited by applicant]
Shanks, H., “Kubernetes HA on vSphere,” ThelTHollow, URL: https://theithollow.com/2020/01/27/kubernetes-ha-on-vsphere/, dated Jan. 27, 2020. [cited by applicant]
Chaubal, C., “Best Practices for Enterprise Kubernetes on the VMware SDDC,” VMWare, URL: https://blogs.vmware.com/apps/2019/11/best-practices-for-enterprise-kubernetes-on-the-vmware-sddc.html, dated Nov. 14, 2019. [cited by applicant]
“vSphere with Tanzu Concepts and Planning,” VMWare, date found via Google as Sep. 21, 2023. [cited by applicant]
Tripathi, D., “Worker node in a Supervisor Cluster is showing Not Ready?,” Medium, URL: https://vmwaretanzu.medium.com/worker-node-in-a-supervisor-cluster-is-showing-not-ready-8994e5d0ecc6, dated Sep. 8, 2021. [cited by applicant]
“Datacenter Rookie,” URL: https://datacenterrookie.wordpress.com/2020/01/06/vmware-project-pacific-making-kubernetes-mainstream/, date found via Google as Jan. 6, 2020. [cited by applicant]
Foley, M., “vSphere 7—Introduction to the vSphere Pod Service,” VMWare vSphere Blog, URL: https://blogs.vmware.com/vsphere/2020/04/vsphere-7-vsphere-pod-service.html, dated Apr. 9, 2020. [cited by applicant]
“Enable vSphere with Tanzu on a Cluster Managed by vSphere Lifecycle Manager,” VMWare Docs, URL: https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-6D9ABEEB-2242-46B2-8B04-ED4D9727A69F.html, da… [cited by applicant]
“How Does VMware Run Kubernetes?,” NetApp BlueXP, URL: https://bluexp.netapp.com/blog/cvo-blg-vmware-kubernetes-running-k8s-on-vsphere-and-tanzu, dated Dec. 29, 2020. [cited by applicant]
“VMware vSphere with Kubernetes 101,” VMWare, date found via Google as Mar. 10, 2020. [cited by applicant]
“Managing Host and Cluster Lifecycle—Update 3,” VMWare, dated Apr. 11, 2022. [cited by applicant]
“Pod Lifecycle,” Kubernetes, URL: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/, date found via Google as Apr. 27, 2024. [cited by applicant]
“Tanzu and Project Pacific VMware,” Medium, URL: https://medium.com/@dwaraka.prahladrao/tanzu-and-project-pacific-vmware-45ac1755c008, dated Sep. 9, 2020. [cited by applicant]
“Kubernetes Architecture,” Prolifics, URL: https://prolifics.com/us/resource-center/specialty-guides/ultimate-guide-to-kubernetes/kubernetes-architecture, date found via Google as Aug. 11, 2020. [cited by applicant]
Denneman, F., “Scheduling vSphere Pods,” frankdenneman.nl, dated Mar. 20, 2020. [cited by applicant]
“Distributed Resource Scheduler,” VMWare, URL: https://www.vmware.com/products/vsphere/drs-dpm.html, date found vai Google as Oct. 6, 2009. [cited by applicant]
Mwila, L., “Kubernetes controllers: a complete guide,” KUBERNETES, dated Jul. 22, 2021. [cited by applicant]
Mankowitz, J., “vSphere with Tanzu—Supervisor and TKG, Tanzu Kubernetes Grid”, URL: https://web.archive.org/web/20231202044245/https://rts.se/vsphere-with-tanzu-supervisor-and-tkg-tanzu-kubernetes-grid/, date found via … [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Oct. 15, 2013), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 11, 2014), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 20, 2014), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 7, 2015), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 9, 2015), from http://stevenpoitras.com/the-nutanix-bible/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Sep. 4, 2015), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 12, 2016), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 9, 2016), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 3, 2017), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 8, 2017), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 3, 2018), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jun. 25, 2018), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 8, 2019), from https://nutanixbible.com/. [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jul. 9, 2019), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Feb. 3, 2020), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Aug. 1, 2020), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Jan. 30, 2021), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Poitras, Steven. “The Nutanix Bible” (Sep. 9, 2022), from https://nutanixbible.com/ (Publication date based on indicated capture date by Archive.org; first publication date unknown). [cited by applicant]
Cano, I. et al., “Curator: Self-Managing Storage for Enterprise Clusters”, [cited by applicant]
“Citrix XenDesktop 7.1 on Microsoft Hyper-V Server 2012 R2 on Nutanix Virtual Computing Platform—Solution Design,” Citrix Validated Solutions, Prepared by: Citrix APAC Solutions, dated Jun. 25, 2014. [cited by applicant]