IP Library › Granted Patent US 11,461,120
Granted Patent B2
US 11,461,120 · App. 16/423,210 · Granted Oct 4, 2022

Methods and apparatus for rack nesting in virtualized server systems

Inventors: Shubham Verma (Bangalore, IN); Ravi Kumar Reddy Kottapalli (Bangalore, IN); Samdeep Nayak (Bangalore, IN); Kannan Balasubramanian (Bangalore, IN); Suket Gakhar (Palo Alto, CA)
Assignee: VMWARE, INC.
G06F9/45558G06F9/3877G06F9/5044G06F2009/4557G06F2009/45566
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,461,120
App. No.
16/423,210
Granted
Oct 4, 2022
Kind
B2
Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed for rack nesting in virtualized server systems. An example apparatus includes a resource discoverer to identify resources to be allocated to the nested rack based on a policy indicative of one or more physical racks from which to identify the resources, and determine candidate resources from the resources to be allocated to the nested rack based on a capacity parameter indicative of a quantity of the resources available to be allocated to the nested rack, the candidate resources to have first hypervisors, and a nested rack controller to generate the nested rack by deploying second hypervisors on the first hypervisors, the second hypervisors to facilitate communication between the candidate resources and one or more virtual machines on the second hypervisors, the nested rack to execute one or more computing tasks based on the communication.

Claims (82)

1. A system to generate a first nested virtual server rack in a virtual server environment, the system comprising:

a first physical server rack of a plurality of physical server racks of the virtual server environment, the first physical server rack to instantiate nested rack manager circuitry, the nested rack manager circuitry to instantiate resource discoverer circuitry and nested rack controller circuitry;

a second physical server rack of the plurality of physical server racks, the second physical server rack to instantiate a non-nested virtual server rack, the second physical server rack to include a plurality of server host nodes, the plurality of server host nodes to include a first server host node with a first hypervisor on the non-nested virtual server rack and a second server host node with a second hypervisor on the non-nested virtual server rack;

the resource discoverer circuitry to:

identify first physical hardware resources from the second physical server rack for generation of the first nested virtual server rack, the first physical hardware resources to be identified:

based on a nested rack allocation parameter indicative of whether one or more of the plurality of server host nodes are allocated for the first nested virtual server rack; and

after a determination that the one or more of the plurality of server host nodes are not allocated to a second nested virtual server rack in the virtual server environment, the determination based on the nested rack allocation parameter;

determine candidate physical hardware resources from the first physical hardware resources based on a capacity parameter that indicates that the first server host node and the second server host node are available to generate the first nested virtual server rack;

the nested rack controller circuitry to

generate the first nested virtual server rack based on deployment of (i) a first nested hypervisor on the first hypervisor and a second nested hypervisor on the second hypervisor and (ii) a first virtual machine on the first nested hypervisor and a second virtual machine on the second nested hypervisor; and

migrate one or more third virtual machines from third hypervisors of second physical hardware resources of a third physical server rack to at least one of the first virtual machine or the second virtual machine after an upgrade of ones of the second physical hardware resources, wherein at least one of the first nested hypervisor or the second nested hypervisor to facilitate communication between the candidate physical hardware resources and the at least one of the first virtual machine or the second virtual machine, the first nested virtual server rack to execute one or more computing tasks based on the communication.

2. The system of claim 1 , wherein the one or more computing tasks are one or more first computing tasks, the nested rack manager circuitry to instantiate resource data aggregator circuitry, and the resource data aggregator circuitry to:

determine a utilization parameter indicative of a quantity of the first physical hardware resources being utilized to execute a second computing task; and

determine the capacity parameter based on the utilization parameter.

3. The system of claim 1 , wherein a first one of the candidate physical hardware resources is in communication with the first hypervisor, the first one of the candidate physical hardware resources is not to support the first nested hypervisor, and the second physical server rack to instantiate rack interposer deployer circuitry to deploy a rack interposer appliance on the first hypervisor, the rack interposer appliance to:

obtain a message from the first virtual machine directed to the first nested hypervisor; and

transmit the message to the first hypervisor instead of the first nested hypervisor.

4. The system of claim 3 , wherein the first one of the candidate physical hardware resources is a graphics processing unit (GPU) or a universal serial bus (USB) device.

5. The system of claim 1 , wherein the nested rack manager circuitry is to instantiate resource deallocator circuitry, and wherein:

the resource discoverer circuitry is to identify the one or more third virtual machines on the second physical hardware resources to migrate to the at least one of the first virtual machine or the second virtual machine of the candidate physical hardware resources;

the nested rack controller circuitry is to:

upgrade the second physical hardware resources from a first version to a second version; and

migrate the at least one of the first virtual machine or the second virtual machine back to the second physical hardware resources after the upgrade of the second physical hardware resources; and

the resource deallocator circuitry is to decommission the first nested virtual server rack after the migration of the at least one of the first virtual machine or the second virtual machine back to the second physical hardware resources.

6. The system of claim 5 , wherein the resource deallocator circuitry is to deallocate the candidate physical hardware resources from the first nested virtual server rack and return the candidate physical hardware resources to a pool of available resources.

7. The system of claim 1 , wherein the first nested virtual server rack is to:

instruct, with the first nested hypervisor, the first hypervisor to execute the one or more computing tasks; and

instruct, with the first hypervisor, one or more of the candidate physical hardware resources to execute the one or more computing tasks.

8. The system of claim 1 , wherein the one or more third virtual machines are instantiated on the first physical server rack.

9. A non-transitory computer readable storage medium comprising instructions which, when executed, cause a machine to at least:

instantiate a nested rack manager on a first physical server rack of a plurality of physical server racks of a virtual server environment, the nested rack manager to instantiate a resource discoverer and a nested rack controller;

instantiate a non-nested virtual server rack on a second physical server rack of the plurality of physical server racks, the second physical server rack to include a plurality of server host nodes, the plurality of server host nodes to include a first server host node with a first hypervisor on the non-nested virtual server rack and a second server host node with a second hypervisor on the non-nested virtual server rack;

cause the resource discoverer to:

identify first physical hardware resources from the second physical server rack as available to instantiate a first nested virtual server rack, the first physical hardware resources to be identified:

based on a nested rack allocation parameter indicative of whether one or more of the plurality of server host nodes are allocated for the first nested virtual server rack; and

after a determination that the one or more of the plurality of server host nodes are not allocated to a second nested virtual server rack in the virtual server environment based on the nested rack allocation parameter; and

determine candidate physical hardware resources from the first physical hardware resources based on a capacity parameter that indicates that the first server host node and the second server host node are available to instantiate the first nested virtual server rack; and

cause the nested rack controller to:

create the first nested virtual server rack based on deployment of (i) a first nested hypervisor on the first hypervisor and a second nested hypervisor on the second hypervisor and (ii) a first virtual machine on the first nested hypervisor and a second virtual machine on the second nested hypervisor; and

migrate one or more applications installed on one or more third virtual machines from third hypervisors of second physical hardware resources of a third physical server rack to at least one of the first virtual machine or the second virtual machine after an upgrade of ones of the second physical hardware resources, wherein at least one of the first nested hypervisor or the second nested hypervisor to facilitate communication between the candidate physical hardware resources and the at least one of the first virtual machine or the second virtual machine, the first nested virtual server rack to execute one or more workloads based on the communication.

10. The non-transitory computer readable storage medium of claim 9 , wherein the one or more workloads are one or more first workloads, and the instructions, when executed, cause the machine to:

determine a utilization parameter indicative of a quantity of the first physical hardware resources being utilized to execute a second workload; and

determine the capacity parameter based on the utilization parameter.

11. The non-transitory computer readable storage medium of claim 9 , wherein a first one of the candidate physical hardware resources is in communication with the first hypervisor, the first one of the candidate physical hardware resources is not to support the first nested hypervisor, and the instructions, when executed, cause the machine to deploy a rack interposer appliance on the first hypervisor, the rack interposer appliance to:

obtain a message from the first virtual machine directed to the first nested hypervisor; and

transmit the message to the first hypervisor instead of the first nested hypervisor.

12. The non-transitory computer readable storage medium of claim 11 , wherein the first one of the candidate physical hardware resources is a graphics processing unit (GPU) or a universal serial bus (USB) device.

13. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, cause the machine to:

identify the one or more third virtual machines on the second physical hardware resources;

upgrade the second physical hardware resources from a first version to a second version;

migrate hosting of the one or more applications back to the second physical hardware resources after the upgrade of the second physical hardware resources; and

decommission the first nested virtual server rack after the migration of the hosting of the one or more third virtual machines back to the second physical hardware resources.

14. The non-transitory computer readable storage medium of claim 13 , wherein the instructions, when executed, cause the machine to deallocate the candidate physical hardware resources from the first nested virtual server rack and return the candidate physical hardware resources to a pool of available resources.

15. The non-transitory computer readable storage medium of claim 9 , wherein the instructions, when executed, cause the machine to cause the first nested virtual server rack to:

instruct, with the first nested hypervisor, the first hypervisor to execute the one or more workloads; and

instruct, with the first hypervisor, one or more of the candidate physical hardware resources to execute the one or more workloads.

16. A method to generate a first nested virtual server rack in a virtual server environment, the method comprising:

establishing a nested rack manager on a first physical server rack of a plurality of physical server racks of the virtual server environment, the nested rack manager including a resource discoverer and a nested rack controller;

establishing an unnested virtual server rack on a second physical server rack of the plurality of physical server racks, the second physical server rack to include a plurality of server host nodes, the plurality of server host nodes to include a first server host node with a first hypervisor on the unnested virtual server rack and a second server host node with a second hypervisor on the unnested virtual server rack;

causing the resource discoverer to:

identify first physical hardware resources from the second physical server racks for generation of the first nested virtual server rack, the identifying of the first physical hardware resources:

based on a nested rack allocation parameter indicative of whether one or more of the plurality of server host nodes are allocated for the first nested virtual server rack; and

after a determination that the one or more of the plurality of server host nodes are not allocated to a second nested virtual server rack in the virtual server environment based on the nested rack allocation parameter; and

determine candidate physical hardware resources from the first physical hardware resources based on a capacity parameter indicative of the first server host node and the second server host node being available for generation of the first nested virtual server rack; and

causing the nested rack controller to:

generate the first nested virtual server rack based on deployment of (i) a first nested hypervisor on the first hypervisor and a second nested hypervisor on the second hypervisor and (ii) a first virtual machine on the first nested hypervisor and a second virtual machine on the second nested hypervisor; and

migrate data on one or more third virtual machines from third hypervisors of second physical hardware resources of a third physical server rack to at least one of the first virtual machine or the second virtual machine after an upgrade of ones of the second physical hardware resources, wherein at least one of the first nested hypervisor or the second nested hypervisor to facilitate communication between the candidate physical hardware resources and the at least one of the first virtual machine or the third virtual machine, the first nested virtual server rack to execute a workload based on the communication.

17. The method of claim 16 , wherein the workload is a first workload, and the method further including:

determining a utilization parameter indicative of a quantity of the first physical hardware resources being utilized to execute a second workload; and

determining the capacity parameter based on the utilization parameter.

18. The method of claim 16 , wherein a first one of the candidate physical hardware resources is in communication with the first hypervisor, the first one of the candidate physical hardware resources unable to support the first nested hypervisor, and the method further including deploying a rack interposer appliance on the first hypervisor, the rack interposer appliance to:

obtain a message from the first virtual machine directed to the first nested hypervisor; and

transmit the message to the first hypervisor instead of the first nested hypervisor.

19. The method of claim 16 , further including:

identifying the data on the one or more third virtual machines on the second physical hardware resources to migrate to the at least one of the first virtual machine or the second virtual machine of the candidate physical hardware resources;

upgrading the second physical hardware resources from a first version to a second version;

migrating the data on the at least one of the first virtual machine or the second virtual machine back to the second physical hardware resources in after upgrading the second physical hardware resources; and

decommissioning the first nested virtual server rack after migrating the data back to the second physical hardware resources.

20. The method of claim 19 , wherein the decommissioning of the first nested virtual server rack includes deallocating the candidate physical hardware resources from the first nested virtual server rack and returning the candidate physical hardware resources to a pool of available resources.

21. The method of claim 16 , wherein the first nested virtual server rack is to execute the one or more computing tasks by:

instructing, with the first nested hypervisor, the first hypervisor to execute the workload; and

instructing, with the first hypervisor, one or more of the candidate physical hardware resources to execute the workload.

Assignments (2)
CHANGE OF NAME Recorded Apr 15, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 067102/0395 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 28, 2019
From: VERMA, SHUBHAM; KOTTAPALLI, RAVI KUMAR REDDY; NAYAK, SAMDEEP; BALASUBRAMANIAN, KANNAN; GAKHAR, SUKET
To: VMWARE, INC.
Reel/Frame 049287/0865 →
Priority Claims (1)
IN 201941003281 · Jan 26, 2019 · national
Continuity (1)
Related Publication 20200241910A1 · Jul 30, 2020