Data center management with rack-controllers
Provided is a process, including: receiving, with a rack-controller, via a first network, an application program interface (API) request; based on the API request, selecting, with the rack-controller, one of a plurality of routines to effectuate control via the second network of at least some of the plurality of rack-mounted computing devices; executing, with the rack-controller, the selected routine and, as a result, sending one or more commands via the second network encoded in a second protocol different from the first protocol to effectuate an action indicated by the API request.
1. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors of a rack-controller effectuate operations to control a plurality of rack-mounted computing devices, the operations comprising:
receiving, with the rack-controller, via a first network, at least two application program interface (API) requests, wherein:
the rack-controller is configured to control the plurality of rack-mounted computing devices mounted in a plurality of different rack units of one or more racks;
the rack-controller is configured to control the rack-mounted computing devices via a second network of one or more out-of-band networks distinct from an in-band network with which data is conveyed between rack-mounted computing devices or between rack-mounted computing devices and the internet and different from the first network;
the rack-controller includes a gateway between the first network and the second network; and
the at least two API requests are encoded in a first protocol;
based on a first one of the API requests, selecting, with the rack-controller, a first one of a plurality of routines to effectuate control via the second network of a first group of the plurality of rack-mounted computing devices, the plurality of routines including:
a first routine that reads a sensor via the second network on one of the rack-mounted computing devices;
a second routine that reads a sensor via the second network on the rack but not on one of the rack-mounted computing devices;
a third routine that scans computing devices on the second network and produces an inventory of the scanned computing devices on the second network;
a fourth routine by which a configuration of an extensible firmware interface (EFI) of a given one of the rack-mounted computing device is adjusted; and
a fifth routine by which a rack-mounted computing device is power cycled;
based on a second one of the API requests, selecting, with the rack-controller, a second one of the plurality of routines to effectuate control via the second network of a second group of the plurality of rack-mounted computing devices, wherein the second selected routine is the fifth routine; and
executing, with the rack-controller,
the first selected routine and, as a result, sending one or more commands via the second network encoded in a second protocol different from the first protocol to effectuate an action indicated by the first API request, and
the second selected routine, and, as a result, sending one or more commands via the second network encoded in a third protocol different from the first protocol to power cycle the second group of the plurality of rack-mounted computing devices, receiving via the second network one or more power-on-self-test (POST) codes, and sending via the first network an indication of the one or more POST codes.
2. The medium of claim 1 , wherein:
the rack-controller is an out-of-band computer having an operating system stored in persistent memory;
the rack-controller is coupled to a rack in which the rack-mounted computing devices are disposed;
the number of rack-mounted computing devices controlled by the rack-controller exceeds seven;
the rack-controller is not on the in-band network;
the first network is an out-of-band Ethernet network;
the second network comprises a direct-current (DC) power-line network with which both power and data are conveyed via a DC power bus;
the first protocol is a hypertext transport protocol and the API is a representational state transfer API;
the sensor read by the first routine is a temperature sensor;
the configuration of the EFI specifies a boot device of the given rack-mounted computing device;
at least some of the routines do not use a baseboard management controller of the rack-mounted computing devices;
the first selected routine is the first routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, a response from the temperature sensor in units other than units of temperature;
converting, with the rack-controller, the response into units of temperature; and
sending, with the rack-controller, via the first network, an HTTP response including the converted response in units of temperature.
3. The medium of claim 1 , wherein:
the second network connects a control plane of a rack over which a plurality of servers mounted in the rack are controlled; and
the first network is an out-of-band network over which a data center is managed based on workload and environmental data gathered from a plurality of racks via a plurality of respective instances of the first network.
4. The medium of claim 1 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the second network, to each device on the second network, a command to send an identifier; and
receiving, with the rack-controller, during each of a plurality of durations of time reserved for respective ones of the devices to use a shared physical medium of the second network, an identifier from the respective devices.
5. The medium of claim 1 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the second network, and a modem coupled a mid-plane of a server, a command to a computing device coupled to the server and the modem, the command causing the computing device to execute a scan of other computing devices coupled to a system management bus of server; and
receiving, with the rack-controller, from the modem, via the second network, identifiers of at least some computing devices detected in scan.
6. The medium of claim 1 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, via the first network, a new version of the EFI;
sending, with the rack-controller, via the second network, the new version of the EFI to a given one of the rack-mounted computing devices and instructions to store the new version of the EFI at given boot target of a plurality of candidate boot targets of the given one of the rack-mounted computing devices;
sending, with the rack-controller, via the second network, to the given rack-mounted computing device, an instruction to change a boot target of the EFI to the given boot target; and
sending, with the rack-controller, via the second network, an instruction to reboot the given one of the rack-mounted computing devices.
7. The medium of claim 1 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
changing a boot target of an EFI of at least one of the rack-mounted computing devices without making the change via an Intelligent Platform Management Interface (IPMI) of at least one of the rack-mounted computing devices.
8. The medium of claim 1 , wherein the operations comprise:
gathering, with the rack-controller, via the second network, agentless monitoring metrics from each of the rack-mounted computing devices, the metrics including processor utilization of the respective rack-mounted computing devices.
9. The medium of claim 8 , wherein:
the agentless monitoring metrics are not reported by an operating system or a process running within the operating system of the rack-mounted computing devices, and wherein the metrics include memory utilization and an indication of processor temperature.
10. The medium of claim 1 , wherein:
the second network comprises an out-of-band power-line network by which both electrical power is supplied to the rack-mounted computing devices and by which data is exchanged.
11. The medium of claim 1 , wherein:
the second network comprises an out-of-band Ethernet network.
12. The medium of claim 1 , wherein:
at least two of the routines are not effectuated via a baseboard management controller of the rack-mounted computing devices.
13. The medium of claim 1 , wherein the operations comprise:
causing a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the second network.
14. The medium of claim 13 , wherein the operations comprise:
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the second network, the selected command.
15. The medium of claim 13 , wherein:
reading from the register comprises querying a value of the register of the component, the component identified by a scan of an address space of the system management bus of the one of the rack-mounted computing devices.
16. The medium of claim 1 , wherein:
the routines comprise a routine by which access to a remote console executing on one of the rack-mounted computing devices is provided, wherein the rack-controller operates as a web proxy providing remote console access.
17. The medium of claim 1 , wherein:
the first routine is selected as the first selected routine with means for selecting a routine based on an API request;
the sensor comprises means for sensing; and
the rack-controller comprises means for controlling a rack.
18. The medium of claim 1 , wherein:
the rack controller is executed in an out-of-band computer having a processor and memory storing instructions that when executed effectuate at least some of the operations of claim 1 ; and
the rack controller is configured to manage rack-mounted computing devices via the second network with an out-of-band power-line communication network.
19. The medium of claim 1 , wherein:
the rack controller is configured to be executed in a rack-mounted computing device; and
the second network comprises an out-of-band Ethernet network.
20. The medium of claim 1 , wherein:
the rack controller is configured to be executed in a rack-mounted computing device; and
the rack controller is configured to manage rack-mounted computing devices via the second network with an out-of-band power-line communication network.
21. The medium of claim 1 , wherein:
the rack controller is executed in an out-of-band computer having a processor and memory storing instructions that when executed effectuate at least some of the operations of claim 1 ; and
the second network comprises an out-of-band Ethernet network.
22. A method, comprising:
receiving, with a rack-controller, via a first network, at least two application program interface (API) requests, wherein:
the rack-controller is configured to control a plurality of rack-mounted computing devices mounted in a plurality of different rack units of one or more racks;
the rack-controller is configured to control the rack-mounted computing devices via a second network of one or more out-of-band networks distinct from an in-band network with which data is conveyed between rack-mounted computing devices or between rack-mounted computing devices and the internet and different from the first network;
the rack-controller includes a gateway between the first network and the second network; and
the at least two API requests are encoded in a first protocol;
based on a first one of the API requests, selecting, with the rack-controller, a first one of a plurality of routines to effectuate control via the second network of a first group of the plurality of rack-mounted computing devices, the plurality of routines including:
a first routine that reads a sensor via the second network on one of the rack-mounted computing devices;
a second routine that reads a sensor via the second network on the rack but not on one of the rack-mounted computing devices;
a third routine that scans computing devices on the second network and produces an inventory of the scanned computing devices on the second network;
a fourth routine by which a configuration of an extensible firmware interface (EFI) of a given one of the rack-mounted computing device is adjusted; and
a fifth routine by which a rack-mounted computing device is power cycled;
based on a second one of the API requests, selecting, with the rack-controller, a second one of the plurality of routines to effectuate control via the second network of a second group of the plurality of rack-mounted computing devices, wherein the second selected routine is the fifth routine; and
executing, with the rack-controller,
the first selected routine and, as a result, sending one or more commands via the second network encoded in a second protocol different from the first protocol to effectuate an action indicated by the first API request, and
the second selected routine and, as a result, sending one or more commands via the second network encoded in a third protocol different from the first protocol to power cycle the second group of the plurality of rack-mounted computing devices, receiving via the second network one or more power-on-self-test (POST) codes, and sending via the first network an indication of the one or more POST codes.
23. A data center, comprising:
a data center management computing device communicatively coupled to an out-of-band network; and
a plurality of racks, each rack comprising:
a plurality of rack-mounted computing devices within the plurality of racks and communicatively coupled to an in-band network;
a rack-controller communicatively coupled to the data center management computing device via the out-of-band network;
a powerline network having a rack-specific network address space independent of address spaces of other powerline networks of other racks;
a plurality of powerline modems each associated with a respective one or more of the rack-mounted computing devices; and
a tangible, non-transitory, machine-readable medium of the rack controller storing instructions that when executed by the rack-controller effectuate operations comprising:
receiving, with a rack-controller, via the out-of-band network, at least two requests;
based on a first one of the requests, selecting, with the rack-controller, a first one of a plurality of routines to effectuate control via the second network of a first group of the plurality of rack-mounted computing devices, the plurality of routines including:
first routine that reads a sensor via the second network on one of the rack-mounted computing devices;
a second routine that reads a sensor via the second network on the rack but not on one of the rack-mounted computing devices;
a third routine that scans computing devices on the second network and produces an inventory of the scanned computing devices on the second network;
a fourth routine by which a configuration of an extensible firmware interface (EFI) of a given one of the rack-mounted computing device is adjusted; and
a fifth routine by which a rack-mounted computing device is power cycled; and
based on a second one of the requests, selecting, with the rack-controller, a second one of the plurality of routines to effectuate control via the second network of a second group of the plurality of rack-mounted computing devices, wherein the second selected routine is the fifth routine; and
executing, with the rack-controller,
the first selected routine and, as a result, sending one or more commands via the powerline network to effectuate an action in response to the request, and
the second selected routine, and, as a result, sending one or more commands via the powerline network to power cycle the second group of the plurality of rack-mounted computing devices, receiving via the powerline network one or more power-on-self-test (POST) codes, and sending via the out-of-band network an indication of the one or more POST codes.
24. The medium of claim 1 , wherein the second protocol is the same as the third protocol.
25. The medium of claim 1 , wherein the operations comprise:
reading a component identifier from a register of a component detected by a scan of an address space of the system management bus of one of the rack-mounted computing devices for components of the one of the rack-mounted computing devices;
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the second network, the selected command.
26. The method of claim 22 , wherein the second protocol is the same as the third protocol.
27. The method of claim 22 , wherein:
the rack-controller is an out-of-band computer having a processor, memory, and an operating system stored in persistent memory;
the rack-controller is coupled to a rack in which the rack-mounted computing devices are disposed;
the number of rack-mounted computing devices controlled by the rack-controller exceeds seven;
the rack-controller is not on the in-band network;
the first network is an out-of-band Ethernet network;
the second network is comprises a direct-current (DC) power-line network with which both power and data are conveyed via a DC power bus;
the first protocol is a hypertext transport protocol and the API is a representational state transfer API;
the sensor read by the first routine is a temperature sensor;
the configuration of the EFI specifies a boot device of the given rack-mounted computing device;
at least some of the routines do not use a baseboard management controller of the rack-mounted computing devices;
the first selected routine is the first routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, a response from the temperature sensor in units other than units of temperature;
converting, with the rack-controller, the response into units of temperature; and
sending, with the rack-controller, via the first network, an HTTP response including the converted response in units of temperature.
28. The method of claim 22 , wherein:
the second network connects a control plane of a rack over which a plurality of servers mounted in the rack are controlled; and
the first network is an out-of-band network over which a data center is managed based on workload and environmental data gathered from a plurality of racks via a plurality of respective instances of the first network.
29. The method of claim 22 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the second network, to each device on the second network, a command to send an identifier; and
receiving, with the rack-controller, during each of a plurality of durations of time reserved for respective ones of the devices to use a shared physical medium of the second network, an identifier from the respective devices.
30. The method of claim 22 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the second network, and a modem coupled a mid-plane of a server, a command to a computing device coupled to the server and the modem, the command causing the computing device to execute a scan of other computing devices coupled to a system management bus of server; and
receiving, with the rack-controller, from the modem, via the second network, identifiers of at least some computing devices detected in scan.
31. The method of claim 22 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, via the first network, a new version of the EFI;
sending, with the rack-controller, via the second network, the new version of the EFI to a given one of the rack-mounted computing devices and instructions to store the new version of the EFI at given boot target of a plurality of candidate boot targets of the given one of the rack-mounted computing devices;
sending, with the rack-controller, via the second network, to the given rack-mounted computing device, an instruction to change a boot target of the EFI to the given boot target; and
sending, with the rack-controller, via the second network, an instruction to reboot the given one of the rack-mounted computing devices.
32. The method of claim 22 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
changing a boot target of an EFI of at least one of the rack-mounted computing devices without making the change via an Intelligent Platform Management Interface (IPMI) of at least one of the rack-mounted computing devices.
33. The method of claim 22 , wherein the operations comprise:
gathering, with the rack-controller, via the second network, agentless monitoring metrics from each of the rack-mounted computing devices, the metrics including processor utilization of the respective rack-mounted computing devices, and
the agentless monitoring metrics are not reported by an operating system or a process running within the operating system of the rack-mounted computing devices, and wherein the metrics include memory utilization and an indication of processor temperature.
34. The method of claim 22 , wherein:
the second network comprises at least one of:
an out-of-band power-line network by which both electrical power is supplied to the rack-mounted computing devices and by which data is exchanged; and
an out-of-band Ethernet network.
35. The method of claim 22 , wherein the operations comprise:
causing a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the second network;
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the second network, the selected command.
36. The method of claim 22 , wherein the operations comprise:
causing a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the second network, wherein reading from the register comprises querying a value of the register of the component, the component identified by a scan of an address space of the system management bus of the one of the rack-mounted computing devices.
37. The method of claim 22 , wherein the operations comprise:
reading a component identifier from a register of a component detected by a scan of an address space of a system management bus of one of the rack-mounted computing devices for components of the one of the rack-mounted computing devices;
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the second network, the selected command.
38. The method of claim 22 , wherein:
the rack controller is configured to be executed in a rack-mounted computing device; and
the second network comprises at least one of an out-of-band Ethernet network and an out-of-band power-line communication network.
39. The system of claim 23 , wherein:
the rack-controller is an out-of-band computer having a processor and an operating system stored in persistent memory;
the rack-controller is coupled to the given rack in which the given plurality of rack-mounted computing devices are disposed;
the number of rack-mounted computing devices controlled by the rack-controller exceeds seven;
the rack-controller is not on the in-band network;
the out-of-band network is an Ethernet network;
the at least two requests are encoded in a hypertext transport protocol;
the sensor read by the first routine is a temperature sensor;
the configuration of the EFI specifies a boot device of the given rack-mounted computing device;
at least some of the routines do not use a baseboard management controller of the rack-mounted computing devices;
the first selected routine is the first routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, a response from the temperature sensor in units other than units of temperature;
converting, with the rack-controller, the response into units of temperature; and
sending, with the rack-controller, via the out-of-band network, an HTTP response including the converted response in units of temperature.
40. The system of claim 23 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the powerline network, to each device on the powerline network, a command to send an identifier; and
receiving, with the rack-controller, during each of a plurality of durations of time reserved for respective ones of the devices to use a shared physical medium of the powerline network, an identifier from the respective devices.
41. The system of claim 23 , wherein:
the first selected routine is the third routine; and
executing, with the rack-controller, the first selected routine comprises:
sending, with the rack-controller, via the powerline network, and a modem coupled a mid-plane of a server, a command to a computing device coupled to the server and the modem, the command causing the computing device to execute a scan of other computing devices coupled to a system management bus of server; and
receiving, with the rack-controller, from the modem, via the powerline network, identifiers of at least some computing devices detected in scan.
42. The system of claim 23 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
receiving, with the rack-controller, via the out-of-band network, a new version of the EFI;
sending, with the rack-controller, via the powerline network, the new version of the EFI to a given one of the rack-mounted computing devices and instructions to store the new version of the EFI at given boot target of a plurality of candidate boot targets of the given one of the rack-mounted computing devices;
sending, with the rack-controller, via the powerline network, to the given rack-mounted computing device, an instruction to change a boot target of the EFI to the given boot target; and
sending, with the rack-controller, via the powerline network, an instruction to reboot the given one of the rack-mounted computing devices.
43. The system of claim 23 , wherein:
the first selected routine is the fourth routine; and
executing, with the rack-controller, the first selected routine comprises:
changing a boot target of an EFI of at least one of the rack-mounted computing devices without making the change via an Intelligent Platform Management Interface (IPMI) of at least one of the rack-mounted computing devices.
44. The system of claim 23 , wherein the operations comprise:
causing a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the powerline network;
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the powerline network, the selected command.
45. The system of claim 3 , wherein the operations comprise:
causing a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the powerline network, wherein reading from the register comprises querying a value of the register of the component, the component identified by a scan of an address space of the system management bus of the one of the rack-mounted computing devices.
46. The system of claim 23 , wherein the operations comprise:
reading a component identifier from a register of a component detected by a scan of an address space of a system management bus of one of the rack-mounted computing devices for components of the one of the rack-mounted computing devices;
receiving, with the rack-controller, the component identifier;
selecting a command corresponding to a capability of the component based on the component identifier; and
sending, with the rack-controller, via the powerline network, the selected command.
47. A tangible, non-transitory, machine-readable medium storing instructions that when executed by one or more processors of a rack-controller effectuate operations to control a plurality of rack-mounted computing devices, the operations comprising:
receiving, with the rack-controller, via a first network, an application program interface (API) request, wherein:
the rack-controller is configured to control the plurality of rack-mounted computing devices mounted in a plurality of different rack units of one or more racks;
the rack-controller is configured to control the rack-mounted computing devices via a second network, different from the first network;
the rack-controller includes a gateway between the first network and the second network;
the rack-controller is configured to cause a component identifier of a component of one of the rack-mounted computing devices to be read from a register of the component and conveyed via a system management bus of the one of the rack-mounted computing devices before being sent to the rack-controller via the second network;
the rack-controller is configured to receive the component identifier;
the second network is an out-of-band network distinct from an in-band network with which data is conveyed between rack-mounted computing devices or between rack-mounted computing devices and the internet; and
the API request is encoded in a first protocol;
based on the API request, selecting, with the rack-controller, one of a plurality of routines to effectuate control via the second network of at least some of the plurality of rack-mounted computing devices, the plurality of routines including:
a first routine that reads a sensor via the second network on one of the rack-mounted computing devices;
a second routine that reads a sensor via the second network on the rack but not on one of the rack-mounted computing devices;
a third routine that scans computing devices on the second network and produces an inventory of the scanned computing devices on the second network; and
a fourth routine by which a configuration of an extensible firmware interface (EFI) of a given one of the rack-mounted computing device is adjusted; and
executing, with the rack-controller, the selected routine and, as a result, sending one or more commands via the second network encoded in a second protocol different from the first protocol to effectuate an action indicated by the API request, wherein sending at least one of the commands comprises selecting a command corresponding to a capability of the component based on the component identifier.
48. The medium of claim 47 , wherein:
reading from the register comprises querying a value of the register of the component, the component identified by a scan of an address space of the system management bus of the one of the rack-mounted computing devices.