Apparatus and method for capacity planning for data center server consolidation and workload reassignment
A server migration tool used to construct data center migration scenarios allowing for a user to rapidly manipulate a large number of input parameters required to describe a transformation from one data center configuration to a new data center configuration. The tool then performs the transformation and allows the user to interact with new data center configuration to understand its performance. A novel parameterization, speed independent service demand (SISD), greatly facilitates scaling performance metrics between different hardware platforms.
1. A server migration tool stored in a memory for predictive analysis of a computer network comprising:
a source data center configuration;
a source parameter set;
a graphical user interface adapted to generate a desired data center configuration;
a transform module, in communication with the graphical user interface, programmed to modify the source parameter set into a destination parameter set by reference to the desired data center configuration; and, the destination parameter set providing a basis for submission to a modeling tool for predictive analysis;
wherein the transform module is a program module programmed to carry out the further steps of:
(a) calculating a throughput for a set of source servers and a set of source devices for a set of source workloads in the source data center configuration;
(b) estimating a service demand for the set of source servers, the set of source devices and a source memory in the source data center configuration;
(c) computing a server transformation matrix;
(d) computing a fraction of a workload reassignment moved between the set of source devices and a set of destination devices;
(e) computing an arrival rate for the desired data center configuration;
(f) computing a source speed independent service demand parameter for the set of source servers and the set of source devices in the source data center configuration;
(g) adjusting a device visit count according to the workload reassignment;
(h) computing a weighted average speed independent service demand parameter for a set of destination servers and a set of destination devices in the desired data center configuration;
(i) computing a service demand for the set of destination servers and the set of destination devices in the desired data center configuration;
(j) computing a memory service demand and a consumption of destination servers in the desired data center configuration;
(k) computing a set of destination parameters for a queuing theory solver from the server transformation matrix; and
wherein the step (a) is further comprised of the substep of applying the equation
TPUT ( k,w )= AB ( w )* OV ( k,w )
for all k where k is an index for a set of devices, and, for all w where w is an index for a set of source workloads.
2. The server migration tool of claim 1 wherein the step (b) is further comprised of the substeps of:
(1) calculating a source server service demand according to the equation:
PDC
(
s
,
w
)
=
UMC
(
s
)
*
P
(
s
,
w
)
*
MUMAX
(
s
)
TPUT
(
s
,
w
)
,
for a set of source servers s and a set of workloads w in the source data center configuration; where UMC(s) is the measured CPU utilization of the set of servers s, P(s,w) is the fractional utilization of the set of servers s assigned to the set of workloads w, MUMAX(s) is the service rate of the set of servers s with all CPUs active, TPUT(s,w) is the throughput for the set of source server s and the set of workloads w for the source data center configuration;
(2) calculating a source disk device service demand according to the equation:
PDD
(
k
,
w
)
=
UMD
(
k
)
*
P
(
k
,
w
)
TPUT
(
k
,
w
)
,
for a set of source disk devices k and a set of workloads w in the source data center configuration; where UMD(k) is the measured utilization of a disk k, P(k,w) is the fractional utilization of a disk k assigned to the set of workloads, and TPUT(k,w) is the throughput for a set of servers k and a set of workloads w for the source data center configuration;
(3) calculating a source NIC device service demand according to the equation:
PDN
(
k
,
w
)
=
UMN
(
k
)
*
P
(
k
,
w
)
TPUT
(
k
,
w
)
,
for a set of source NIC devices and a set of workloads w in the source data center configuration; where UMD(k) is the measured utilization of a NIC k, P(k,w) is the fractional utilization of a NIC k assigned to the set of workloads w, and TPUT(k,w) is the throughput for a set of devices k and a set of workloads w for the source data center configuration;
(4) calculating a source memory service demand according to the equations:
MDYN
(
s
,
0
)
=
MP
(
s
)
-
(
MCORE
(
s
)
+
∑
W
MFIXED
(
s
,
w
)
)
,
MDYN
(
s
,
w
)
=
MDYN
(
s
,
0
)
*
P
(
s
,
w
)
,
and
,
PDM
(
s
,
w
)
=
MDYN
(
s
,
w
)
TPUT
(
s
,
w
)
.
where MDYN(s,0) is a total dynamic memory usage of the set of servers s for the set of workloads, MCYN(s,w) is a dynamic memory usage per workload for the set of servers s, TPUT(s,w) is the throughput for the set of servers s and the set of workloads w for the source data center configuration, MP(s) is a total memory usage of the set of servers s, MCORE(s) is a memory usage of the set of servers s when inactive, MFIXED(s,w) is a fixed memory usage of a set of applications running on the set of servers s, and P(s,w) is a fraction of the set of servers s utilization assigned to the set of workloads w .
3. A server migration tool stored in a memory for predictive analysis of a computer network comprising:
a source data center configuration;
a source parameter set;
a graphical user interface adapted to generate a desired data center configuration;
a transform module, in communication with the graphical user interface, programmed to modify the source parameter set into a destination parameter set by reference to the desired data center configuration; and, the destination parameter set providing a basis for submission to a modeling tool for predictive analysis;
wherein the transform module is a program module programmed to carry out the further steps of:
(a) calculating a throughput for a set of source servers and a set of source devices for a set of source workloads in the source data center configuration;
(b) estimating a service demand for the set of source servers, the set of source devices and a source memory in the source data center configuration;
(c) computing a server transformation matrix;
(d) computing a fraction of a workload reassignment moved between the set of source devices and a set of destination devices;
(e) computing an arrival rate for the desired data center configuration;
(f) computing a source speed independent service demand parameter for the set of source servers and the set of source devices in the source data center configuration;
(g) adjusting a device visit count according to the workload reassignment;
(h) computing a weighted average speed independent service demand parameter for a set of destination servers and a set of destination devices in the desired data center configuration;
(i) computing a service demand for the set of destination servers and the set of destination devices in the desired data center configuration;
(j) computing a memory service demand and a consumption of destination servers in the desired data center configuration;
(k) computing a set of destination parameters for a queuing theory solver from the server transformation matrix; and
wherein the step (c) is further comprised of the substeps of:
(1) specifying that a set of source servers s in source data center configuration is to be consolidated onto a set NDS(s) of destination servers defined by the parameter NDS(s) in the desired data center configuration,
(2) computing F(s,d,w) according to the equation:
F
(
s
,
d
,
w
)
=
1
NDS
(
s
)
where
∀
s
∀
w
∑
d
=
1
C
2
F
(
s
,
d
,
w
)
=
1
and where F(s,d,w) is so computed for the set of source servers s and the set of destination servers d and the set of workloads w.
4. The server migration tool of claim 3 wherein substep (2) further comprises:
(i) specifying that for each source server of the set of source servers s and for a set of workloads w in the source data center configuration that a relocation percentage RP(s,w) of the set of workloads w is reassigned to the set of destination servers,
(ii) computing a matrix F(s,d,w) according to:
F
(
s
,
d
,
w
)
=
RP
(
s
,
w
)
100
1
NDS
(
s
)
where
∀
s
∀
w
∑
d
=
1
C
2
F
(
s
,
d
,
w
)
=
1
and where F(s,d,w) is so computed for the set of source servers s and the set of destination servers d and the set of workloads w; and where a parameter C 2 is a number of destination servers.
5. A server migration tool stored in a memory for predictive analysis of a computer network comprising:
a source data center configuration;
a source parameter set;
a graphical user interface adapted to generate a desired data center configuration;
a transform module, in communication with the graphical user interface, programmed to modify the source parameter set into a destination parameter set by reference to the desired data center configuration; and, the destination parameter set providing a basis for submission to a modeling tool for predictive analysis;
wherein the transform module is a program module programmed to carry out the further steps of:
(a) calculating a throughput for a set of source servers and a set of source devices for a set of source workloads in the source data center configuration;
(b) estimating a service demand for the set of source servers, the set of source devices and a source memory in the source data center configuration;
(c) computing a server transformation matrix;
(d) computing a fraction of a workload reassignment moved between the set of source devices and a set of destination devices;
(e) computing an arrival rate for the desired data center configuration;
(f) computing a source speed independent service demand parameter for the set of source servers and the set of source devices in the source data center configuration;
(g) adjusting a device visit count according to the workload reassignment;
(h) computing a weighted average speed independent service demand parameter for a set of destination servers and a set of destination devices in the desired data center configuration;
(i) computing a service demand for the set of destination servers and the set of destination devices in the desired data center configuration;
(j) computing a memory service demand and a consumption of destination servers in the desired data center configuration;
(k) computing a set of destination parameters for a queuing theory solver from the server transformation matrix; and
wherein the step (d) further comprises the substeps of:
(1) calculating a fraction of reassigned workload w moved from a disk i to a disk j according to the equation:
BD
(
i
,
j
,
w
)
=
1
ND
(
DSVR
(
j
)
)
,
where a parameter DSVR(j) contains a server index s in the desired data center configuration to which the disk j is attached and the parameter ND(DSVR(j)) is equal to a total number of disks attached to the set of servers s and with the further constraint that
∀
i
∀
w
∑
j
=
1
D
2
BD
(
i
,
j
,
w
)
=
1
,
where a parameter D 2 is a number of disks in the desired data center configuration; and
(2) calculating a fraction of the workload reassignment moved from a NIC I to a NIC j according to the equation:
BN
(
i
,
j
,
w
)
=
1
NN
(
NSVR
(
j
)
)
,
where a parameter NSVR(j) contains a server index s in the desired data center configuration to which a parameter NIC j is attached and a parameter ND(NSVR(j)) is equal to a total number of NIC s attached to the set of servers s and with the further constraint that
∀
i
∀
w
∑
j
=
1
N
2
BN
(
i
,
j
,
w
)
=
1
,
where N 2 is a number of NICS in the desired data center configuration.
6. The server migration tool of claim 5 wherein the step e) further comprises the substep of calculating a new arrival rate A(w,p) in the desired data center configuration for the set of workloads w at time interval p from the base arrival rate AB(w) for workload w in the source data center configuration
A ( w,p )= AB ( w )· G ( p,w )
where G(p,w) is a specified growth multiplier.
7. The server migration tool of claim 5 wherein the step f) further comprises the substeps of:
(1) calculating a source CPU speed independent service demand for the set of workloads w according to:
SIDC ( t,w )= CS ( t )· PDC ( t,w )
where CS(t) is a CPU rating of source server t and PDC(t, w) is a service demand for source server t;
(2) calculating a disk speed independent service demand for the set of workloads w according to:
SIDD ( k,w )= DS ( k )· PDD ( k,w )
where DS(k) is a source disk speed rating of disk k and PDD(k,w) is a service demand for source disk k; and
(3) calculating a NIC speed independent service demand for the set of workloads w according to:
SIDN ( k,w )= NS ( k )· PDN ( k,w )
where NS(k) is a source NIC speed rating of NIC k and PDN(k,w) is a service demand for source NIC k.
8. The server migration tool of claim 5 wherein the step g) includes the substeps:
(1) calculating a number of server visit counts, RSVU(d,w), for each destination server d in the desired data center configuration for the set of workloads w after a workload reassignment according to:
∀
d
∀
w
RSVU
(
d
,
w
)
=
∑
s
=
1
C
1
PSVU
(
s
,
w
)
·
F
(
s
,
d
,
w
)
where PSVU(s,w) is a number of server visit counts for the set of workloads w and the set of source servers s in the source data center configuration, and where F(s, d, w) is a server transformation matrix describing the fraction of the set of workloads w reassigned from the set of source servers s to the set of destination servers d;
(2) calculating a dimensionalized server visit counts parameter, SVCD(d,w) for each of the set of destination servers d in the desired data center configuration for the set of workloads w after a workload reassignment according to SVCD(d,w)=RSVU(d,w)/SD(d) where SD(d) is the dimension of the set of destination servers d;
(3) calculating a dimensionalized device visit counts parameter, DVCD(k,w), for a destination disk k in the desired data center configuration according to:
DVCD
(
k
,
w
)
=
RSVU
(
DSVR
(
k
)
,
w
)
DD
(
k
)
·
SD
(
DSVR
(
k
)
)
,
where DSVR(k) is a server index to which the disk k is attached and DD(k) is a dimension of destination disk k; and
(4) calculating a dimensionalized device visit counts parameter, DVCD(k,w), for the destination NIC k in the desired data center configuration according to:
DVCD
(
k
,
w
)
=
RSVU
(
DSVR
(
k
)
,
w
)
DD
(
k
)
·
SD
(
DSVR
(
k
)
)
,
where DSVR(k) is a server index to which NIC k is attached and DD(k) is a dimension of destination NIC k.
9. The server migration tool of claim 5 wherein the step h) further comprises the substeps of:
(1) computing a weighted server speed independent service demand SIDC(d,w) for the set of destination servers d and the set of workloads w given the speed independent service demands SIDC(s,w) for the set of source servers s in the source data center configuration according to:
SIDU
(
d
,
w
)
=
∑
s
=
1
C
1
SIDC
(
s
,
w
)
·
PSVU
(
s
,
w
)
·
F
(
s
,
d
,
w
)
∑
s
=
1
C
1
PSVU
(
s
,
w
)
·
F
(
s
,
d
,
w
)
,
where PSVU(s,w) is a number of server visit counts for workload w and source server s, and where F(s, d, w) is a server transformation matrix containing a fraction of workload w reassigned from source server s to destination server d;
(2) computing the weighted disk speed independent service demand SIDD(j,w) for all destination disks j and all workloads w given the speed independent service demands SIDD(s,w) for the source disks in the source data center configuration according to:
SIDD
(
j
,
w
)
=
∑
s
=
1
C
1
∑
i
∈
DISKS
(
s
)
SIDD
(
j
,
w
)
·
PSVU
(
DSVR
(
i
)
,
w
)
·
F
(
s
,
d
,
w
)
·
BD
(
i
,
j
,
w
)
∑
s
=
1
C
1
∑
i
∈
DISKS
(
s
)
PSVU
(
DSVR
(
i
)
,
w
)
·
F
(
s
,
d
,
w
)
·
BD
(
i
,
j
,
w
)
,
where DSVR(i) is the server index to which disk i is attached, DISKS(s) is a list of disk indices for disks attached to source server s, C 1 is the number of source servers, and BD(i,j,w) is the fraction of relocated workload w moved from disk i to disk j;
(3) computing the weighted NIC speed independent service demand SIDN(j, w) for all destination NICs j and all workloads w given the speed independent service demands SIDN(s,w) for the source NICs in the source data center configuration according to:
SIDN
(
j
,
w
)
=
∑
s
=
1
C
1
∑
i
∈
NICS
(
s
)
SIDN
(
j
,
w
)
·
PSVU
(
NSVR
(
i
)
,
w
)
·
F
(
s
,
d
,
w
)
·
BN
(
i
,
j
,
w
)
∑
s
=
1
C
1
∑
i
∈
NICS
(
s
)
PSVU
(
NSVR
(
i
)
,
w
)
·
F
(
s
,
d
,
w
)
·
BN
(
i
,
j
,
w
)
,
where NSVR(i) is the server index to which NIC i is attached, NICS(s) is a list of NIC indices for NICs attached to source server s, C 1 is the number of source servers, and BN(i,j,w) is the fraction of relocated workload w moved from NIC i to NIC j.
10. The server migration tool of claim 5 wherein the step (i) further comprises the substeps of:
(1) summing the destination workload arrival rates A(w) which is the arrival rate for workload w in a given time interval to find a total arrival rate A(0) for all workloads in the desired data center configuration according to
A
(
0
)
=
∑
w
=
1
Z
A
(
w
)
,
where Z is the total number of workloads in the desired data center configuration;
(2) computing the relative throughput Y(d,w) for all workloads w for every destination server d according to Y(d,w)=(A(w)/A(0))*SVCD(d,w) where SVCD(d,w) is the dimensionalized server visit count for destination server d and workload w;
(3) computing the relative throughput Y′(k,w) for all workloads w for every destination disk k according to Y′(d,w)=(A(w)/A(0))*DVCD(d,w) where DVCD(k,w) is the dimensionalized server visit count for destination disk k and workload w;
(4) computing the relative throughput Y″(k,w) for all workloads w for every destination NIC k according to Y″(k,w)=(A(w)/A(0))*DVCD(k,w) where DVCD(k,w) is the dimensionalized server visit count for destination NIC k and workload w;
(5) computing the service demand, PDC(d,w) for all workloads w and for every destination server d according to PDC(d,w)=SIDC(d,w)/CS(d) where SIDC(d) is the weighted average speed independent service demand for destination server d and workload w, and CS(d) is the CPU speed rating of server d;
(6) computing the service demand, PDD(d,w) for all workloads w and for every destination disk k according to PDD(k,w)=SIDD(k,w)/DS(k) where SIDD(k) where SIDD(k) is the weighted average speed independent service demand for destination disk k and workload w, and DS(k) is the disk speed rating of disk k;
(7) computing the service demand, PDN(d,w) for all workloads w and for every destination NIC k according to PDN(k,w)=SIDN(k,w)/DS(k) where SIDN(k) where SIDN(k) is the weighted average speed independent service demand for destination NIC k and workload w, and DS(k) is the NIC speed rating of NIC k.
11. The server migration tool of claim 5 including the further steps of computing the total service demand for servers by:
(1) calculating the total relative throughput, Y(d,0) of destination server d
according
to
Y
(
d
,
0
)
=
∑
w
=
1
Z
Y
(
d
,
w
)
;
and
(2) calculating the total service demand, PDC(d,0) for destination server d according to
PDC
(
d
,
0
)
=
(
∑
w
=
1
Z
Y
(
d
,
w
)
*
PDC
(
d
,
w
)
)
Y
(
d
,
0
)
.
12. A server migration tool for predictive analysis of a computer network comprising:
a source data center configuration;
a source parameter set;
a graphical user interface adapted to generate a desired data center configuration;
a transform module, in communication with the graphical user interface, programmed to modify the source parameter set into a destination parameter set by reference to the desired data center configuration; and,
the destination parameter set providing a basis for submission to a modeling tool for predictive analysis;
wherein the transform module is a program module programmed to carry out the further steps of:
(a) calculating a throughput for a set of source servers and a set of source devices for a set of source workloads in the source data center configuration;
(b) estimating a service demand for the set of source servers, the set of source devices and a source memory in the source data center configuration;
(c) computing a server transformation matrix;
(d) computing a fraction of a workload reassignment moved between the set of source devices and a set of destination devices;
(e) computing an arrival rate for the desired data center configuration;
(f) computing a source speed independent service demand parameter for the set of source servers and the set of source devices in the source data center configuration;
(g) adjusting a device visit count according to the workload reassignment;
(h) computing a weighted average speed independent service demand parameter for a set of destination servers and a set of destination devices in the desired data center configuration;
(i) computing a service demand for the set of destination servers and the set of destination devices in the desired data center configuration;
(j) computing a memory service demand and a consumption of destination servers in the desired data center configuration;
(k) computing a set of destination parameters for a queuing theory solver from the server transformation matrix; and
(l) computing the total service demand for disks by:
(1) calculating the total relative throughput, Y′(d,0) of destination disk k according to
Y
′
(
k
,
0
)
=
∑
w
=
1
Z
Y
′
(
k
,
w
)
;
and
(2) calculating the total service demand, PDD(k,0) for destination disk k according to
PDD
(
k
,
0
)
=
(
∑
w
=
1
Z
Y
′
(
k
,
w
)
*
PDD
(
k
,
w
)
)
Y
′
(
k
,
0
)
.
13. The server migration tool of claim 12 including the further step of computing the total service demand for NICs by:
(1) calculating the total relative throughput, Y″(k,0) of destination NIC k according to,
Y
″
(
k
,
0
)
=
∑
w
=
1
Z
Y
″
(
k
,
w
)
;
and
(2) calculating the total service demand, PDN(k,0) for destination disk k according to
PDN
(
k
,
0
)
=
(
∑
w
=
1
Z
Y
″
(
k
,
w
)
*
PDN
(
k
,
w
)
)
Y
(
k
,
0
)
.
14. The server migration tool of claim 12 including the further step of computing the total service demand for network delay devices by:
(1) computing the service demand for all workloads w and for every network delay device n according to PDL(n,w)=R(n,w)·DVCD(n, w) where R(n,w) is the network response time for network delay device n and workload w; DVCD(n,w) is the dimensionalized device visit count for network delay device n and workload w; and
(2) calculating the total service demand, PDL(n,0) for network delay device n and destination server d according to
PDL
d
(
k
,
0
)
=
(
∑
w
=
1
Z
Y
(
d
,
w
)
*
PDL
(
n
,
w
)
)
Y
(
d
,
0
)
.
15. The server migration tool of claim 12 including the further step of computing the total service demand for network transmission devices by:
(1) computing the network transmission utilization, UMT(n,w) for network transmission device n and workload w according to
UMT ( n,w )= UDR ( n,w )/ TS ( n ),
where UDR(n,w) is the data rate for workload w on network transmission device n and TS(n) is the network transmission speed of device n; and
(2) calculating the total service demand, PDT(n,0) for network transmission device n according to
PDT
(
n
,
w
)
=
UDR
(
n
,
w
)
*
SC
(
n
)
DVCD
(
n
,
w
)
,
where SC(n) is the number of network streams on network transmission device n and DVCD(n,w) is the dimensionalized device visit count for network transmission device n and workload w.
16. The server migration tool of claim 12 wherein the step j) further comprises the substeps:
(1) computing the memory service demand, PDM(d,w) for every destination server d and all workloads w according to:
PDM
(
d
,
w
)
=
∑
s
=
1
C
1
PDM
(
s
,
w
)
·
SFM
(
s
,
d
,
w
)
·
F
(
s
,
d
,
w
)
∑
s
=
1
C
1
F
(
s
,
d
,
w
)
where F(s,d,w) is a server transformation matrix containing the fraction of workload w reassigned from source server s to destination server d and SFM(s,d,w) is the memory usage scaling factor for workload w between source server s and destination server d and C 1 is the total number of source servers s;
(2) computing a workload present function, G(d,w), to indicate if workload w is present on destination server d according to:
G
(
d
,
w
)
≡
1
if
∑
s
=
1
s
=
C
1
F
(
s
,
d
,
w
)
>
0
G
(
d
,
w
)
≡
0
if
∑
s
=
1
s
=
C
1
F
(
s
,
d
,
w
)
=
0
;
(3) computing the dynamic memory, MDYN(d,w), required for workload w on destination server d according to:
MDYN ( d,w )= TPUT ( d,w )* PDM ( d,w ),
where TPUT(d,w) is the throughput of workload w on server d;
(4) computing the total projected memory consumption, MP(d), on destination server d according to
MP
(
d
)
=
MCORE
(
d
)
+
∑
w
=
1
Z
(
MFIXED
(
d
,
w
)
*
G
(
d
,
w
)
+
TPUT
(
d
,
w
)
*
PDM
(
d
,
w
)
)
;
(5) calculating UMM(d)=MP(d)/MC(d), the total memory utilization of destination server d; and
(6) calculating the total dynamic memory utilization of server d according to:
DMWU
(
d
,
w
)
=
TPUT
(
d
,
w
)
*
PDM
(
d
,
w
)
MC
(
d
)
.