Portable workload performance prediction for the cloud
View Patent ↗Systems and methods are disclosed to perform performance prediction for cloud-based databases by building on a computer a cloud database performance model using a set of training workloads; and using a learned model on the computer to predict database performance in the cloud for a new workload, wherein for each reference workload r and hardware configuration h, system throughput t r,h , average throughput of α r and standard deviation σ r , comprising normalizing each throughput as: t r , h _ = t r , h - a r σ r .
1. A method to perform performance prediction for cloud-based databases, comprising:
building on a computer a cloud database performance model using a set of training workloads; and
using a learned model on the computer to predict database performance in the cloud for a new workload,
wherein for each reference workload r and hardware configuration h, system throughput t r,h , average throughput of α r and standard deviation σ r , comprising normalizing each throughput as:
t
r
,
h
_
=
t
r
,
h
-
a
r
σ
r
.
2. The method of claim 1 , comprising improving the performance model using data from a newly deployed workload by incorporating new workload and true performance data in a cloud server.
3. The method of claim 1 , comprising deploying each training workload on a test server only once.
4. The method of claim 1 , comprising applying one or more resource spoilers to control available resources for deployed workloads at a test server.
5. The method of claim 1 , comprising processing metrics including query latency and query throughput.
6. The method of claim 1 , comprising deploying the workload into a cloud server to collect actual performance.
7. The method of claim 1 , comprising building a model using machine learning and collaborative filtering that maps a test server performance on a workload to corresponding performance in the cloud.
8. The method of claim 1 , comprising iteratively executing the workload under a set of configurations with different resource constraints.
9. The method of claim 1 , comprising applying a learning method that iteratively chooses the next configuration to use to reveal information based on prior learning.
10. The method of claim 1 , comprising applying a criteria to stop training based on a prediction confidence threshold or a running budget.
11. The method of claim 1 , comprising estimating a similarity between i and j as:
w
i
,
j
=
1
S
i
,
j
∑
h
∈
S
i
,
j
t
i
,
h
t
j
,
h
_
where weight w i,j approximates a Pearson similarity between a normalized performance of workload i and workload j.
12. The method of claim 1 , comprising forecasting a workload's QpM on a new hardware platform, h, by taking a similarity-based weighted average of normalized throughputs:
t
i
,
h
_
=
∑
j
|
j
∈
S
i
,
j
≠
ϕ
,
h
∈
H
j
w
i
,
j
t
j
,
h
∑
j
|
j
∈
S
i
,
j
≠
ϕ
,
h
∈
H
j
w
i
,
j
.
13. A system to perform performance prediction for cloud-based databases, comprising:
a test computer to locally collect database performance data;
one or more cloud servers to run cloud databases;
computer code for building on a computer a cloud database performance model using one or more training workloads; and
computer code for using a learned model on the computer to predict database performance in the cloud for a new workload,
wherein for each reference workload r and hardware configuration h, system throughput r,h , average throughput of α r and standard deviation σ r , comprising normalizing each throughput as:
t
r
,
h
_
=
t
r
,
h
-
a
r
σ
r
.
14. The system of claim 13 , comprising code for improving the performance model using data from a newly deployed workload by incorporating new workload and true performance data in a cloud server.
15. The system of claim 13 , comprising code for deploying each training workload on a test server only once.
16. The system of claim 13 , comprising code for applying one or more resource spoilers to control available resources for deployed workloads at a test server.
17. The system of claim 13 , comprising code for processing metrics including query latency and query throughput.
18. The system of claim 13 , comprising code for estimating a similarity between i and j as:
w
i
,
j
=
1
S
i
,
j
∑
h
∈
S
i
,
j
t
i
,
h
t
j
,
h
_
where weight w i,j approximates a Pearson similarity between a normalized performance of workload i and workload j.
19. The system of claim 13 , comprising code for forecasting a workload's QpM on a new hardware platform, h, by taking a similarity-based weighted average of normalized throughputs:
t
i
,
h
_
=
∑
j
|
j
∈
S
i
,
j
≠
ϕ
,
h
∈
H
j
w
i
,
j
t
j
,
h
∑
j
|
j
∈
S
i
,
j
≠
ϕ
,
h
∈
H
j
w
i
,
j
.