Estimation of power consumption for a job based on adjusted calculation of similarities between jobs
An apparatus calculates, based on attribute information of a new job and jobs that have been executed, a first similarity level of the attribute information between the new job and the jobs by using a calculation expression, identifies a job whose attribute information is most similar to that of the new job as a first candidate job, and estimates power consumption to be consumed by the new job at power consumption of the first candidate job. The apparatus calculates, for at least one of the jobs, a second similarity level of power consumption between the at least one of the jobs and the new job, identifies a job whose power consumption is most similar to that of the new job as a second candidate job, and adjusts the calculation expression to increase the first similarity level to be calculated between the new job and the second candidate job.
1. A non-transitory, computer-readable recording medium having stored therein a program for causing a computer to execute a process comprising:
calculating, based on words within a plurality of documents respectively corresponding to a plurality of jobs, an attribute of a first job that is newly submitted and an attribute of each of a plurality of second jobs that have been executed, a first similarity level indicating a level of similarity of the words within a first document corresponding to the first job and each of a plurality of second documents respectively corresponding to the plurality of second jobs by using a first calculation expression;
identifying, based on the first similarity level calculated for each of the plurality of second jobs, one of the plurality of second jobs whose corresponding second document is most similar to the first document as a first candidate job;
estimating, based on second power waveforms respectively indicating power consumed by executing the plurality of second jobs with a high-performance computing system, first power consumption to be consumed by executing the first job at power consumption indicated by a second power waveform of the first candidate job;
obtaining, after the first job is executed, a first job power waveform indicating power consumption that has been consumed by executing the first job;
calculating, based on the first job power waveform and the second power waveform, for at least one of the plurality of second jobs, a second similarity level indicating a level of similarity of power consumption between the first job and the at least one of the plurality of second jobs;
identifying, based on the second similarity level calculated for the at least one of the plurality of second jobs, one of the at least one of the plurality of second jobs whose second power waveform is most similar to the first job power waveform as a second candidate job; and
adjusting the first calculation expression to increase the first similarity level to be calculated between the first job and the second candidate job, wherein:
the calculating the first similarity level includes calculating the first similarity level by using weights of individual topics determined based on the words within the plurality of documents, the first calculation expression using the weights to cause a similarity level between two documents including a particular topic to increase as a value of weight of the particular topic increases;
the adjusting the first calculation expression includes increasing the value of weight of a topic included in the document of the second candidate job; and
controlling power consumption of the high-performance computing system based on the adjusted first calculation expression.
2. The non-transitory, computer-readable recording medium of claim 1 , wherein
the calculating the first similarity level includes:
calculating a topic distribution indicating rates of occurrence of topics included in the first job and each of the plurality of second jobs, and
determining a similarity level of topic distribution between the first job and each of the plurality of second jobs as the first similarity level between the first job and each of the plurality of second jobs.
3. The non-transitory, computer-readable recording medium of claim 1 , wherein
the calculating the first similarity level includes:
determining, in accordance with the weight of a topic included in the first job, whether to calculate the first similarity level by employing the first calculation expression using the weight of the topic or by employing a second calculation expression not using the weight of the topic, and
calculating the first similarity level by employing the first calculation expression or the second calculation expression depending on a result of the determining.
4. The non-transitory, computer-readable recording medium of claim 1 , wherein
the calculating the second similarity level is performed for a predetermined number of second jobs which are selected, from among the plurality of second jobs, in order starting from a second job whose first similarity level is highest among the plurality of second jobs.
5. The non-transitory, computer-readable recording medium of claim 1 , wherein
the calculating the second similarity level is performed for each of second jobs, among the plurality of second jobs, whose first similarity level is equal to or greater than a threshold.
6. The non-transitory, computer-readable recording medium of claim 1 , wherein the first similarity level is calculated based on a latent dirichlet allocation (LDA) estimation model.
7. The non-transitory, computer-readable recording medium of claim 1 , wherein the first similarity level and the second similarity level are calculated based on different attributes.
8. The non-transitory, computer-readable recording medium of claim 1 , wherein the first similarity value is calculated with at least one of a cosine similarity and vector space method.
9. The non-transitory, computer-readable recording medium according to claim 1 , wherein the controlling power consumption controls the power consumption of the high-performance computing system to be below a maximum power demand.
10. The non-transitory, computer-readable recording medium of claim 1 , wherein the first calculation expression includes a the probability with respect to the plurality of documents and words within the plurality of documents
P
(
z
d
,
n
=
k
❘
w
d
,
n
=
v
,
w
\
d
,
n
,
z
\
d
,
n
,
α
,
β
)
∝
N
k
,
v
\
d
,
n
+
β
N
k
\
d
,
n
+
β
V
(
N
d
,
k
\
d
,
n
+
α
)
(
2
)
P is the probability, N is the total number of words in a collection of documents, V is the total number of kinds of word, d is an index of a particular document, n is an index of a particular word, v is an index of a particular kind of word, w is a particular word, z is a particular topic, A backslash indicates the difference with respect to a collection, β is a parameter of word distribution, and expression (2) is a sampling expression expressing a topic z d,n of a word w d,n in a document d.
11. A method performed by a computer, the method comprising:
calculating, based on words within a plurality of documents respectively corresponding to a plurality of jobs, an attribute of a first job that is newly submitted and an attribute of each of a plurality of second jobs that have been executed, a first similarity level indicating a level of similarity of the words within a first document corresponding to the first job and each of a plurality of second documents respectively corresponding to the plurality of second jobs by using a first calculation expression;
identifying, based on the first similarity level calculated for each of the plurality of second jobs, one of the plurality of second jobs whose corresponding second document is most similar to the first document as a first candidate job;
estimating, based on second power waveforms respectively indicating power consumed by executing the plurality of second jobs with a high-performance computing system, first power consumption to be consumed by executing the first job at power consumption indicated by a second power waveform of the first candidate job;
obtaining, after the first job is executed, a first job power waveform indicating power consumption that has been consumed by executing the first job;
calculating, based on the first job power waveform and the second power waveform, for at least one of the plurality of second jobs, a second similarity level indicating a level of similarity of power consumption between the first job and the at least one of the plurality of second jobs;
identifying, based on the second similarity level calculated for the at least one of the plurality of second jobs, one of the at least one of the plurality of second jobs whose second power waveform is most similar to the first job power waveform as a second candidate job; and
adjusting the first calculation expression to increase the first similarity level to be calculated between the first job and the second candidate job, wherein:
the calculating the first similarity level includes calculating the first similarity level by using weights of individual topics determined based on the words within the plurality of documents, the first calculation expression using the weights to cause a similarity level between two documents including a particular topic to increase as a value of weight of the particular topic increases;
the adjusting the first calculation expression includes increasing the value of weight of a topic included in the document of the second candidate job; and
controlling power consumption of the high-performance computing system based on the adjusted first calculation expression.
12. An apparatus comprising:
a memory; and
a processor coupled to the memory and configured to:
calculate, based on words within a plurality of documents respectively corresponding to a plurality of jobs, an attribute of a first job that is newly submitted and an attribute of each of a plurality of second jobs that have been executed, a first similarity level indicating a level of similarity of the words within a first document corresponding to the first job and each of a plurality of second documents respectively corresponding to the plurality of second jobs by using a first calculation expression;
identify, based on the first similarity level calculated for each of the plurality of second jobs, one of the plurality of second jobs whose corresponding second document is most similar to the first document as a first candidate job;
estimate, based on second power waveforms respectively indicating power consumed by executing the plurality of second jobs with a high-performance computing system, first power consumption to be consumed by executing the first job at power consumption indicated by a second power waveform of the first candidate job;
obtain, after the first job is executed, a first job power waveform indicating power consumption that has been consumed by executing the first job;
calculate, based on the first job power waveform and the second power waveform, for at least one of the plurality of second jobs, a second similarity level indicating a level of similarity of power consumption between the first job and the at least one of the plurality of second jobs;
identify, based on the second similarity level calculated for the at least one of the plurality of second jobs, one of the at least one of the plurality of second jobs whose second power waveform is most similar to the first job power waveform as a second candidate job; and
adjust the first calculation expression to increase the first similarity level to be calculated between the first job and the second candidate job, wherein:
the processor further calculates the first similarity level includes calculating the first similarity level by using weights of individual topics determined based on the words within the plurality of documents, the first calculation expression using the weights to cause a similarity level between two documents of the including a particular topic to increase as a value of weight of the particular topic increases;
adjusts the first calculation expression includes increasing the value of weight of a topic included in the document of the second candidate job; and
controls power consumption of the high-performance computing system based on the adjusted first calculation expression.
13. The apparatus of claim 12 , wherein
the processor is further configured to:
calculate a topic distribution indicating rates of occurrence of topics included in the first job and each of the plurality of second jobs, and
determine a similarity level of topic distribution between the first job and each of the plurality of second jobs as the first similarity level between the first job and each of the plurality of second jobs.
14. The apparatus of claim 12 , wherein
the processor calculates the first similarity level by determining, in accordance with the weight of a topic included in the first job, whether to calculate the first similarity level by employing the first calculation expression using the weight of the topic or by employing a second calculation expression not using the weight of the topic, and employing the first calculation expression or the second calculation expression depending on a result of the determining.
15. The apparatus of claim 12 , wherein
the processor calculates the second similarity level for a predetermined number of second jobs which are selected, from among the plurality of second jobs, in order starting from a second job whose first similarity level is highest among the plurality of second jobs.
16. The apparatus of claim 12 , wherein the first similarity level is calculated based on a latent dirichlet allocation (LDA) estimation model.
17. The apparatus of claim 12 , wherein the first similarity level and the second similarity level are calculated based on different attributes.