Perform destages of tracks with holes in a storage system by training a machine learning module
A machine learning module receives inputs comprising attributes of a storage controller, where the attributes affect performance parameters for performing stages and destages in the storage controller. In response to an event, the machine learning module generates, via forward propagation, an output value that indicates whether to fill holes in a track of a cache by staging data to the cache prior to destage of the track. A margin of error is calculated based on comparing the generated output value to an expected output value, where the expected output value is generated from an indication of whether it is correct to fill holes in a track of the cache by staging data to the cache prior to destage of the track. An adjustment is made of weights of links that interconnect nodes of the plurality of layers via back propagation to reduce the margin of error.
1. A method for training a machine learning module, the method comprising:
receiving, by the machine learning module, inputs comprising attributes of a storage controller, wherein the attributes affect performance parameters for performing stages and destages in the storage controller;
in response to an event, generating, via forward propagation through a plurality of layers of the machine learning module, an output value that indicates whether to fill holes in a track of a cache by staging data to the cache prior to destage of the track;
calculating a margin of error based on comparing the generated output value to an expected output value, wherein the expected output value is generated from an indication of whether it is correct to fill holes in a track of the cache by staging data to the cache prior to destage of the track; and
adjusting weights of links that interconnect nodes of the plurality of layers via back propagation to reduce the margin of error, to improve a prediction of whether or not to fill holes in tracks prior to destage of the track, wherein the margin of error for training the machine learning module on completion of staging is based at least on computing a difference of an optimum adapter bandwidth and a current adapter bandwidth, and on computing a difference of a maximum stage response time and a current rank response time.
2. The method of claim 1 , wherein a margin of error for training the machine learning module is based on:
a measure of a maximum stage response time beyond which allocating more task control blocks for staging to the cache reduces performance in the storage controller;
a measure of a maximum destage response time beyond which allocating more task control blocks for destaging from the cache reduce a performance in the storage controller; and
a measure of an optimum adapter bandwidth that comprises a bandwidth beyond which an adapter starts thrashing.
3. The method of claim 2 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
4. The method of claim 2 , the method further comprising:
in response to completion of destaging, computing the margin of error based on:
a difference of the optimum adapter bandwidth and a current adapter bandwidth; and
a difference of a current rank response time and the maximum destage response time.
5. The method of claim 4 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
6. The method of claim 1 , wherein the machine learning module is a neural network comprising:
an input layer that receives inputs comprising the attributes of the storage controller, the attributes based on at least a number of holes in a track that is staged or destaged, current values of rank response times, adapter response times, number of stage or destage task control blocks that are allocated, and non-volatile storage usage;
one or more hidden layers; and
an output layer that generates an output value, the method further comprising:
assigning, initial weights in the neural network based on estimates; and
adjusting the weights in the neural network, during the training, in response to completion of destaging from the cache or staging to the cache.
7. A system for training a machine learning module, the system comprising:
a memory; and
a processor coupled to the memory, wherein the processor performs operations, the operations comprising:
receiving, by the machine learning module, inputs comprising attributes of a storage controller, wherein the attributes affect performance parameters for performing stages and destages in the storage controller;
in response to an event, generating, via forward propagation through a plurality of layers of the machine learning module, an output value that indicates whether to fill holes in a track of a cache by staging data to the cache prior to destage of the track;
calculating a margin of error based on comparing the generated output value to an expected output value, wherein the expected output value is generated from an indication of whether it is correct to fill holes in a track of the cache by staging data to the cache prior to destage of the track; and
adjusting weights of links that interconnect nodes of the plurality of layers via back propagation to reduce the margin of error, to improve a prediction of whether or not to fill holes in tracks prior to destage of the track, wherein the margin of error for training the machine learning module on completion of staging is based at least on computing a difference of an optimum adapter bandwidth and a current adapter bandwidth, and on computing a difference of a maximum stage response time and a current rank response time.
8. The system of claim 7 , wherein a margin of error for training the machine learning module is based on:
a measure of a maximum stage response time beyond which allocating more task control blocks for staging to the cache reduces performance in the storage controller;
a measure of a maximum destage response time beyond which allocating more task control blocks for destaging from the cache reduce a performance in the storage controller; and
a measure of an optimum adapter bandwidth that comprises a bandwidth beyond which an adapter starts thrashing.
9. The system of claim 8 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
10. The system of claim 8 , the operations further comprising:
in response to completion of destaging, computing the margin of error based on:
a difference of the optimum adapter bandwidth and a current adapter bandwidth; and
a difference of a current rank response time and the maximum destage response time.
11. The system of claim 10 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
12. The system of claim 7 , wherein the machine learning module is a neural network, the neural network comprising:
an input layer that receives inputs comprising the attributes of the storage controller, the attributes based on at least a number of holes in a track that is staged or destaged, current values of rank response times, adapter response times, number of stage or destage task control blocks that are allocated, and non-volatile storage usage;
one or more hidden layers; and
an output layer that generates an output value, the operations further comprising:
assigning, initial weights in the neural network based on estimates; and
adjusting the weights in the neural network, during the training, in response to completion of destaging from the cache or staging to the cache.
13. A computer program product for training a machine learning module, the computer program product comprising a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code configured to perform operations in a computational device, the operations comprising:
receiving, by the machine learning module, inputs comprising attributes of a storage controller, wherein the attributes affect performance parameters for performing stages and destages in the storage controller;
in response to an event, generating, via forward propagation through a plurality of layers of the machine learning module, an output value that indicates whether to fill holes in a track of a cache by staging data to the cache prior to destage of the track;
calculating a margin of error based on comparing the generated output value to an expected output value, wherein the expected output value is generated from an indication of whether it is correct to fill holes in a track of the cache by staging data to the cache prior to destage of the track; and
adjusting weights of links that interconnect nodes of the plurality of layers via back propagation to reduce the margin of error, to improve a prediction of whether or not to fill holes in tracks prior to destage of the track, wherein the margin of error for training the machine learning module on completion of staging is based at least on computing a difference of an optimum adapter bandwidth and a current adapter bandwidth, and on computing a difference of a maximum stage response time and a current rank response time.
14. The computer program product of claim 13 , wherein a margin of error for training the machine learning module is based on:
a measure of a maximum stage response time beyond which allocating more task control blocks for staging to the cache reduces performance in the storage controller;
a measure of a maximum destage response time beyond which allocating more task control blocks for destaging from the cache reduce a performance in the storage controller; and
a measure of an optimum adapter bandwidth that comprises a bandwidth beyond which an adapter starts thrashing.
15. The computer program product of claim 14 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
16. The computer program product of claim 14 , the operations further comprising:
in response to completion of destaging, computing the margin of error based on:
a difference of the optimum adapter bandwidth and a current adapter bandwidth; and
a difference of a current rank response time and the maximum destage response time.
17. The computer program product of claim 16 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.
18. The computer program product of claim 13 , wherein the machine learning module is a neural network comprising:
an input layer that receives inputs comprising the attributes of the storage controller, the attributes based on at least a number of holes in a track that is staged or destaged, current values of rank response times, adapter response times, number of stage or destage task control blocks that are allocated, and non-volatile storage usage;
one or more hidden layers; and
an output layer that generates an output value, the operations further comprising:
assigning, initial weights in the neural network based on estimates; and
adjusting the weights in the neural network, during the training, in response to completion of destaging from the cache or staging to the cache.
19. A storage controller for training a machine learning module, wherein the storage controller is configured to perform operations, the operation, comprising:
receiving, by the machine learning module, inputs comprising attributes of the storage controller, wherein the attributes affect performance parameters for performing stages and destages in the storage controller;
in response to an event, generating, via forward propagation through a plurality of layers of the machine learning module, an output value that indicates whether to fill holes in a track of a cache by staging data to the cache prior to destage of the track;
calculating a margin of error based on comparing the generated output value to an expected output value, wherein the expected output value is generated from an indication of whether it is correct to fill holes in a track of the cache by staging data to the cache prior to destage of the track; and
adjusting weights of links that interconnect nodes of the plurality of layers via back propagation to reduce the margin of error, to improve a prediction of whether or not to fill holes in tracks prior to destage of the track, wherein the margin of error for training the machine learning module on completion of staging is based at least on computing a difference of an optimum adapter bandwidth and a current adapter bandwidth, and on computing a difference of a maximum stage response time and a current rank response time.
20. The storage controller of claim 19 , wherein a margin of error for training the machine learning module is based on:
a measure of a maximum stage response time beyond which allocating more task control blocks for staging to the cache reduces performance in the storage controller;
a measure of a maximum destage response time beyond which allocating more task control blocks for destaging from the cache reduce a performance in the storage controller; and
a measure of an optimum adapter bandwidth that comprises a bandwidth beyond which an adapter starts thrashing.
21. The storage controller of claim 20 , wherein a positive margin of error is used to train the machine learning module towards filling the holes and a negative margin of error is used to train the machine learning module towards not filling the holes.