Storage controller and storage device
A storage controller, including a processor configured to perform a plurality of tasks; and a scheduling module configured to schedule the plurality of tasks through reinforcement learning, and provide a scheduling result to the processor, wherein the scheduling module includes: a resource analysis module configured to analyze a usage history and a usage status of a resource; an access pattern analysis module configured to analyze an access pattern; a queue analysis module configured to analyze information included in a queue; and a performance analysis module configured to analyze a task execution result, wherein the scheduling module is further configured to perform the reinforcement learning using state information and reward information, and wherein the state information and the reward information are determined based on the performing of the plurality of tasks.
1 . A storage controller comprising:
a processor configured to perform a plurality of tasks based on a command received from a host; and
a scheduling module configured to schedule the plurality of tasks through reinforcement learning, and provide a scheduling result to the processor,
wherein the scheduling module includes:
a resource analysis module configured to analyze a usage history and a usage status of a resource used for the plurality of tasks;
an access pattern analysis module configured to analyze an access pattern corresponding to the plurality of tasks;
a queue analysis module configured to analyze information included in a queue according to the command provided from the host; and
a performance analysis module configured to analyze a task execution result of the processor,
wherein the scheduling module is further configured to perform the reinforcement learning using state information and reward information,
wherein the state information and the reward information are generated by the storage controller based on the performing of the plurality of tasks by the storage controller, and
wherein the scheduling module further includes an encoding module configured to receive the state information and the reward information, and to convert the state information and the reward information into converted information in a format suitable for the reinforcement learning configured to be performed in a storage device,
wherein the storage controller is configured to operate in a first mode, a second mode, a third mode, and a fourth mode,
wherein, in the first mode, the storage controller performs the plurality of tasks according to the scheduling result and generates the scheduling result by performing the reinforcement learning based on the command,
wherein, in the second mode, the storage controller performs the plurality of tasks according to a predetermined rule without following the scheduling result,
wherein, in the third mode, the storage controller performs the reinforcement learning without performing the plurality of tasks according to the scheduling result, and
wherein, in the fourth mode, the storage controller performs the reinforcement learning according to a predetermined period.
2 . The storage controller of claim 1 , wherein the scheduling module is further configured to:
use the usage history and the usage status, the access pattern, and the information included in the queue, as the state information,
perform the reinforcement learning using the task execution result as the reward information, and
generate the scheduling result by determining a priority of the plurality of tasks based on the reinforcement learning.
3 . The storage controller of claim 1 , wherein the scheduling module is further configured to perform the reinforcement learning according to a predetermined period.
4 . The storage controller of claim 1 , wherein the scheduling module is further configured to perform the reinforcement learning and to provide the scheduling result to the processor based on a request of the processor.
5 . The storage controller of claim 1 , further comprising a dedicated memory configured to store the scheduling result generated by the scheduling module, the state information, and the reward information.
6 . The storage controller of claim 1 , wherein the scheduling module further includes a machine learning module configured to perform the reinforcement learning and to generate the scheduling result.
7 . A storage device comprising:
a non-volatile memory; and
a storage controller configured to perform a plurality of tasks corresponding to the non-volatile memory based on a command received from an outside of the storage device,
wherein the storage controller includes:
a machine learning module configured to generate a scheduling result, in which a priority of the plurality of tasks are determined, by performing reinforcement learning based on state information and reward information, wherein the state information and the reward information are generated by the storage controller by performing the plurality of tasks by the storage controller;
a resource analysis module configured to analyze a usage history and a usage status of a resource used for the plurality of tasks;
an access pattern analysis module configured to analyze an access pattern corresponding to the plurality of tasks; and
a queue analysis module configured to analyze information included in a queue according to the command, and
wherein the usage history, the usage status, the access pattern and the information included in the queue are provided to the machine learning module as the state information, and
wherein the storage controller further includes an encoding module configured to receive the usage history, the usage status, the access pattern, and the information included in the queue, and to encode the usage history, the usage status, the access pattern, and the information included in the queue into encoded information suitable for the reinforcement learning configured to be performed in the storage device,
wherein, based on the storage device operating in a first mode, the storage controller is configured to perform the plurality of tasks according to the scheduling result, and based on the storage device operating in a second mode, the storage controller is configured to perform the plurality of tasks according to a predetermined rule without following the scheduling result, and
wherein, based on the storage device operating in the first mode, the storage controller is further configured to generate the scheduling result by performing the reinforcement learning based on the command, and based on the storage device operating in a fourth mode, the storage controller is configured to perform the reinforcement learning according to a predetermined period.
8 . The storage device of claim 7 , wherein the storage controller further includes a performance analysis module configured to:
analyze a task execution result of the storage controller, and
provide the task execution result to the machine learning module as the reward information.
9 . The storage device of claim 8 , wherein the task execution result includes at least one from among a throughput of the storage controller measured in a unit of time, a latency of the command, or a quality of service (QoS) corresponding to the command.
10 . A storage device comprising:
a non-volatile memory; and
a storage controller configured to perform a plurality of tasks corresponding to the non-volatile memory based on a command received from an outside of the storage controller,
wherein the storage controller is configured to:
generate state information including a usage history and a usage status of a resource used for the plurality of tasks, an access pattern performed for the plurality of tasks and information included in a queue corresponding to the command, based on performing the plurality of tasks by the storage controller,
perform reinforcement learning using the state information and a reward information including an execution result corresponding to the plurality of tasks, and
perform scheduling by determining a priority of the plurality of tasks in accordance with a result of the reinforcement learning, and
wherein the storage controller includes an encoding module configured to receive the usage history, the usage status, the access pattern, the information included in the queue, and the execution result, and to convert the usage history, the usage status, the access pattern, the information included in the queue, and the execution result, into encoded information suitable for the reinforcement learning configured to be performed in the storage device,
wherein the storage controller is configured to operate in a first mode, a second mode, a third mode, and a fourth mode,
wherein, in the first mode, the storage controller performs the plurality of tasks according to the scheduling and generates a reinforcement learning result based on the command,
wherein, in the second mode, the storage controller performs the plurality of tasks according to a predetermined rule without performing the scheduling based on the reinforcement learning result,
wherein, in the third mode, the storage controller performs the reinforcement learning without performing the scheduling based on the reinforcement learning result, and
wherein, in the fourth mode, the storage controller performs the reinforcement learning according to a predetermined period.
11 . The storage device of claim 10 , wherein the resource includes the non-volatile memory, and a plurality of modules included in the storage controller.
12 . The storage device of claim 10 , wherein the storage controller is further configured to perform the reinforcement learning according to a predetermined period.