Learning system and learning method comprising an event list database
A learning system according to the present invention includes an event list database for storing a plurality of event lists, each of the event lists being a set including a series of state-action pairs which reaches a state-action pair immediately before earning a reward, an event list managing section for classifying state-action pairs into the plurality of event lists for storing, and a learning control section for updating expectation of reward of a state-action pair which is an element of each of the event lists.
1. A learning system comprising:
an event list database for storing a plurality of event lists, each of the event lists being a set including a series of state-action pairs which reaches a state-action pair immediately before earning a reward;
an event list managing section for classifying state-action pairs into the plurality of event lists for storing; and
a learning control section for updating expectation of reward of a state-action pair which is an element of each of the plurality of event lists.
2. A learning system according to claim 1 further comprising a temporary list storing section wherein every time an action is selected the event list managing section has the state-action pair stored in the temporary list storing section and every time a reward is earned the event list managing section has a state-action pair in a set of state-action pairs stored in the temporary list storing section, which has not been stored in the event list database, stored as an element of the event list of the state-action pair immediately before earning the reward in the event list database.
3. A learning system according to claim 1 wherein every time a reward is earned the learning control section updates, using a value of the reward, expectation of reward of a state-action pair which is an element of the event list of the state-action pair immediately before earning the reward and updates, using 0 as a value of reward, expectation of reward of a state-action pair which is an element of the event lists except the event list of the state-action pair immediately before earning the reward.
4. A learning method in a learning system having an event list database for storing a plurality of event lists, each of the event lists being a set including a series of state-action pairs which reaches a state-action pair immediately before earning a reward, an event list managing section and a learning control section, the method comprising the steps of:
classifying, by the event list managing section, state-action pairs into the plurality of event lists for storing; and
updating, by the learning control section, expectation of reward of a state-action pair which is an element of each of the plurality of event lists.
5. A learning method according to claim 4 wherein every time an action is selected the event list managing section has the state-action pair temporarily stored and every time a reward is earned the event list managing section has a state-action pair in a set of state-action pairs temporarily stored, which has not been stored in the event list database, stored as an element of the event list of the state-action pair immediately before earning the reward in the event list database.
6. A learning method according to claim 4 wherein every time a reward is earned the learning control section updates, using a value of the reward, expectation of reward of a state-action pair which is an element of the event list of the state-action pair immediately before earning the reward and updates, using 0 as a value of reward, expectation of reward of a state-action pair which is an element of the event lists except the event list of the state-action pair immediately before earning the reward.