INSTANCE WEIGHTED LEARNING MACHINE LEARNING MODEL

1 . A method of employing an instance weighted learning (IWL) machine learning model, the method comprising:

identifying a temporal sequence of reinforcement learning machine learning training instances, each of the training instances including a state-action pair;

determining a first quality value for a first training instance in the temporal sequence of reinforcement learning machine learning training instances;

determining a second quality value for a second training instance in the temporal sequence of reinforcement learning machine learning training instances, the first quality value being higher than the second quality value;

associating the first quality value with the first training instance; and

associating the second quality value with the second training instance.

2 . The method as recited in claim 1 , further comprising:

training a classifier using the first training instance weighted with a first weighting factor that is a function of the first quality value, and

training the classifier using the second training instance weighted with a second weighting factor that is a function of the second quality value.

3 . The method as recited in claim 2 , wherein the training of the classifier is influenced more by the first training instance than by the second training instance due to the first quality value being higher than the second quality value.

4 . The method as recited in claim 2 , wherein:

the first quality value is a represented by positive number which causes the first weighting factor to be a positive weighting factor;

the training the classifier using the first training instance weighted with the positive first weighting factor tends to encourage learning to support training instances that are similar to the first training instance;

the second quality value is a represented by negative number which causes the second weighting factor to be a negative weighting factor; and

the training the classifier using the second training instance weighted with the negative second weighting factor tends to discourage learning to support training instances that are similar to the second training instance.

5 . The method as recited in claim 2 , wherein the classifier comprises a multilayer perceptron (MLP) neural network, another multilayer neural network, a decision tree, or a support vector machine.

6 . The method as recited in claim 1 , wherein the determining the first quality value for the first training instance and the determining the second quality value for the second training instance include:

determining a reward of a current training instance in the temporal sequence;

determining a first discounted portion of the reward for inclusion in the first quality value; and

determining a second discounted portion of the reward for inclusion in the second quality value,

wherein the first training instance and the second training instance occur previous to the current training instance in the temporal sequence.

7 . A non-transitory computer-readable medium storing a program configured to cause a processor to execute the method as recited in claim 1 .

8 . A method of employing an instance weighted learning (IWL) machine learning model to train a classifier, the method comprising:

identifying a set of machine learning training instances;

determining a first quality value for a first training instance in the set of machine learning training instances;

determining a second quality value for a second training instance in the set of machine learning training instances, the first quality value being higher than the second quality value;

associating the first quality value with the first training instance;

associating the second quality value with the second training instance;

training a classifier using the first training instance weighted using the first quality value; and

training the classifier using the second training instance weighted using the second quality value,

wherein the training of the classifier is influenced more by the first training instance than by the second training instance due to the first quality value being higher than the second quality value.

9 . The method as recited in claim 8 , wherein:

the weighting of the first training instance using the first quality value includes weighting the first training instance with a first weighting factor that is a function of the first quality value; and

the weighting of the second training instance using the second quality value includes weighting the second training instance with a second weighting factor that is a function of the second quality value.

10 . The method as recited in claim 9 , wherein the set of machine learning training instances includes a temporal sequence of machine learning training instances.

11 . The method as recited in claim 10 , wherein the determining the first quality value for the first training instance and the determining the second quality value for the second training instance include:

determining a reward of a current training instance in the temporal sequence;

determining a first discounted portion of the reward for inclusion in the first quality value; and

determining a second discounted portion of the reward for inclusion in the second quality value.

12 . The method as recited in claim 11 , wherein:

the first training instance and the second training instance both occur previous to the current training instance in the temporal sequence; and

each of the first discounted portion and the second discounted portion is reduced the farther that corresponding training instance is positioned in the temporal sequence from the current training instance.

13 . The method as recited in claim 8 , wherein:

the first quality value is a represented by positive number which causes the first weighting factor to be a positive weighting factor;

the training the classifier using the first training instance weighted with the positive first weighting factor tends to encourage learning to support training instances that are similar to the first training instance;

the second quality value is a represented by negative number which causes the second weighting factor to be a negative weighting factor; and

the training the classifier using the second training instance weighted with the negative second weighting factor tends to discourage learning to support training instances that are similar to the second training instance.

14 . A non-transitory computer-readable medium storing a program configured to cause a processor to execute the method as recited in claim 8 .

15 . A method of employing an instance weighted learning (IWL) machine learning model to train a classifier, the method comprising:

identifying a temporal sequence of machine learning training instances;

determining a first quality value for a first training instance in the temporal sequence of machine learning training instances;

determining a second quality value for a second training instance in the temporal sequence of machine learning training instances, the first quality value being higher than the second quality value;

associating the first quality value with the first training instance;

associating the second quality value with the second training instance;

training a classifier using the first training instance weighted with a first weighting factor that is a function of the first quality value;

training the classifier using the second training instance weighted with a second weighting factor that is a function of the second quality value;

wherein the training of the classifier is influenced more by the first training instance than by the second training instance due to the first quality value being higher than the second quality value.

16 . The method as recited in claim 15 , wherein the first weighting factor and the second weighting factor are determined according to any monotonically increasing function that satisfies the following formula:

if ( q 1 ≧q 2 ), then ( u ( q 1 )≧ u ( q 2 )), where:

q 1 is the first quality value;

q 2 is the second quality value;

u(q 1 ) is the first weighting factor; and

u(q 2 ) is the second weighting factor.

17 . The method as recited in claim 15 , wherein:

the first weighting factor is determined according the following formula:

u ( q 1 )=( a+b·q 1 ); and

the second weighting factor is determined according the following formula:

u ( q 2 )=( a+b·q 2 ), where

q 1 is the first quality value;

u(q 1 ) is the first weighting factor;

a is a first empirical parameter;

b is a second empirical parameter.

q 2 is the second quality value; and

u(q 2 ) is the second weighting factor.

18 . The method as recited in claim 15 , wherein:

each of the training instances is a multiple output dependency (MOD) machine learning training instance and each of the training instances includes multiple interdependent output components; and

the training the classifier using the first training instance and the training the classifier using the second training instance include employing a hierarchical based sequencing (HBS) machine learning model or a multiple output relaxation (MOR) machine learning model to train a separate classifier for each one of the multiple interdependent output components.

19 . The method as recited in claim 18 , wherein each MOD machine learning training instance is a lead response management (LRM) MOD machine learning training instance.

20 . A non-transitory computer-readable medium storing a program configured to cause a processor to execute the method as recited in claim 15 .