System and method for rule generation using data processed by a binary classifier

1. A computer-implemented method that operates in conjunction with machine learning binary classification, the method for generating a rule relating to whether or not loan default will occur, the method comprising:

receiving original data originating from a plurality of loan applications, the original data including for each loan application of the plurality of loan applications: an income of a loan applicant, a requested loan duration, and a requested loan amount;

training a machine learning binary classifier using the original data to obtain a trained machine learning binary classifier that predicts whether loan default will occur;

additionally determining, by the computer, a set of data from the original data, the set of data having background sequences corresponding to instances of the loan default not occurring and foreground sequences corresponding to instances of the loan default occurring, each sequence position of each of the background sequences and each sequence position of each of the foreground sequences having a respective feature value, wherein a feature value is an input factor that influences whether or not the loan default occurs;

for each selected foreground sequence of a plurality of the foreground sequences, computing, by the computer, a plurality of difference values by, for each difference value:

obtaining a respective subsequence pattern comprising K feature values in a respective set of K positions of the selected foreground sequence, wherein K is at least one and no greater than a length of the selected foreground sequence; and

obtaining the difference value by computing a ratio of: (i) a first probability that the respective subsequence pattern is present in other foreground sequences, the other foreground sequences being different from the selected foreground sequence; and (ii) a second probability that the respective subsequence pattern is present in the background sequences, wherein the second probability is computed by multiplying together K probability values, each one of the K probability values corresponding to a different respective position of the K positions of the selected foreground sequence, and each one of the K probability values equal to a probability that the feature value at the respective position in the selected foreground sequence is located at the respective position in the background sequences, and wherein the probability that the feature value at the respective position in the selected foreground sequence is located at the respective position in the background sequences is obtained based on a frequency of how often the feature value is present at the respective position in the background sequences;

determining, by the computer, the rule based on a largest difference value for one of the plurality of foreground sequences, wherein the rule is a representation of one or more factors that contribute to a decision of the trained machine learning binary classifier;

transmitting information from the rule for presentation at a user interface in order to supplement an output from the trained machine learning binary classifier.

2. The computer-implemented method of claim 1 , wherein obtaining the rule comprises:

computing a score based on at least one of: (i) how often the respective subsequence pattern corresponding to the largest difference value is present in the foreground sequences, and (ii) how often the respective subsequence pattern corresponding to the largest difference value is present in the background sequences;

comparing the score to a threshold;

mapping the respective subsequence pattern corresponding to the largest difference value to the rule when the score passes the threshold.

3. The computer-implemented method of claim 1 , wherein the plurality of the foreground sequences is all of the foreground sequences.

4. The computer-implemented method of claim 1 , wherein computing a plurality of difference values for each selected foreground sequence of a plurality of the foreground sequences and determining the rule is repeated using subsequence patterns of a different length.

5. The computer-implemented method of claim 1 ,

wherein the original data comprises a plurality of samples, each sample corresponding to an instance of the loan default either occurring or not occurring, and each sample having a plurality of features, each one of the plurality of features taking on one of a plurality of values;

the method further comprising:

for each sample in the original data: mapping a value of a feature to another feature value of a finite number of categorized feature values;

partitioning the samples into the foreground sequences and the background sequences to result in the set of data, wherein each sample is designated as either: (i) a foreground sequence if the sample corresponds to an instance of the loan default occurring, or (ii) a background sequence if the sample corresponds to an instance of the loan default not occurring.

6. The computer-implemented method of claim 1 , wherein the rule is an if-then rule specifying an if-then condition relating to at least one of: whether loan default will occur and whether loan default will not occur.

7. The method of claim 1 , wherein a weight matrix storing a frequency of each feature value at each of one or more positions of the background sequences is used when computing the second probability that the respective subsequence pattern is present in the background sequences.

8. A system to operate in conjunction with machine learning binary classification, the system to generate a rule relating to whether or not loan default will occur, the system comprising:

a memory to store original data originating from a plurality of loan applications, the original data including for each loan application of the plurality of loan applications: an income of a loan applicant, a requested loan duration, and a requested loan amount;

a processor to train a machine learning binary classifier using the original data to obtain a trained machine learning binary classifier that predicts whether loan default will occur;

a rule extractor to determine a set of data from the original data, the set of data having background sequences corresponding to instances of the loan default not occurring and foreground sequences corresponding to instances of the loan default occurring, each sequence position of each of the background sequences and each sequence position of each of the foreground sequences having a respective feature value, wherein a feature value is an input factor that influences whether or not the loan default occurs;

the rule extractor further to:

for each selected foreground sequence of a plurality of the foreground sequences, compute a plurality of difference values by, for each difference value:

obtaining a respective subsequence pattern comprising K feature values in a respective set of K positions of the selected foreground sequence, wherein K is at least one and no greater than a length of the selected foreground sequence; and

obtaining the difference value by computing a ratio of: (i) a first probability that the respective subsequence pattern is present in other foreground sequences, the other foreground sequences being different from the selected foreground sequence; and (ii) a second probability that the respective subsequence pattern is present in the background sequences, wherein the second probability is computed by multiplying together K probability values, each one of the K probability values corresponding to a different respective position of the K positions of the selected foreground sequence, and each one of the K probability values equal to a probability that the feature value at the respective position in the selected foreground sequence is located at the respective position in the background sequences, and wherein the probability that the feature value at the respective position in the selected foreground sequence is located at the respective position in the background sequences is obtained based on a frequency of how often the feature value is present at the respective position in the background sequences;

determine the rule based on a largest difference value for one of the plurality of foreground sequences, wherein the rule is a representation of one or more factors that contribute to a decision of the trained machine learning binary classifier;

transmit information from the rule for presentation at a user interface in order to supplement an output from the trained machine learning binary classifier.

9. The system of claim 8 , wherein the rule extractor is to obtain the rule by:

computing a score based on at least one of: (i) how often the respective subsequence pattern corresponding to the largest difference value is present in the foreground sequences, and (ii) how often the respective subsequence pattern corresponding to the largest difference value is present in the background sequences;

comparing the score to a threshold;

mapping the respective subsequence pattern corresponding to the largest difference value to the rule when the score passes the threshold.

10. The system of claim 8 , wherein the plurality of the foreground sequences is all of the foreground sequences.

11. The system of claim 8 , wherein the rule extractor is to repeat computing a plurality of difference values for each selected foreground sequence of a plurality of the foreground sequences and repeat determining the rule, using subsequence patterns of a different length.

12. The system of claim 8 , wherein the original data comprises a plurality of samples, each sample corresponding to an instance of the loan default either occurring or not occurring, and each sample having a plurality of features, each one of the plurality of features taking on one of a plurality of values;

wherein the system is to generate the set of data by:

for each sample in the original data: mapping a value of a feature to another feature value of a finite number of categorized feature values;

partitioning the samples into the foreground sequences and the background sequences to result in the set of data, wherein each sample is designated as either: (i) a foreground sequence if the sample corresponds to an instance of the loan default occurring, or (ii) a background sequence if the sample corresponds to an instance of the loan default not occurring.

13. The system of claim 8 , wherein the memory is to store the rule for presentation of the rule to a user.

14. The system of claim 8 , wherein the rule is an if-then rule specifying an if-then condition relating to at least one of: whether loan default will occur and whether loan default will not occur.

15. The system of claim 8 , wherein a weight matrix storing a frequency of each feature value at each of one or more positions of the background sequences is used when computing the second probability that the respective subsequence pattern is present in the background sequences.