Granted Patent B2

US 12,699,942 · App. 18/345,922 · Granted Aug 4, 2026

Inventors: Michael Goodwin (Arlington, TX); Xiangchen Huo (Dallas, TX); Antonio Iniguez (Mesa, AZ); Brian Karp (Denver, CO); Jody Vandever (Earlham, IA); Robin Baldeo (Charlotte, NC); Sooji Ha (Moorestown, NJ); Efa Shukarey (Woodbury, MN); Prashamsh Takkalapally (Cumming, GA); Tsai-Hsuan Tsai (Harrison, NJ)

Assignee: Wells Fargo Bank, N.A.

G06Q10/0635G06Q10/06312

Abstract

Claims (50)

1 . A method comprising:

receiving, by a computing system, issue data for a plurality of issues from a risk management system, wherein the issue data for each respective issue of the plurality of issues includes an issue identifier and a due date for resolution of the respective issue;

generating, by the computing system and using at least one machine learning model and based on the issue data, due date revision risk scores for the plurality of issues, wherein a due date revision risk score for the respective issue indicates a predicted probability that the due date for resolution of the respective issue will need revision within a fixed period of time;

generating, by the computing system, a warning message for at least the respective issue of the plurality of issues based on the due date revision risk score for the respective issue being greater than a threshold;

tagging, by the computing system, the respective issue of the warning message for use as a feature input for training the at least one machine learning model;

determining, by the computing system, one or more features corresponding to a level of modified organizational attention applied to the tagged respective issue in response to the warning message; and

revising, by the computing system and as part of training the at least one machine learning model, parameters of the at least one machine learning model based on due date revision outcomes associated with the plurality of issues and using the feature input associated with the tagged respective issue, wherein revising the parameters of the at least one machine learning model includes:

retuning hyperparameters of the at least one machine learning model based on monitored due date revision outcomes that are associated with one or more issues of the plurality of issues for which warning messages were generated and based on the one or more features corresponding to the level of modified organizational attention applied to the tagged respective issue.

2 . The method of claim 1 , further comprising generating data representative of a dashboard for display via a display device, wherein the dashboard indicates one or more issue identifiers of one or more issues of the plurality of issues along with the due date revision risk scores associated with the one or more issues, and wherein the dashboard further indicates a ranking of the one or more issues based on the due date revision risk scores associated with the one or more issues.

3 . The method of claim 1 , wherein the fixed period of time is a first fixed period of time, and further comprising generating, by the computing system using at least one second machine learning model, second due date revision risk scores for the plurality of issues, wherein a second due date revision risk score for the respective issue indicates a predicted probability that the due date for resolution of the respective issue will need revision within a second fixed period of time, wherein the second fixed period of time is longer than the first fixed period of time.

4 . The method of claim 3 , wherein generating the second due date revision risk scores for the plurality of issues comprises generating the second due date revision risk scores for the plurality of issues in parallel with generating the due date revision risk scores for the plurality of issues.

5 . The method of claim 1 ,

wherein the one or more features are one or more first features,

wherein the at least one machine learning model is a random forest model that includes an ensemble of predictive elements,

wherein each predictive element determines whether the due date will need to be revised within the fixed period of time based on one or more second features of the issue data for the respective issue, and

wherein the due date revision risk score for the respective issue comprises a percentage of the predictive elements that determined that the due date will need to be revised within the fixed period of time.

6 . The method of claim 1 , further comprising preprocessing, by the computing system, the issue data for the plurality of issues, wherein preprocessing includes creating one or more variables from raw data of the issue data for the plurality of issues, wherein the one or more variables include one or more of numeric, categorical, or text variables.

7 . The method of claim 1 , wherein the one or more features are one or more first features, and further comprising processing, by the computing system, one or more variables created from the issue data for the plurality of issues into one or more second features used as input to the at least one machine learning model, wherein processing includes performing feature extraction on the one or more variables.

8 . The method of claim 7 , wherein performing feature extraction includes generating text features as term frequency-inverse document frequency (TFIDF) vectors from text variables created from the issue data for the plurality of issues, wherein the TFIDF vectors comprise numeric presentations of the text variables.

9 . The method of claim 7 , wherein performing feature extraction includes generating numeric features by imputing and scaling numeric variables created from the issue data for the plurality of issues.

10 . The method of claim 7 , wherein performing feature extraction includes generating categorical features by encoding categorical variables created from the issue data for the plurality of issues.

11 . The method of claim 1 , further comprising monitoring performance of the at least one machine learning model over time, wherein monitoring performance includes monitoring the due date revision outcomes for the one or more issues of the plurality of issues for which warning messages were generated as the monitored due date revision outcomes, and wherein the monitored due date revision outcomes comprise one or more of revised due dates, missed due dates with or without due date revision, or achieved due dates with or without due date revision.

12 . A computing system comprising:

one or more storage devices; and

processing circuitry in communication with the one or more storage devices, the processing circuitry configured to:

receive issue data for a plurality of issues from a risk management system, wherein the issue data for each respective issue of the plurality of issues includes an issue identifier and a due date for resolution of the respective issue;

generate, using at least one machine learning model and based on the issue data, due date revision risk scores for the plurality of issues, wherein a due date revision risk score for the respective issue indicates a predicted probability that the due date for resolution of the respective issue will need revision within a fixed period of time;

generate a warning message for at least the respective issue of the plurality of issues based on the due date revision risk score for the respective issue being greater than a threshold;

tag the respective issue of the warning message for use as a feature input for training the at least one machine learning model;

determine one or more features corresponding to a level of modified organizational attention applied to the tagged respective issue in response to the warning message; and

revise, as part of training the at least one machine learning model, parameters of the at least one machine learning model based on due date revision outcomes associated with the plurality of issues and using the feature input associated with the tagged respective issue, wherein to revise the parameters of the at least one machine learning model, the processing circuitry is configured to:

retune hyperparameters of the at least one machine learning model based on monitored due date revision outcomes that are associated with one or more issues of the plurality of issues for which warning messages were generated and based on the one or more features corresponding to the level of modified organizational attention applied to the tagged respective issue.

13 . The computing system of claim 12 , wherein the processing circuitry is further configured to generate data representative of a dashboard for display via a display device, wherein the dashboard indicates one or more issue identifiers of one or more issues of the plurality of issues along with the due date revision risk scores associated with the one or more issues, and wherein the dashboard further indicates a ranking of the one or more issues based on the due date revision risk scores associated with the one or more issues.

14 . The computing system of claim 12 , wherein the fixed period of time is a first fixed period of time, and wherein the processing circuitry is further configured to generate, using at least one second machine learning model, second due date revision risk scores for the plurality of issues, wherein a second due date revision risk score for the respective issue indicates a predicted probability that the due date for resolution of the respective issue will need revision within a second fixed period of time, wherein the second fixed period of time is longer than the first fixed period of time.

15 . The computing system of claim 14 , wherein to generate the second due date revision risk scores for the plurality of issues the processing circuitry is configured to generate the second due date revision risk scores for the plurality of issues in parallel with generating the due date revision risk scores for the plurality of issues.

16 . The computing system of claim 12 ,

wherein the one or more features are one or more first features,

wherein the at least one machine learning model is a random forest model that includes an ensemble of predictive elements,

wherein each predictive element determines whether the due date will need to be revised within the fixed period of time based on one or more second features of the issue data for the respective issue, and

wherein the due date revision risk score for the respective issue comprises a percentage of the predictive elements that determined that the due date will need to be revised within the fixed period of time.

17 . The computing system of claim 12 , wherein the processing circuitry is further configured to preprocess the issue data for the plurality of issues including creating one or more variables from raw data of the issue data for the plurality of issues, wherein the one or more variables include one or more of numeric, categorical, or text variables.

18 . The computing system of claim 12 , wherein the one or more features are one or more first features, wherein the processing circuitry is further configured to process one or more variables created from the issue data for the plurality of issues into one or more second features used as input to the at least one machine learning model, wherein processing includes performing feature extraction on the one or more variables.

19 . A non-transitory computer-readable storage medium comprising instructions that, when executed, cause processing circuitry to:

receive issue data for a plurality of issues from a risk management system, wherein the issue data for each respective issue of the plurality of issues includes an issue identifier and a due date for resolution of the respective issue;

generate, using at least one machine learning model and based on the issue data, due date revision risk scores for the plurality of issues, wherein a due date revision risk score for the respective issue indicates a predicted probability that the due date for resolution of the respective issue will need revision within a fixed period of time;

generate a warning message for at least the respective issue of the plurality of issues based on the due date revision risk score for the respective issue being greater than a threshold;

tag the respective issue of the warning message for use as a feature input for training the at least one machine learning model;

determine one or more features corresponding to a level of modified organizational attention applied to the tagged respective issue in response to the warning message; and

revise, as part of training the at least one machine learning model, parameters of the at least one machine learning model based on due date revision outcomes associated with the plurality of issues and using the feature input associated with the tagged respective issue, wherein to revise the parameters of the at least one machine learning model, the instructions cause the processing circuitry to:

retune hyperparameters of the at least one machine learning model based on monitored due date revision outcomes that are associated with one or more issues of the plurality of issues for which warning messages were generated and based on the one or more features corresponding to the level of modified organizational attention applied to the tagged respective issue.

Assignments (2)

Continuity (1)

References Cited (63)