Granted Patent B2

US 12,596,738 · App. 19/313,684 · Granted Apr 7, 2026

Inventors: Ganesh Prasad Bhat (New Jersey, NY); Zheyu Wang (Shanghai, CN); Haolin Jin (Shanghai, CN); Sourabh Deb (Tampa, FL); Jason Ryan Engelbrecht (London, GB); Payal Jain (London, GB); Tariq Husayn Maonah (London, GB); Mariusz Saternus (Cracow, PL); Daniel Lewandowski (Cracow, PL); Biraj Krushna Rath (London, GB); Stuart Murray (London, GB); Philip Davies (London, GB); Julisia Jackson (Irving, TX); Chamindra Desilva (London, GB); Shardul Malviya (London, GB); Wayne Liao (London, GB); Deepak Jain (London, GB); Samantha Cory (London, GB); Vishal Mysore (Mississauga, CA); Ramkumar Ayyadurai (Jersey City, NJ); James Myers (Clearwater, FL)

Assignee: Citibank, N.A.

G06F16/338G06F16/383

Abstract

Claims (97)

1 . One or more non-transitory, computer-readable storage medium comprising instructions recorded thereon, wherein the instructions, when executed by at least one data processor of a system, cause the system to:

receive, via an input interface, a query from a user;

determine a plurality of characteristics of the query, the plurality of characteristics comprising: (1) a complexity of the query, (2) a subject matter domain of the query, (3) regulatory constraints for the query, and (4) performance requirements for processing the query;

retrieve, from a model matrix, a plurality of profiles for a plurality of large language models (LLMs), each of the plurality of profiles comprising: (1) performance attributes, (2) resource consumption, and (3) regulatory compliance parameters;

select a particular LLM to process the query by:

applying one or more criteria to balance the resource consumption and the performance attributes of one or more LLMs of the plurality of LLMs with the performance requirements for processing the query;

evaluating regulatory compliance by cross-referencing the regulatory compliance parameters for one or more LLMs against the regulatory constraints for the query;

ranking the plurality of LLMs according to: (1) results of the applying of the one or more criteria and (2) results of the regulatory compliance;

prioritizing LLMs of the plurality of LLMs having successfully processed past queries matching the subject matter domain or the complexity of the query; and

selecting a particular LLM of the plurality of LLMs according to selection logic that accounts for the ranking and the prioritizing;

generate a structured, human-readable explanation of the selection of the particular LLM, the structured, human-readable explanation comprising decision factors, rationale for LLM selection, and alternative LLMs considered when selecting the particular LLM; and

record, in an immutable audit trail data structure, the selection of the particular LLM and the structured, human-readable explanation, wherein the audit trail data structure is secured to provide tamper-evident recordkeeping.

2 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for determining the plurality of characteristics of the query further cause the system to:

classify the query into a subject matter domain using a domain classification model, wherein the domain classification model is configured to assign the query to a domain based on one or more features extracted from the query;

analyze the complexity of the query by comparing features of the query against a knowledge base comprising predefined complexity metrics and patterns;

determine the regulatory constraints for the query by invoking a rules engine configured to identify jurisdictional and sector-specific compliance requirements for the query; and

determine the performance requirements for the query by analyzing the query using a performance predictor, wherein the performance predictor is configured to assess expected response time, memory requirements, and priority level for processing the query.

3 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for selecting the particular LLM to process the query further cause the system to:

apply the one or more criteria to compare, for each LLM, the resource consumption and the performance attributes against the performance requirements of the query, wherein the one or more criteria comprise a tradeoff analysis between minimizing computational cost and maximizing response quality;

generate, for the plurality of LLMs, a plurality of scores indicating a degree of fit to the performance requirements of the query while managing resource consumption; and

generate a ranking of the plurality of LLMs based on the plurality of scores, wherein the selection of the particular LLM is based at least in part on the ranking.

4 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for evaluating regulatory compliance by cross-referencing the regulatory compliance parameters for each LLM against the regulatory constraints for the query further cause the system to:

determine, for each of the plurality of profiles for the plurality of LLMs, whether the profile includes a plurality of certifications and authorizations required by the regulatory constraints for the query; and

exclude from further consideration any LLM for which the profile lacks at least one of the plurality of certifications and authorizations.

5 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for generating the structured, human-readable explanation of the selection of the particular LLM further cause the system to:

parse the selection logic used to select the particular LLM to determine a plurality of selection factors;

select, from a plurality of explanation templates, an explanation template having a format corresponding to the plurality of selection factors and a type of the query; and

embed, within the explanation template, both the selection logic for selecting the particular LLM and a list of alternative LLMs, each alternative LLM annotated with at least one reason for non-selection.

6 . The one or more non-transitory, computer-readable storage medium of claim 1 , wherein the instructions for recording in the immutable audit trail data structure further cause the system to:

generate a hash of the structured, human-readable explanation;

record a timestamp in the hash using a secure time server;

encrypt the hash with a cryptographic signature; and

transmit the hash to a tamper-evident, append-only log.

7 . A method comprising:

receiving a query from a user;

determining a plurality of characteristics of the query, the plurality of characteristics comprising regulatory constraints for the query and performance requirements for processing the query;

retrieving a plurality of profiles for a plurality of models, each of the plurality of profiles comprising: (1) performance attributes, (2) resource consumption, and (3) regulatory compliance parameters;

applying one or more criteria to balance the resource consumption and the performance attributes of each model of the plurality of models with the performance requirements for processing the query;

evaluating regulatory compliance by cross-referencing the regulatory compliance parameters for each model against the regulatory constraints for the query;

selecting a particular model according to: (1) results of the applying of the one or more criteria and (2) results of the regulatory compliance;

generating a structured, human-readable explanation of the selection of the particular model; and

recording, in an immutable audit trail data structure, the selection of the particular model and the structured, human-readable explanation.

8 . The method of claim 7 , wherein determining the plurality of characteristics of the query further comprises:

determining the regulatory constraints for the query by invoking a rules engine configured to identify jurisdictional and sector-specific compliance requirements for the query; and

determining the performance requirements for the query by analyzing the query using a performance predictor, wherein the performance predictor is configured to assess expected response time, memory requirements, and priority level for processing the query.

9 . The method of claim 7 , wherein selecting the particular model to process the query further comprises:

applying the one or more criteria to compare, for each model, the resource consumption and the performance attributes against the performance requirements of the query, wherein the one or more criteria comprise a tradeoff analysis between minimizing computational cost and maximizing response quality;

generating, for the plurality of models, a plurality of scores indicating a degree of fit to the performance requirements of the query while managing resource consumption; and

generating a ranking of the plurality of models based on the plurality of scores, wherein the selection of the particular model is based at least in part on the ranking.

10 . The method of claim 7 , wherein evaluating regulatory compliance by cross-referencing the regulatory compliance parameters for each model against the regulatory constraints for the query further comprises:

determining, for each of the plurality of profiles for the plurality of models, whether the profile includes a plurality of certifications and authorizations required by the regulatory constraints for the query; and

excluding from further consideration any model for which the profile lacks at least one of the plurality of certifications and authorizations.

11 . The method of claim 7 , wherein generating the structured, human-readable explanation of the selection of the particular model further comprises:

determining a plurality of selection factors for selecting the particular model;

selecting, from a plurality of explanation templates, an explanation template having a format corresponding to the plurality of selection factors and a type of the query; and

embedding, within the explanation template, both the plurality of selection factors for selecting the particular model and a list of alternative models, each alternative model annotated with at least one reason for non-selection.

12 . The method of claim 7 , wherein recording in the immutable audit trail data structure further comprises:

generating a hash of the structured, human-readable explanation;

recording a timestamp in the hash using a secure time server;

encrypting the hash with a cryptographic signature; and

transmitting the hash to a tamper-evident, append-only log.

13 . The method of claim 7 , wherein the plurality of characteristics further comprise a complexity of the query and a subject matter domain of the query, further comprising:

prioritizing models of the plurality of models having successfully processed past queries matching the subject matter domain or the complexity of the query; and

selecting the particular model of the plurality of models further based on the prioritizing.

14 . A system comprising:

a storage device; and

one or more processors communicatively coupled to the storage device storing instructions thereon, that cause the one or more processors to:

receive a request from a user;

determine a plurality of characteristics of the request, the plurality of characteristics comprising regulatory constraints for the request and performance requirements for processing the request;

retrieve one or more profiles for one or more models, each of the one or more profiles comprising: (1) performance attributes, (2) resource consumption, and (3) regulatory compliance parameters;

apply one or more criteria relating to the resource consumption and the performance attributes of the one or more models and the performance requirements for processing the request;

evaluate regulatory compliance by cross-referencing the regulatory compliance parameters for each model against the regulatory constraints for the request;

select a particular model according to: (1) results of the applying of the one or more criteria and (2) results of the regulatory compliance;

generate a structured, human-readable explanation of the selection of the particular model; and

record, in an immutable audit trail data structure, the selection of the particular model and the structured, human-readable explanation.

15 . The system of claim 14 , wherein the instructions for determining the plurality of characteristics of the request further cause the one or more processors to:

determine the regulatory constraints for the request by invoking a rules engine configured to identify jurisdictional and sector-specific compliance requirements for the request; and

determine the performance requirements for the request by analyzing the request using a performance predictor, wherein the performance predictor is configured to assess expected response time, memory requirements, and priority level for processing the request.

16 . The system of claim 14 , wherein the instructions for selecting the particular model to process the request further cause the one or more processors to:

apply the one or more criteria to compare, for each model, the resource consumption and the performance attributes against the performance requirements of the request, wherein the one or more criteria comprise a tradeoff analysis between minimizing computational cost and maximizing response quality;

generate, for the one or more models, a one or more scores indicating a degree of fit to the performance requirements of the request while managing resource consumption; and

generate a ranking of the one or more models based on the one or more scores, wherein the selection of the particular model is based at least in part on the ranking.

17 . The system of claim 14 , wherein the instructions for evaluating regulatory compliance by cross-referencing the regulatory compliance parameters for each model against the regulatory constraints for the request further cause the one or more processors to:

determine, for each of the one or more profiles for the one or more models, whether the profile includes a plurality of certifications and authorizations required by the regulatory constraints for the request; and

exclude from further consideration any model for which the profile lacks at least one of the plurality of certifications and authorizations.

18 . The system of claim 14 , wherein the instructions for generating the structured, human-readable explanation of the selection of the particular model further cause the one or more processors to:

determine a plurality of selection factors for selecting the particular model;

select, from a plurality of explanation templates, an explanation template having a format corresponding to the plurality of selection factors and a type of the request; and

embed, within the explanation template, both the plurality of selection factors for selecting the particular model and a list of alternative models, each alternative model annotated with at least one reason for non-selection.

19 . The system of claim 14 , wherein the instructions for recording in the immutable audit trail data structure further cause the one or more processors to:

generate a hash of the structured, human-readable explanation;

record a timestamp in the hash using a secure time server;

encrypt the hash with a cryptographic signature; and

transmit the hash to a tamper-evident, append-only log.

20 . The system of claim 14 , wherein the plurality of characteristics further comprise a complexity of the request and a subject matter domain of the request, and wherein the instructions further cause the one or more processors to:

prioritize models of the one or more models having successfully processed past queries matching the subject matter domain or the complexity of the request; and

select the particular model of the one or more models further based on the prioritizing.

Assignments (1)

Continuity (13)

References Cited (326)