Machine learning model improvement
There is disclosed, in an example, a computer-implemented system and method, which includes providing a large set of validation prompts; testing a first ML intent model with the large set of validation prompts, wherein the first ML intent model is to select for respective validation prompts a first intent from an intent set; testing a second ML intent model with the large set of validation prompts, wherein the second ML intent model is to select for the same validation prompts a second intent from the intent set; selecting a reduced set of validation prompts, comprising validation prompts for which the first intent and second intent do not match; receiving an analysis of the reduced set of validation prompts, including indicia of hits, wherein one of the ML intent models inferred a correct intent; and selecting as a preferred model an ML model of the first ML intent model or second ML model that provided more hits.
1 . A computer-implemented method, comprising:
providing a large set of validation prompts;
testing a first machine learning (ML) intent model with the large set of validation prompts, wherein the first ML intent model is to select for respective validation prompts a first intent from an intent set;
testing a second ML intent model with the large set of validation prompts, wherein the second ML intent model is to select for the same validation prompts a second intent from the intent set;
selecting a reduced set of validation prompts, comprising validation prompts for which the first intent and second intent do not match;
receiving an analysis of the reduced set of validation prompts, including indicia of hits, wherein one of the ML intent models inferred a correct intent;
selecting as a preferred model an ML model of the first ML intent model or second ML model that provided more hits; and
deploying the preferred model to process artificial intelligence (AI) workloads.
2 . The method of claim 1 , wherein the analysis of the reduced set of validation prompts is a human analysis.
3 . The method of claim 1 , wherein the second ML model is an updated version of the first ML intent model trained on a large set of training prompts.
4 . The method of claim 3 , wherein the second ML model receives supervised training on the large set of training prompts.
5 . The method of claim 3 , wherein the second ML model receives unsupervised training on the large set of training prompts.
6 . The method of claim 3 , further comprising generating the large set of training prompts with aid of a large language model (LLM).
7 . The method of claim 6 , further comprising a human analyst prompting the LLM to generate the large set of training prompts.
8 . The method of claim 7 , further comprising the human analyst rejecting undesirable training prompts.
9 . The method of claim 3 , further comprising the human analyst prompting the LLM to generate a set of varied identifiers for entities.
10 . The method of claim 3 , wherein the large set of training prompts comprise more than 50 training prompts per intent in the intent set.
11 . The method of claim 1 , wherein the intent set comprises more than 50 intents.
12 . The method of claim 1 , wherein the first ML intent model and second ML model are natural language processing (NLP) models.
13 . The method of claim 1 , wherein the first ML intent model and second ML model are to provide an interactive voice response (IVR) system.
14 . The method of claim 1 , wherein the first ML intent model and second ML model are to provide an interactive voice assistant (IVA).
15 . The method of claim 1 , wherein the first ML intent model and second ML model are to provide a customer service function.
16 . One or more tangible, nontransitory computer-readable storage media having stored thereon executable instructions to:
provide a large set of validation prompts;
test a first machine learning (ML) intent model with the large set of validation prompts, wherein the first ML intent model is to select for respective validation prompts a first intent from an intent set;
test a second ML intent model with the large set of validation prompts, wherein the second ML intent model is to select for the same validation prompts a second intent from the intent set;
select a reduced set of validation prompts, comprising validation prompts for which the first intent and second intent do not match;
receive an analysis of the reduced set of validation prompts, including indicia of hits, wherein one of the ML intent models inferred a correct intent;
select as a preferred model an ML model of the first ML intent model or second ML model that provided more hits; and
deploy the preferred model to process artificial intelligence (AI) workloads.
17 . The one or more tangible, nontransitory computer-readable storage media of claim 16 , wherein the second ML model is an updated version of the first ML intent model trained on a large set of training prompts.
18 . A computing apparatus, comprising:
a hardware platform comprising a processor circuit and a memory; and
instructions encoded within the memory to instruct the processor circuit to:
provide a large set of validation prompts;
test a first machine learning (ML) model with the large set of validation prompts, wherein the first ML model is to select for respective validation prompts a first intent from an intent set;
test a second ML model with the large set of validation prompts, wherein the second ML model is to select for the same validation prompts a second intent from the intent set;
select a reduced set of validation prompts, comprising validation prompts for which the first intent and second intent do not match;
receive an analysis of the reduced set of validation prompts, including indicia of hits, wherein one of the ML models inferred a correct intent;
select as a preferred model an ML model of the first ML model or second ML model that provided more hits; and
deploy the preferred model to process artificial intelligence (AI) workloads.
19 . The computing apparatus of claim 18 , wherein the second ML model is an updated version of the first ML model trained on a large set of training prompts.