Domain-specific hallucination detection and correction for machine learning models
An incident ticket having a worklog field for a resolution field and a worklog providing a history of actions taken during attempts to resolve an incident may be received. The incident ticket may be processed using a domain-specific machine learning model trained using training data that includes a plurality of resolved incident tickets, to thereby generate at least one resolution statement. Source data used by the domain-specific machine learning model in providing the at least one resolution statement may be determined, the source data including one of the worklog and the training data. A hallucination score may be assigned to the at least one resolution statement, based on the source data, to identify hallucinated content within the at least one resolution statement. The at least one resolution statement may be modified to remove the hallucinated content and thereby obtain a resolution for inclusion in the resolution field.
1 . A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed by at least one computing device, are configured to cause the at least one computing device to:
analyze training data for a domain-specific machine learning model in conjunction with training of the domain-specific machine learning model, the training data including a plurality of resolved incident tickets, to thereby generate training analysis results;
receive an incident ticket of an incident domain, the incident ticket having a resolution field and a worklog field for a worklog providing a history of actions taken during attempts to resolve an incident;
process the incident ticket using the domain-specific machine learning model trained using the training data, to thereby output at least one resolution statement from the domain-specific machine learning model;
determine, after outputting the at least one resolution statement, source data used by the domain-specific machine learning model in providing the at least one resolution statement, the source data including one of the worklog and the training data;
assign a hallucination score to the at least one resolution statement, based on the source data and the training analysis results, to identify hallucinated content within the at least one resolution statement; and
modify the at least one resolution statement, including removing the hallucinated content, to thereby obtain a resolution for inclusion in the resolution field.
2 . The computer program product of claim 1 , wherein the plurality of resolved incident tickets each include a completed worklog and a completed resolution.
3 . The computer program product of claim 1 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
generate a domain vocabulary from the training data as part of the training analysis results; and
assign the hallucination score based in part on a comparison of the at least one resolution statement and the domain vocabulary.
4 . The computer program product of claim 1 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
identify metadata fields of the plurality of resolved incident tickets as part of the training analysis results; and
generate clusters of the plurality of resolved incident tickets based on the metadata fields.
5 . The computer program product of claim 4 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
determine a metadata field of the incident ticket; and
identify the source data within a cluster of the clusters having a metadata field that matches the metadata field of the incident ticket.
6 . The computer program product of claim 5 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
evaluating a similarity of content of incident tickets within the cluster with respect to an input of the domain-specific machine learning model; and
identify the source data as including a subset of the content, based on the similarity.
7 . The computer program product of claim 1 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
determine a metadata field of the incident ticket;
perform a search of the training data, based on the metadata field, to obtain a subset of incident tickets from the training data using the training analysis results; and
identify the source data as including content from the subset, based on a degree of similarity between the content and an input of the domain-specific machine learning model.
8 . The computer program product of claim 1 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
provide the resolution together with information identifying the source data within the training data.
9 . The computer program product of claim 1 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
assign the hallucination score using a hallucination equation defining an aggregated sum of hallucination factors, in which each hallucination factor is associated with a corresponding weight.
10 . The computer program product of claim 9 , wherein the instructions, when executed, are further configured to cause the at least one computing device to:
assign the hallucination score using a classifier trained to assign each weight of the hallucination equation.
11 . A computer-implemented method, the method comprising:
analyzing training data for a domain-specific machine learning model in conjunction with training of the domain-specific machine learning model, the training data including a plurality of resolved incident tickets, to thereby generate training analysis results;
receiving an incident ticket of an incident domain, the incident ticket having a resolution field and a worklog field for a worklog providing a history of actions taken during attempts to resolve an incident;
processing the incident ticket using the domain-specific machine learning model trained using the training data, to thereby output at least one resolution statement from the domain-specific machine learning model;
determining, after outputting the at least one resolution statement, source data used by the domain-specific machine learning model in providing the at least one resolution statement, the source data including one of the worklog and the training data;
assigning a hallucination score to the at least one resolution statement, based on the source data and the training analysis results, to identify hallucinated content within the at least one resolution statement; and
modifying the at least one resolution statement, including removing the hallucinated content, to thereby obtain a resolution for inclusion in the resolution field.
12 . The method of claim 11 , further comprising:
identifying metadata fields of the plurality of resolved incident tickets as part of the training analysis results; and
generating clusters of the plurality of resolved incident tickets based on the metadata fields.
13 . The method of claim 12 , further comprising:
determining a metadata field of the incident ticket; and
identifying the source data within a cluster of the clusters having a metadata field that matches the metadata field of the incident ticket.
14 . The method of claim 13 , further comprising:
evaluating a similarity of content of incident tickets within the cluster with respect to an input of the domain-specific machine learning model; and
identifying the source data as including a subset of the content, based on the similarity.
15 . The method of claim 11 , further comprising:
providing the resolution together with information identifying the source data within the training data.
16 . The method of claim 11 , further comprising:
assigning the hallucination score using a hallucination equation defining an aggregated sum of hallucination factors, in which each hallucination factor is associated with a corresponding weight.
17 . The method of claim 16 , further comprising:
assigning the hallucination score using a classifier trained to assign each weight of the hallucination equation.
18 . A system comprising:
at least one memory including instructions; and
at least one processor that is operably coupled to the at least one memory and that is arranged and configured to execute instructions that, when executed, cause the at least one processor to:
analyze training data for a domain-specific machine learning model in conjunction with training of the domain-specific machine learning model, the training data including a plurality of resolved incident tickets, to thereby generate training analysis results;
receive an incident ticket of an incident domain, the incident ticket having a resolution field and a worklog field for a worklog providing a history of actions taken during attempts to resolve an incident;
process the incident ticket using the domain-specific machine learning model trained using the training data, to thereby output at least one resolution statement from the domain-specific machine learning model;
determine, after outputting the at least one resolution statement, source data used by the domain-specific machine learning model in providing the at least one resolution statement, the source data including one of the worklog and the training data;
assign a hallucination score to the at least one resolution statement, based on the source data and the training analysis results, to identify hallucinated content within the at least one resolution statement; and
modify the at least one resolution statement, including removing the hallucinated content, to thereby obtain a resolution for inclusion in the resolution field.
19 . The system of claim 18 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
determine a metadata field of the incident ticket;
perform a search of the training data, based on the metadata field, to obtain a subset of incident tickets from the training data using the training analysis results; and
identify the source data as including content from the subset, based on a degree of similarity between the content and an input of the domain-specific machine learning model.
20 . The system of claim 18 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
assign the hallucination score using a hallucination equation defining an aggregated sum of hallucination factors, in which each hallucination factor is associated with a corresponding weight.