Granted Patent B1

US 12,373,649 · App. 19/052,036 · Granted Jul 29, 2025

Inventors: Sherin Mathews (Fremont, CA); Samuel Assefa (Watertown, MA)

Assignee: U.S. BANCORP, NATIONAL ASSOCIATION

G06F40/30G06F16/243G06F40/289

Abstract

Claims (86)

1. A system for remediating hallucinations in a first language model, the system comprising one or more processors and one or more machine-readable media storing program instructions that, when executed by the one or more processors, causes the one or more processors to perform operations comprising:

receiving, from a chat interface, a natural language query, the natural language query comprising a request for a plurality of facts;

executing the first language model using the natural language query to generate a text-based response to the natural language query;

generating a filtered text-based response including one or more statements from the text-based response that satisfy one or more filtering criteria;

segmenting, using a second language model, the filtered text-based response into a plurality of segments each corresponding to a different fact-based response to the natural language query;

for each of the plurality of segments:

converting the segment into a set of queries each configured to be used to search a data repository;

for each query of the set of queries:

generating a query response to the query by using the query to search the data repository; and

generating a factuality score of the segment for the text-based response to the natural language query based on the set of queries generated for the segment;

generating an aggregate factuality score for the natural language query based on the factuality score for each of the plurality of segments; and

computing atomic decontextualization of the response, responsive to determining the factuality score is below a threshold for metrics in which a high value corresponds with a high factuality or is above the threshold for metrics in which a low value corresponds to an increase in the factuality score.

2. The system of claim 1 , wherein filtering the text-based response comprises removing one or more statements from the text-based response that includes at least one characteristic from a group of characteristics consisting of:

a first-person pronoun;

a presence of one or more question marks;

a pronoun of a defined type; and

an absence of a named entity.

3. The system of claim 1 , wherein the operations further comprise:

generating a plurality of metrics for the natural language query, the plurality of metrics comprising the aggregate factuality score; and

comparing, by the one or more processors, each of the plurality of metrics to a respective threshold.

4. A method for remediating hallucinations in a language model, comprising:

receiving, by one or more processors, a natural language query;

executing, by the one or more processors, the language model using the natural language query to generate a text-based response to the natural language query;

segmenting, by the one or more processors, the text-based response into a plurality of segments that each corresponds to a different fact-based response to the natural language query;

generating, by the one or more processors, one or more queries for each of the plurality of segments, each of the one or more queries configured to be used to search a data repository;

for each query of the one or more queries generated for each of the plurality of segments, generating, by the one or more processors, a query response to the query by using the query to search the data repository;

generating, by the one or more processors, one or more metrics based on the query response generated for each query of the one or more queries generated for each of the plurality of segments; and

performing decontextualization of the response, at an atomic level, by the one or more processors, based on at least one of the one or more metrics not satisfying a threshold.

5. The method of claim 4 , further comprising:

filtering, by the one or more processors, the text-based response to the natural language query by removing language from the text-based response that satisfy one or more filtering criteria,

wherein segmenting the text-based response comprises segmenting, by the one or more processors, the filtered text-based response.

6. The method of claim 5 , wherein filtering the text-based response comprises removing, by the one or more processors, one or more statements from the text-based response that includes at least one characteristic from a group of characteristics consisting of:

a first-person pronoun;

a presence of one or more question marks;

a pronoun of a defined type; and

an absence of a named entity.

7. The method of claim 4 , wherein segmenting the text-based response into a plurality of segments comprises:

identifying, by the one or more processors using a second language model, individual facts included in the text-based response, wherein each individual fact is a segment of the plurality of segments.

8. The method of claim 4 , wherein generating the one or more queries for each of the plurality of segments comprises, for each segment:

generating, by the one or more processors using a second language model, the one or more queries for the segment using a second language model.

9. The method of claim 4 , wherein generating the one or more metrics comprises:

generating, by the one or more processors for each segment of the plurality of segments, a factuality score indicating a factuality of the segment to responding to the text-based response to the natural language query, the method further comprising:

generating, by the one or more processors, an aggregate factuality score for the natural language query based on the factuality score for each segment of the plurality of segments,

wherein performing decontextualization, by the one or more processors, the language model responsive to the aggregate factuality score not satisfying a threshold.

10. The method of claim 4 , wherein generating the one or more metrics comprises, for each query response:

generating, by the one or more processors, a hallucinated named entity error metric,

wherein training the language model comprises training, by the one or more processors, the language model responsive to the hallucinated named entity error not satisfying a threshold.

11. The method of claim 4 , further comprising:

generating, by the one or more processors in response to determining the at least one metric does not satisfy the threshold, a second plurality of metrics for the response to the natural language query;

determining, by the one or more processors, at least one second metric of the second plurality of metrics that does not satisfy a second threshold;

wherein training the language model comprises training, by the one or more processors, the language model using a training technique selected from a plurality of training techniques based on a type of the at least one second metric.

12. The method of claim 4 , further comprising:

storing, by the one or more processors, one or more ground truth sentences; and

generating, by the one or more processors for each query response, one or more query metrics for the query response based on a comparison between the query response and a stored ground truth sentence.

13. The method of claim 4 , wherein generating the one or more metrics comprises:

generating, by the one or more processors, a fluency, a diversity, and a repetition for a segment of the plurality of segments,

wherein training the language model based on the one or more metrics comprises:

determining at least one of the generated fluency diversity, or repetition satisfy the threshold; and

training the language model responsive to the determining the at least one of the generated fluency diversity, or repetition satisfy the threshold.

14. One or more non-transitory, machine-readable media storing program instructions that, when executed by one or more processors, causes the one or more processors to perform operations comprising:

receiving a natural language query;

executing a language model using the natural language query to generate a text-based response to the natural language query;

segmenting the text-based response into a plurality of segments that each corresponds to a different fact-based response to the natural language query;

generating one or more queries for each of the plurality of segments, each of the one or more queries configured to be used to search a data repository;

for each query of the one or more queries generated for each of the plurality of segments, generating a query response to the query by using the query to search the data repository;

generating one or more metrics based on the query response generated for each query of the one or more queries generated for each of the plurality of segments; and

executing atomic level decontextualization of the response, based on the one or more metrics being below a threshold for metrics in which a high value corresponds with a high factuality or being above the threshold for metrics in which a low value corresponds to an increase in the factuality score.

15. The one or more non-transitory, machine-readable media of claim 14 , the operations further comprising:

filtering the text-based response to the natural language query by removing language from the text-based response that satisfy one or more filtering criteria,

wherein segmenting the text-based response comprises segmenting the filtered text-based response.

16. The one or more non-transitory, machine-readable media of claim 15 , wherein filtering the text-based response comprises removing one or more statements from the text-based response that includes at least one characteristic from a group of characteristics consisting of:

a first-person pronoun;

a presence of one or more question marks;

a pronoun of a defined type; and

an absence of a named entity.

17. The one or more non-transitory, machine-readable media of claim 14 , wherein segmenting the text-based response into a plurality of segments comprises:

identifying, by the one or more processors using a second language model, individual facts included in the text-based response, wherein each individual fact is a segment of the plurality of segments.

18. The one or more non-transitory, machine-readable media of claim 14 , wherein generating the one or more queries for each of the plurality of segments comprises, for each segment:

generating, by the one or more processors using a second language model, the one or more queries for the segment using a second language model.

19. The one or more non-transitory, machine-readable media of claim 14 , wherein generating the one or more metrics comprises:

generating for each query response, a factuality score indicating a factuality of the query response to responding to the text-based response to the natural language query, the operations further comprising:

generating, by the one or more processors, an aggregate factuality score for the natural language query based on the factuality score for each query response generated for each of the plurality of segments,

wherein executing decontextualization comprises of atomic level decontextualization responsive to the aggregate factuality score not satisfying a threshold.

20. The one or more non-transitory, machine-readable media of claim 14 , wherein generating the one or more metrics comprises, for each query response:

generating a hallucinated named entity error metric,

wherein training the language model comprises training, by the one or more processors, the language model responsive to the hallucinated named entity error not satisfying a threshold.

Assignments (1)

References Cited (48)

Cited By (3)

US 12,554,726 US 12,566,769 US 12,670,160