System of correcting LLM hallucinations
View Patent ↗A system of correcting inaccuracies in chatbot responses, the system comprising a processing circuitry (PC) configured to: receive a chatbot's response to a user prompt, and receive data sources associated with the response; present, to the user, the chatbot response, and receive, from the user, one or more text segments, the one or more text segments being associated with a user-suspected hallucination or inaccuracy of the chatbot response; perform, on one or more of the data sources, at least one of: a text search, and a semantic search of one or more of the text segments; and present, to the user, data derivative of the text search and/or semantic search, thereby confirmation or correction of the chatbot response.
1 . A system of correcting inaccuracies in chatbot responses, the system comprising a processing circuitry (PC) configured to:
receive at least part of a chatbot's response to a user prompt, and receive data indicative of data sources, the data sources being indicated by the chatbot as relevant to the chatbot's response;
present, to the user, data derivative of the chatbot response, and receive, from the user, data indicative of one or more text segments,
the one or more text segments being associated with a user-suspected hallucination or inaccuracy of the chatbot response;
perform, on one or more of the data sources, at least one of:
a text search, and
a semantic search
of one or more of the text segments; and
present, to the user, data derivative of the text search and/or semantic search, thereby providing confirmation or correction of the chatbot response.
2 . The system of claim 1 , wherein at least one of the one or more data sources is provided by the chatbot.
3 . The system of claim 1 , wherein the received data indicative of one or more text segments is derivative of, at least, a user utilization of a device's user interface while viewing the chatbot response.
4 . The system of claim 1 , wherein the data derivative of the text search and/or semantic search is null, thereby being indicative of a hallucination in the chatbot response.
5 . The system of claim 1 , wherein the data derivative of the text search and/or semantic search is a text segment of a data source, thereby providing a clarification, confirmation, or correction of the response.
6 . The system of claim 1 , wherein the chatbot is a large language mode (LLM).
7 . The system of claim 1 , the data derivative of the text search and/or semantic search is: a corrected chatbot response.
8 . A processing circuitry-based method of correcting inaccuracies in chatbot responses, the method comprising:
receiving at least part of a chatbot's response to a user prompt, and receive data indicative of data sources, the data sources being indicated by the chatbot as relevant to the chatbot's response;
presenting, to the user, data derivative of the chatbot response, and receive, from the user, data indicative of one or more text segments,
the one or more text segments being associated with a user-suspected hallucination or inaccuracy of the chatbot response;
performing, on one or more of the data sources, at least one of:
a text search, and
a semantic search
of one or more of the text segments; and
presenting, to the user, data derivative of the text search and/or semantic search, thereby providing confirmation or correction of the chatbot response.
9 . A computer program product comprising a computer readable non-transitory storage medium containing program instructions, which program instructions when read by a processing circuitry, cause the processing circuitry to perform a method of correcting inaccuracies in chatbot responses, the method comprising:
receiving at least part of a chatbot's response to a user prompt, and receive data indicative of data sources, the data sources being indicated by the chatbot as relevant to the chatbot's response;
presenting, to the user, data derivative of the chatbot response, and receive, from the user, data indicative of one or more text segments,
the one or more text segments being associated with a user-suspected hallucination or inaccuracy of the chatbot response;
performing, on one or more of the data sources, at least one of:
a text search, and
a semantic search
of one or more of the text segments; and
presenting, to the user, data derivative of the text search and/or semantic search, thereby providing confirmation or correction of the chatbot response.