Systems and methods for detecting errors and hallucinations in generative model output data
Provided herein are systems and methods that can detect errors and hallucinations in output data produced by generative models. The detection systems described herein may compare information from the generative model output data with ground truth information to determine whether the generative model output data comprises errors and/or hallucinations. The systems and methods described herein may generate an output indicative of whether the generative model output data comprises errors and/or hallucinations. The systems and methods described herein can readily detect false information generated by NLG systems, thus potential harms of reliance on false information can be minimized.
1 . A system for detecting errors and hallucinations in generative model output data, the system comprising one or more processors configured to cause the system to:
receive output data from a generative model;
generate a first data structure based on the generative model output data, wherein the first data structure comprises a first graph comprising a first plurality of nodes and a first plurality of edges;
compare the first data structure to a second data structure, wherein the second data structure represents fact data, to determine whether the generative model output data comprises one or more of an error and a hallucination;
based on a determination that the generative model output data comprises the one or more of an error and a hallucination, generate an output indicating the determination.
2 . The system of claim 1 , wherein determining that the generative model output data comprises an error comprises determining that a portion of the generative model output data contradicts ground truth information used to generate the second data structure.
3 . The system of claim 1 , wherein determining that the generative model output data comprises a hallucination comprises determining that a portion of the generative model output data is not substantiated by ground truth information used to generate the second data structure.
4 . The system of claim 1 , wherein the one or more processors are configured to cause the system to generate the second data structure based on ground truth information.
5 . The system of claim 4 , wherein the ground truth information comprises one or more of inputs of the generative model and authoritative documentation.
6 . The system of claim 1 , wherein the one or more processors are configured to cause the system to display the generated output indicating the determination.
7 . The system of claim 1 , wherein the generated output comprises a representation of ground truth information that contradicts the generative model output data.
8 . The system of claim 1 , wherein the generated output comprises an indication of one or more portions of the generative model output data that is not substantiated by ground truth information.
9 . The system of claim 1 , wherein the one or more processors are configured to cause the system to, based on a determination that the generative model output data does not comprise the one or more of an error and a hallucination, generate a second output indicating the determination.
10 . The system of claim 9 , wherein the second generated output comprises a representation of ground truth information that substantiates the generative model output data.
11 . The system of claim 1 , wherein the one or more processors are configured to cause the system to, based on a determination that the generative model output data comprises the one or more of an error and a hallucination, cause the generative model to generate a new output.
12 . The system of claim 1 , wherein each node of the first plurality of nodes represents a respective extracted entity of the generative model output data and each edge of the first plurality of edges represents a relationship between two of the extracted entities.
13 . The system of claim 1 , wherein each node of the first plurality of nodes represents a respective resource description framework triple, and wherein each resource description framework triple is associated with a respective extracted entity of the generative model output data.
14 . The system of claim 1 , wherein the second data structure comprises a second graph comprising a second plurality of nodes and a second plurality of edges, wherein one or more nodes of the second plurality of nodes represents a respective resource description framework triple associated with a respective extracted entity of the fact data.
15 . A method for detecting errors and hallucinations in generative model output data, comprising:
receiving output data from a generative model;
generating a first data structure based on the generative model output data, wherein the first data structure comprises a first graph comprising a first plurality of nodes and a first plurality of edges;
comparing the first data structure to a second data structure to determine that the generative model output data comprises one or more of an error and a hallucination; and
based on the determination that the generative model output data comprises the one or more of an error and a hallucination, generating an output indicating the determination.
16 . A non-transitory computer-readable storage medium storing one or more programs for detecting errors and hallucinations in generative model output data, the programs for execution by one or more processors of an electronic device that when executed by the device, cause the device to:
receive output data from a generative model;
generate a first data structure based on the generative model output data, wherein the first data structure comprises a first graph comprising a first plurality of nodes and a first plurality of edges;
compare the first data structure to a second data structure to determine whether the generative model output data comprises one or more of an error and a hallucination; and
based on a determination that the generative model output data comprises the one or more of an error and a hallucination, generate an output indicating the determination.