SYSTEMS AND METHODS FOR SUPERVISING AND IMPROVING GENERATIVE AI CONTENT
A system, method, and a computer program product for detecting an error in artificial intelligence (AI) generated content. A neural network receives the AI generated content and at least one prompt and determines a concept in the AI generated content. Source content corresponding to the concept is retrieved. The neural network receives the source content, the concept, and a list of errors in a configuration file to determine an error in the AI generated content.
1 . A system comprising:
a memory configured to store instructions for an artificial intelligence (AI) system; and
a processor coupled to the memory and configured to read the instructions from the memory to cause the system to perform operations, the operations comprising:
receiving an AI generated content at a neural network;
determining, using the neural network and at least one prompt, a concept in the AI generated content;
receiving source content for the concept in the AI generated content;
identifying, using the neural network, the source content, the concept, and a list of error types, an error in the AI generated content; and
inserting the AI generated content with the error into a queue, wherein the AI generated content in the queue is displayed on an editable AI interface.
2 . The system of claim 1 , wherein the operations further comprise:
displaying the AI generated content on the editable AI interface;
receiving, via the editable AI interface, instructions to modify the AI generated content based on the error; and
providing the modified AI generated content for display on another user device.
3 . The system of claim 2 , wherein the operations further comprise:
providing the modified AI generated content to at least one neural network in a plurality of neural networks for retraining the at least one neural network using the modified AI generated content.
4 . The system of claim 1 , wherein the operations for identifying the error in the AI generated content further comprise:
querying a database for the source content that includes the concept; and
comparing a concept in the source content with the concept in the AI generated content to identify the error.
5 . The system of claim 1 , wherein the neural network is a language model.
6 . The system of claim 1 , wherein the operations further comprise:
identifying one or more trigger words within the AI generated content, wherein the one or more trigger words are in a pre-defined list.
7 . The system of claim 1 , wherein operations further comprise:
receiving an updated AI generated content that corresponds to the AI generated content with the error;
identifying that there is no error in the updated AI generated content; and
removing the AI generated content from the queue, wherein the updated AI generated content is generated in response to new source content available in a database.
8 . A method comprising:
determining, using a neural network executing on a processor and at least one prompt, a concept in an AI generated content;
querying a database for source content from which the AI generated content was generated, wherein the source content is associated with the concept;
identifying, using the neural network, the source content, and the concept, an error in the AI generated content; and
inserting the AI generated content with the error into a queue, wherein the AI generated content in the queue is displayed on an editable AI interface.
9 . The method of claim 8 , further comprising:
displaying the AI generated content on an editable AI interface; and
modifying, using the editable AI interface, the AI generated content based on the error.
10 . The method of claim 9 , further comprising:
providing the modified AI generated content to the neural network for retraining the neural network using the modified AI generated content.
11 . The method of claim 8 , further comprising:
providing, to the neural network, a list of errors in a configuration file, wherein the list of errors includes the error.
12 . The method of claim 8 , further comprising:
identifying one or more trigger words within the AI generated content, wherein the one or more trigger words are in a pre-defined list; and
wherein the determining is based on identifying the one or more trigger words within the AI generated content.
13 . The method of claim 8 , further comprising:
receiving an updated AI generated content;
identifying that there is no error in the updated AI generated content; and
removing the AI generated content from the queue, wherein the updated AI generated content is generated in response to the database being updated with a new source content.
14 . The method of claim 8 , wherein the AI generated content comprises at least one of:
a patient summary;
a discharge barrier; or
a physician note.
15 . A non-transitory computer-readable medium having instructions thereon, that when executed by a processor, cause the processor to perform operations comprising:
receiving an AI generated summary;
determining, using a neural network and at least one prompt, a concept in the AI generated summary;
querying a database for source content for the concept in the AI generated summary;
identifying, using the neural network, the source content, and the AI generated summary, an error in the AI generated summary; and
inserting the AI generated summary in a queue available in an editable AI interface.
16 . The non-transitory computer-readable medium of claim 15 , wherein the operations further comprising:
displaying the AI generated summary on the editable AI interface; and
receiving instructions to modify, using an editable AI interface, the AI generated summary based on the error.
17 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprising:
providing the modified AI generated summary to the neural network for retraining the neural network using the modified AI generated summary.
18 . The non-transitory computer-readable medium of claim 17 , wherein the AI generated summary is parsed by a language model.
19 . The non-transitory computer-readable medium of claim 15 , the operations further comprising:
identifying one or more trigger words within the AI generated summary, wherein the one or more trigger words are stored in a configuration file; and
identifying the error in the AI generated summary when the AI generated summary includes the one or more trigger words.
20 . The non-transitory computer-readable medium of claim 15 , the operations further comprising:
receiving an updated AI generated summary;
identifying that there is no error in the updated AI generated summary; and
removing the AI generated summary from the queue, wherein the updated AI generated summary is generated in response to new source content available in the database.