SYSTEMS AND METHODS FOR DATA PARSING USING LARGE LANGUAGE MODEL
A computer-implemented method for report parsing using a large language model. The method includes receiving a plurality of raw reports; filtering the plurality of raw reports; extracting raw data from the plurality of raw reports; providing the extracted raw data to a large language model (LLM); providing a prompt to the LLM; receiving a response from the LLM, the response including data labels derived from the extracted raw data; validating the received response against the plurality of raw reports; and training a machine learning model using the received response.
1 . A computer-implemented method for report parsing using a large language model, the method comprising:
receiving a plurality of raw reports;
filtering the plurality of raw reports;
extracting raw data from the plurality of raw reports;
providing the extracted raw data to a large language model (LLM);
providing a prompt to the LLM;
receiving a response from the LLM, the response including data labels derived from the extracted raw data;
validating the received response against the plurality of raw reports; and
training a machine learning model using the received response.
2 . The computer-implemented method of claim 1 , further comprising, upon determining that one or more reports among the plurality of raw reports failed validation:
modifying the prompt or one or more reports among the plurality of raw reports; and
providing the modified prompt or the modified one or more reports to the LLM.
3 . The computer-implemented method of claim 1 , wherein determining that one or more reports among the plurality of raw reports failed validation further comprises comparing a number of the one or more reports that failed validation against a predetermined threshold.
4 . The computer-implemented method of claim 1 , further comprising, upon determining that a number of reports among the plurality of raw reports that failed validation has increased:
modifying the prompt to remove any modifications, or
modifying the prompt to match an intermediate version of the prompt, or
request manual intervention by a user.
5 . The computer-implemented method of claim 1 , wherein the plurality of raw reports includes one or more of: genomic assays, surgical reports, diagnostic information, histochemical stainings, or second opinions.
6 . The computer-implemented method of claim 1 , wherein the prompt includes instructions to cite a portion of an underlying raw report among the plurality of raw reports for each data label derived from the extracted raw data.
7 . The computer-implemented method of claim 6 , wherein validating the received response against the plurality of raw reports comprises comparing each derived data label against the cited portion of the underlying raw report.
8 . The computer-implemented method of claim 1 , wherein the prompt includes instructions for determining data labels.
9 . The computer-implemented method of claim 1 , wherein filtering the plurality of raw reports further comprises:
detecting keywords in each raw report among the plurality of raw reports, or
parsing each raw report among the plurality of raw reports using a second large language model (second LLM).
10 . A system for report parsing using a large language model, the system comprising:
a data storage device storing instructions for report parsing using a large language model in an electronic storage medium; and
a processor configured to execute the instructions to perform operations comprising:
receiving a plurality of raw reports;
filtering the plurality of raw reports;
extracting raw data from the plurality of raw reports;
providing the extracted raw data to a large language model (LLM);
providing a prompt to the LLM;
receiving a response from the LLM, the response including data labels derived from the extracted raw data;
validating the received response against the plurality of raw reports; and
training a machine learning model using the received response.
11 . The system of claim 10 , wherein the operations further comprise, upon determining that one or more reports among the plurality of raw reports failed validation:
modifying the prompt or one or more reports among the plurality of raw reports; and
providing the modified prompt or the modified one or more reports to the LLM.
12 . The system of claim 10 , wherein determining that one or more reports among the plurality of raw reports failed validation further comprises comparing a number of the one or more reports that failed validation against a predetermined threshold.
13 . The system of claim 10 , wherein the operations further comprise, upon determining that a number of reports among the plurality of raw reports that failed validation has increased:
modifying the prompt to remove any modifications, or
modifying the prompt to match an intermediate version of the prompt, or
request manual intervention by a user.
14 . The system of claim 10 , wherein the prompt includes instructions to cite a portion of an underlying raw report among the plurality of raw reports for each data label derived from the extracted raw data.
15 . The system of claim 14 , wherein validating the received response against the plurality of raw reports comprises comparing each derived data label against the cited portion of the underlying raw report.
16 . A non-transitory machine-readable medium storing instructions that, when executed by a computing system, causes the computing system to perform operations for report parsing using a large language model, the operations comprising:
receiving a plurality of raw reports;
filtering the plurality of raw reports;
extracting raw data from the plurality of raw reports;
providing the extracted raw data to a large language model (LLM);
providing a prompt to the LLM;
receiving a response from the LLM, the response including data labels derived from the extracted raw data;
validating the received response against the plurality of raw reports; and
training a machine learning model using the received response.
17 . The non-transitory machine-readable medium of claim 16 , the operations further comprising, upon determining that one or more reports among the plurality of raw reports failed validation:
modifying the prompt or one or more reports among the plurality of raw reports; and
providing the modified prompt or the modified one or more reports to the LLM.
18 . The non-transitory machine-readable medium of claim 16 , wherein determining that one or more reports among the plurality of raw reports failed validation further comprises comparing a number of the one or more reports that failed validation against a predetermined threshold.
19 . The non-transitory machine-readable medium of claim 16 , the operations further comprising, upon determining that a number of reports among the plurality of raw reports that failed validation has increased:
modifying the prompt to remove any modifications, or
modifying the prompt to match an intermediate version of the prompt, or
request manual intervention by a user.
20 . The non-transitory machine-readable medium of claim 16 , wherein the prompt includes instructions to cite a portion of an underlying raw report among the plurality of raw reports for each data label derived from the extracted raw data.