Methods, systems, and computer program products for selecting criteria subsets for performing a medical necessity review with automatic evidence highlighting

1 . A method comprising:

receiving, by one or more processors, input information associated with a health record of a patient, the input information comprising a plurality of input variable tokens;

embedding, by the one or more processors, the plurality of input variable tokens using a natural language processing embedding module to generate a plurality of input variable token vectors, respectively;

aggregating, by the one or more processors, the plurality of input variable token vectors to generate a patient health record vector;

generating an identification of a criterion used for determining an appropriateness of a care plan for the patient based on the patient health record vector, by the one or more processors using an application specific integrated circuit (ASIC) for an artificial neural network including an input layer having a first plurality of neurons corresponding to the plurality of input variable token vectors, a plurality of hidden layers having a second plurality of neurons, and an output layer having a third plurality of neurons each corresponding to a different criteria subset for determining the appropriateness of the care plan for the patient, wherein the first and second pluralities of neurons have associated weights that pass data from one layer to another layer in response to determining that an output of a rectified linear unit (ReLU) activation function exceeds a threshold after performing a multiply-accumulate (MAC) operation on inputs to one of the first and second pluralities of neurons using the associated weights;

generating, by the one or more processors using the artificial neural network, a ranking of respective ones of the plurality of input variable tokens based on how much each of the plurality of input variable tokens contributed to the identification of the criterion used for determining the appropriateness of the care plan; and

providing, by the one or more processors, automatic evidence highlighting of the patient health record using the ranking of the respective ones of the plurality of input variable tokens to identify specific clinical information within the patient health record which contributed most to identifying the criterion used for determining the appropriateness of the care plan.

2 . The method of claim 1 , wherein generating the ranking of the respective ones of the plurality of input variable tokens comprises:

generating, by the one or more processors, the ranking of the respective ones of the plurality of input variable tokens using an Integrated Gradient (IG) methodology.

3 . The method of claim 2 , wherein generating the ranking of the respective ones of the plurality of input variable tokens using the IG methodology comprises:

generating, by the one or more processors, a baseline vector;

determining, by the one or more processors, a baseline gradient magnitude value of the artificial neural network for the baseline vector;

determining, by the one or more processors, a plurality of token gradient magnitude values of the artificial neural network for the plurality of input variable token vectors, respectively;

determining, by the one or more processors, a plurality of gradient change magnitude values of the artificial neural network based on a plurality of differences between the plurality of token gradient magnitude values and the baseline gradient magnitude value, respectively; and

generating, by the one or more processors, the ranking of the respective ones of the plurality of input variable tokens based on the plurality of gradient change magnitude values.

4 . The method of claim 3 , wherein the baseline vector is not representative of any information that affects generating identification of the criterion using the artificial neural network.

5 . The method of claim 3 , further comprising:

generating, by the one or more processors, the plurality of input variable tokens using a bidirectional transformer; wherein embedding the plurality of input variable tokens to generate the plurality of input variable token vectors comprises:

embedding, by the one or more processors, the plurality of input variable tokens to generate the plurality of input variable token vectors using the bidirectional transformer.

6 . The method of claim 5 , wherein the bidirectional transformer is a Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) bidirectional transformer.

7 . The method of claim 3 , further comprising:

generating, by the one or more processors, a mapping between the plurality of input variable tokens and segments of the input information.

8 . The method of claim 7 , further comprising:

generating, by the one or more processors, a ranking of the segments of the input information based on the ranking of the respective ones of the plurality of input variable tokens and the mapping.

9 . The method of claim 8 , wherein the segments of the input information comprise sentences, medications, lab tests, and/or procedures.

10 . A system, comprising:

an application-specific integrated circuit (ASIC);

one or more processors; and

one or more memories storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving input information associated with a health record of a patient, the input information comprising a plurality of input variable tokens;

embedding the plurality of input variable tokens using a natural language processing embedding module to generate a plurality of input variable token vectors, respectively;

aggregating the plurality of input variable token vectors to generate a patient health record vector;

generating an identification of a criterion used for determining an appropriateness of a care plan for the patient based on the patient health record vector, using the ASIC for an artificial neural network including an input layer having a first plurality of neurons corresponding to the plurality of input variable token vectors, a plurality of hidden layers having a second plurality of neurons, and an output layer having a third plurality of neurons each corresponding to a different criteria subset for determining an appropriateness of a care plan for the patient, wherein the first and second pluralities of neurons have associated weights that pass data from one layer to another layer in response to determining that an output of a rectified linear unit (ReLU) activation function exceeds a threshold after performing a multiply-accumulate (MAC) operation on inputs to one of the first and second pluralities of neurons using the associated weights;

generating, using the artificial neural network, a ranking of respective ones of the plurality of input variable tokens based on how much each of the plurality of input variable tokens contributed to the identification of the criterion used for determining the appropriateness of the care plan; and

providing automatic evidence highlighting of the patient health record using the ranking of the respective ones of the plurality of input variable tokens to identify specific clinical information within the patient health record which contributed most to identifying the criterion used for determining the appropriateness of the care plan.

11 . The system of claim 10 , wherein generating the ranking of the respective ones of the plurality of input variable tokens comprises:

generating the ranking of the respective ones of the plurality of input variable tokens using an Integrated Gradient (IG) methodology.

12 . The system of claim 11 , wherein generating the ranking of the respective ones of the plurality of input variable tokens using the IG methodology comprises:

generating a baseline vector;

determining a baseline gradient magnitude value of the artificial neural network for the baseline vector;

determining a plurality of token gradient magnitude values of the artificial neural network for the plurality of input variable token vectors, respectively;

determining a plurality of gradient change magnitude values of the artificial neural network based on a plurality of differences between the plurality of token gradient magnitude values and the baseline gradient magnitude value, respectively; and

generating the ranking of the respective ones of the plurality of input variable tokens based on the plurality of gradient change magnitude values.

13 . The system of claim 12 , wherein the operations further comprise:

generating the plurality of input variable tokens using a bidirectional transformer;

wherein embedding the plurality of input variable tokens to generate the plurality of input variable token vectors comprises:

embedding the plurality of input variable tokens to generate the plurality of input variable token vectors using the bidirectional transformer.

14 . The system of claim 12 , wherein the operations further comprise:

generating a mapping between the plurality of input variable tokens and segments of the input information.

15 . The system of claim 14 , wherein the operations further comprise:

generating a ranking of the segments of the input information based on the ranking of the respective ones of the plurality of input variable tokens and the mapping.

16 . One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving input information associated with a health record of a patient, the input information comprising a plurality of input variable tokens;

embedding the plurality of input variable tokens using a natural language processing embedding module to generate a plurality of input variable token vectors, respectively;

aggregating the plurality of input variable token vectors to generate a patient health record vector;

generating an identification of a criterion used for determining a appropriateness of a care plan for the patient based on the patient health record vector, using an artificial neural network in an application-specific integrated circuit (ASIC) including an input layer having a first plurality of neurons corresponding to the plurality of input variable token vectors, a plurality of hidden layers having a second plurality of neurons, and an output layer having a third plurality of neurons each corresponding to a different criteria subset for determining the appropriateness of the care plan for the patient, wherein the first and second pluralities of neurons have associated weights that pass data from one layer to another layer in response to determining that an output of a rectified linear unit (ReLU) activation function exceeds a threshold after performing a multiply-accumulate (MAC) operation on inputs to one of the first and second pluralities of neurons using the associated weights;

generating, using the artificial neural network, a ranking of respective ones of the plurality of input variable tokens based on how much each of the plurality of input variable tokens contributed to the identification of the criterion used for determining the appropriateness of the care plan; and

providing automatic evidence highlighting of the patient health record using the ranking of the respective ones of the plurality of input variable tokens to identify specific clinical information within the patient health record which contributed most to identifying the criterion used for determining the appropriateness of the care plan.

17 . The one or more non-transitory computer-readable claim 16 , wherein generating the ranking of the respective ones of the plurality of input variable tokens comprises:

generating a baseline vector;

determining a baseline gradient magnitude value of the artificial neural network for the baseline vector;

determining a plurality of token gradient magnitude values of the artificial neural network for the plurality of input variable token vectors, respectively;

determining a plurality of gradient change magnitude values of the artificial neural network based on a plurality of differences between the plurality of token gradient magnitude values and the baseline gradient magnitude value, respectively; and

generating the ranking of the respective ones of the plurality of input variable tokens based on the plurality of gradient change magnitude values.

18 . The one or more non-transitory computer-readable claim 17 , wherein the operations further comprise:

generating the plurality of input variable tokens using a bidirectional transformer;

wherein embedding the plurality of input variable tokens to generate the plurality of input variable token vectors comprises:

embedding the plurality of input variable tokens to generate the plurality of input variable token vectors using the bidirectional transformer.

19 . The one or more non-transitory computer-readable claim 16 , wherein the operations further comprise:

generating a mapping between the plurality of input variable tokens and segments of the input information.

20 . The one or more non-transitory computer-readable claim 19 , wherein the operations further comprise:

generating a ranking of the segments of the input information based on the ranking of the respective ones of the plurality of input variable tokens and the mapping.