Providing a secure and collaborative feedback mechanism for machine learning models

1 . A method for providing feedback mechanism for machine learning models, comprising:

receiving, by a device and from a server, a machine learning model and training data to train the machine learning model;

receiving, by the device and from a user device, a user input for the machine learning model,

wherein the user input includes:

a request for a prediction of a relapse probability for a patient,

information associated with health of the patient, and

an authorization information of the user;

processing, by the device, the training data and the user input, with the machine learning model, to generate the relapse probability prediction for the patient, and an explanation of the relapse probability prediction for the patient,

wherein the machine learning model includes a predictive machine learning model to generate the relapse probability prediction for the patient and an explainable machine learning model to generate the explanation of the relapse probability prediction for the patient;

providing, by the device, the relapse probability prediction for the patient and the explanation of the relapse probability prediction for the patient to the user device;

receiving, by the device and from the user device, prediction feedback for the relapse probability prediction for the patient and explanation feedback for the explanation of the relapse probability prediction for the patient;

generating, via the user device, a change (AP) for the relapse probability prediction for the patient as the prediction feedback and a change (AE) for the explanation of the relapse probability prediction for the patient as the explanation feedback;

determining, by the device, coherency between the prediction feedback and the explanation feedback on the basis of |P o +dT+dN+dM−P a |<c,

wherein P o is the prediction, P a is the prediction feedback, dT is a change in a T stage feature, dN is a change in a N stage feature, dM is a change in a M stage feature and c is a correction;

receiving, by the device, entries of predictions, prediction feedback and explanation feedback from other users from an interaction log of the other users;

computing, by the device, an agreement between the prediction feedback of the user and the prediction feedback received from the other users and an agreement between the explanation feedback of the user and the explanation feedback received from the other users based on the determined coherency between the prediction feedback and the explanation feedback of the user;

determining, by the device, that the agreement is less than or equal to a quorum threshold;

determining, by the device, that the agreement is achieved between the prediction feedback and the explanation feedback based on the determination that the agreement is less than or equal to the quorum threshold;

updating, by the device, the machine learning model based on the determination that the agreement is achieved between the prediction feedback and the explanation feedback, to generate an updated machine learning model;

cryptographically protecting, by the device, the updated machine learning model to generate an updated and cryptographically protected machine learning model, by:

generating a hash based on the prediction feedback and the explanation feedback, and

generating a block for the updated machine learning model based on the hash,

wherein the block is part of a block chain and corresponds to the updated and cryptographically protected machine learning model;

performing, by the device, one or more actions based on the updated and cryptographically protected machine learning model; and

implementing, by the device, the updated and cryptographically protected machine learning model to make predictions associated with the relapse probability of the patient to expedite treatment of the patient.

2 . The method of claim 1 , further comprising:

preventing an update of the machine learning model based on the agreement not being achieved between the prediction feedback and the explanation feedback.

3 . The method of claim 1 , wherein calculating the agreement level between the prediction feedback and the explanation feedback comprises:

calculating the agreement level between the prediction feedback and the explanation feedback based on weights assigned to domain experts providing the prediction feedback and the explanation feedback.

4 . The method of claim 1 , wherein determining that the agreement is achieved between the prediction feedback and the explanation feedback based on the threshold comprises:

determining that the agreement is achieved between the prediction feedback and the explanation feedback based on weights assigned to domain experts providing the prediction feedback and the explanation feedback.

5 . The method of claim 1 , wherein the prediction feedback includes feedback, about the prediction, provided by one or more domain experts, and wherein the explanation feedback includes feedback, about the explanation, provided by one or more domain experts.

6 . The method of claim 1 , wherein updating the machine learning model comprises:

updating the machine learning model, to generate the updated machine learning model, based on the prediction feedback and the explanation feedback.

7 . The method of claim 1 , wherein determining whether the agreement is achieved between the prediction feedback and the explanation feedback based on an average corrected prediction, an average of explanation corrections, and the quorum threshold.

8 . A device, for providing feedback mechanism for machine learning models, comprising:

one or more memories; and

one or more processors, coupled to the one or more memories, configured to:

receive, from a server, the machine learning model and training data to train the machine learning model;

receive, from a user device, a user input for the machine learning model;

wherein the user input includes:

a request for a prediction of a relapse probability for a patient,

information associated with the health of the patient, sand

an authorization information of the user;

process the training data and the user input, with the machine learning model, to generate the relapse probability prediction for the patient, and an explanation of the relapse probability prediction for the patient,

wherein the machine learning model includes a predictive machine learning model to generate the relapse probability prediction for the patient and an explainable machine learning model to generate the explanation of the relapse probability prediction for the patient;

provide the relapse probability prediction for the patient and the explanation of the relapse probability prediction for the patient to the user device;

receive, from the user device, prediction feedback for the relapse probability prediction for the patient and explanation feedback for the explanation of the relapse probability prediction for the patient;

generate, via the user device, a change (AP) for the relapse probability prediction for the patient as the prediction feedback and a change (AE) for the explanation of the relapse probability prediction for the patient as the explanation feedback;

determining, by the device, coherency between the prediction feedback and the explanation feedback on the basis of |P o +dT+dN+dM−P a |<c,

wherein P o is the prediction, P a is the prediction feedback, dT is a change in a T stage feature, dN is a change in a N stage feature, dM is a change in a M stage feature and c is a correction;

receive entries of predictions, prediction feedback and explanation feedback from other users from an interaction log of the other users;

compute an agreement between the prediction feedback of the user and the prediction feedback received from the other users and an agreement between the explanation feedback of the user and the explanation feedback received from the other users based on the determined coherency between the prediction feedback and the explanation feedback of the user;

determine that the agreement is less than or equal to a quorum threshold;

determine that the agreement is achieved between the prediction feedback and the explanation feedback based on determination that the agreement is less than or equal to the quorum threshold;

update the machine learning model based on the determination that the agreement is achieved between the prediction feedback and the explanation feedback, to generate an updated machine learning model;

cryptographically protect the updated machine learning model to generate an updated and cryptographically protected machine learning model by,

generating a hash based on the prediction feedback and the explanation feedback, and

generating a block for the updated machine learning model based on the hash,

wherein the block is part of a block chain and corresponds to the updated and cryptographically protected machine learning model;

perform one or more actions based on the updated and cryptographically protected machine learning model; and

implement the updated and cryptographically protected machine learning model to make predictions associated with the relapse probability of the patient to expedite treatment of the patient.

9 . The device of claim 8 , wherein the device is a decision support system.

10 . The device of claim 8 , wherein the one or more processors, to perform the one or more actions based on the updated and cryptographically protected machine learning model, are configured to:

provide the updated and cryptographically protected machine learning model for display; or

cause the updated and cryptographically protected machine learning model to be implemented.

11 . The device of claim 8 , wherein the one or more processors, to perform the one or more actions based on the updated and cryptographically protected machine learning model, are configured to:

retrain the updated and cryptographically protected machine learning model with the training data.

12 . The device of claim 8 , wherein the one or more processors, to perform the one or more actions based on the updated and cryptographically protected machine learning model, are configured to:

receive additional prediction feedback and additional explanation feedback based on the updated and cryptographically protected machine learning model.

13 . The device of claim 8 , wherein the one or more processors, to perform the one or more actions based on the updated and cryptographically protected machine learning model, are configured to:

generate a new prediction and a new explanation based on the updated and cryptographically protected machine learning model.

14 . The device of claim 8 , wherein the one or more processors to update the machine learning model, are configured to:

encrypt the interaction log with an updated prediction, updated explanation, an original prediction, and an original explanation.

15 . A non-transitory computer-readable medium storing a set of instructions, the set of instructions for providing feedback mechanism for machine learning models, comprising:

one or more instructions that, when executed by one or more processors of a device, cause the device to:

receive, from a server, the machine learning model and training data to train the machine learning model;

receive, from a user device, a user input for the machine learning model;

wherein the user input includes:

a request for a prediction of a relapse probability for a patient,

information associated with the health of the patient, and

an authorization information of the user;

process the training data and the user input, with the machine learning model, to generate the relapse probability prediction for the patient, and an explanation of the relapse probability prediction for the patient,

wherein the machine learning model includes a predictive machine learning model to generate the relapse probability prediction for the patient and an explainable machine learning model to generate the explanation of the relapse probability prediction for the patient;

provide the relapse probability prediction for the patient and the explanation of the relapse probability prediction for the patient to the user device;

receive, from the user device, prediction feedback for the relapse probability prediction for the patient and explanation feedback for the explanation of the relapse probability prediction for the patient;

generate, via the user device, a change (AP) for the relapse probability prediction for the patient as the prediction feedback and a change (AE) for the explanation of the relapse probability prediction for the patient as the explanation feedback;

determine coherency between the prediction feedback and the explanation feedback on the basis of |P o +dT+dN+dM−P a |<c,

wherein P o is the prediction, P a is the prediction feedback, dT is a change in a T stage feature, dN is a change in a N stage feature, dM is a change in a M stage feature and c is a correction;

receive entries of predictions, prediction feedback and explanation feedback from other users from an interaction log of the other users;

compute an agreement between the prediction feedback of the user and the prediction feedbacks received from the other users and an agreement between the explanation feedback of the user and the explanation feedback received from the other users based on the determined coherency between the prediction feedback and the explanation feedback of the user;

determine that the agreement is less than or equal to a quorum threshold;

determine that the agreement is achieved between the prediction feedback and the explanation feedback based on determination that the agreement is less than or equal to the quorum threshold;

update the machine learning model based on the determination that the agreement is being achieved between the prediction feedback and the explanation feedback, to generate an updated machine learning model;

cryptographically protect the updated machine learning model to generate an updated and cryptographically protected machine learning model by,

generating a hash based on the prediction feedback and the explanation feedback, and

generating a block for the updated machine learning model based on the hash,

wherein the block is part of a block chain and corresponds to the updated and cryptographically protected machine learning model;

perform one or more actions based on the updated and cryptographically protected machine learning model; and

implement the updated and cryptographically protected machine learning model to make future predictions associated with the relapse probability of the patient to expedite treatment of the patient.

16 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to calculate the agreement level between the prediction feedback and the explanation feedback, cause the device to:

calculate the agreement level between the prediction feedback and the explanation feedback based on weights assigned to domain experts providing the prediction feedback and the explanation feedback.

17 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to determine that the agreement is achieved between the prediction feedback and the explanation feedback based on the threshold, cause the device to:

determine that the agreement is achieved between the prediction feedback and the explanation feedback based on weights assigned to domain experts providing the prediction feedback and the explanation feedback.

18 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to update the machine learning model, cause the device to:

update the machine learning model, to generate the updated machine learning model, based on the prediction feedback and the explanation feedback.

19 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to determine the agreement is achieved between the prediction feedback and the explanation feedback based on an average corrected prediction, an average of explanation corrections, and the quorum threshold.

20 . The non-transitory computer-readable medium of claim 15 , wherein the one or more instructions, that cause the device to update the machine learning model, cause the device to:

encrypt the interaction log with an updated prediction, updated explanation, an original prediction, and an original explanation.