Training method and apparatus, dialogue processing method and system, and medium

1 . A method for training a reinforcement learning model based on neural networks for a medical dialog system, performed by a processor and comprising:

acquiring unlabeled data and labeled data for training the reinforcement learning model, wherein the unlabeled data is medical dialog data, and/or the labeled data is medical case data;

generating an experience pool for training the reinforcement learning model with reference to the labeled data, based on the unlabeled data, wherein the experience pool includes one or more sequences consisting of a first state(s), an action (a), a reward score (r), and a second state (s′); and

training the reinforcement learning model using the experience pool, wherein training comprises:

making the reinforcement learning model and the environment interact using one or more sequences from the experience pool;

generating one or more new sequences during the interaction, wherein each of the one or more of the newly generated sequences include a first state(s), an action (a), a reward score (r), and a second state (s′), the reward score being related to feedback made under guidance of the labeled data and a credibility of the action;

iteratively updating the experience pool by adding the newly generated sequences to the experience pool; and

updating parameters of the reinforcement learning model through continuous simulation and iteration using the updated experience pool for subsequent reinforcement learning iterations, thereby obtaining final training results;

wherein target information extracted from the labeled data is used as a training target for training the reinforcement learning model, and wherein the experience pool is generated based on training information extracted from the unlabeled data under the guidance of the target information extracted from the labeled data, and

wherein the credibility of the action which is determined based on an occurrence probability and specificity of the action in a key information set in a medical field, and wherein occurrence probability is an occurrence probability of the action in corresponding medical dialog data, and the specificity is a specificity of the action under a specific disease.

2 . The method according to claim 1 , wherein each sequence in the experience pool represents a state transition, and wherein the second state (s′) is a state to which the environment migrates after the action (a) is applied when the environment is in the first state (s).

3 . The method according to claim 1 , wherein the action is a dialog action acquired from the medical dialog data; and wherein the first state comprise historical information in the medical dialog data which is before the acquired dialog action.

4 . A dialog processing method for a medical dialog system, performed by a processor and comprising:

acquiring dialog information;

generating reply information based on a reinforcement learning model based on neural networks; and

responding to the dialog information based on the reply information;

wherein the reinforcement learning model is trained through the following operations:

acquiring unlabeled data and labeled data for training the reinforcement learning model, wherein the unlabeled data is medical dialog data, and/or the labeled data is medical case data;

generating an experience pool for training the reinforcement learning model with reference to the labeled data, based on the unlabeled data, wherein the experience pool includes one or more sequences consisting of a first state(s), an action (a), a reward score (r), and a second state (s′); and

training the reinforcement learning model using the experience pool, wherein training comprises:

making the reinforcement learning model and the environment interact using one or more sequences from the experience pool;

generating one or more new sequences during the interaction, wherein each of the one or more of the newly generated sequences include a first state(s), an action (a), a reward score (r), and a second state (s′), the reward score being related to feedback made under guidance of the labeled data and a credibility of the action;

iteratively updating the experience pool by adding the newly generated sequences to the experience pool; and

updating parameters of the reinforcement learning model through continuous simulation and iteration using the updated experience pool for subsequent reinforcement learning iterations, thereby obtaining final training results;

wherein target information extracted from the labeled data is used as a training target for training the reinforcement learning model, and wherein the experience pool is generated based on training information extracted from the unlabeled data under the guidance of the target information extracted from the labeled data, and

wherein the credibility of the action which is determined based on an occurrence probability and specificity of the action in a key information set in a medical field, and wherein occurrence probability is an occurrence probability of the action in corresponding medical dialog data, and the specificity is a specificity of the action under a specific disease.

5 . The method according to claim 4 , wherein responding to the dialog information based on the reply information comprises:

converting the reply information into natural language and outputting the natural language.

6 . A medical dialog system, comprising:

a user interface;

a processor;

a memory, having computer program instructions stored thereon, which when executed by the processor, cause the processor to:

receive natural language input information from a user and display the natural language input information on the user interface;

display one or more questions associated with a symptom mentioned in the natural language input information on the user interface;

for each question, receive an answer responding to a question from the user, and display

the answer on the user interface, and generate and display diagnostic results for the symptom on the user interface, after question and answer operation is finished,

wherein the medical dialog system is based on a reinforcement learning model based on neural networks trained through the following operations:

acquiring unlabeled data and labeled data for training the reinforcement learning model, wherein the unlabeled data is medical dialog data, and/or the labeled data is medical case data;

generating an experience pool for training the reinforcement learning model with reference to the labeled data, based on the unlabeled data, wherein the experience pool includes one or more sequences consisting of a first state(s), an action (a), a reward score (r), and a second state (s′); and

training the reinforcement learning model using the experience pool, wherein training comprises:

making the reinforcement learning model and the environment interact using one or more sequences from the experience pool;

generating one or more new sequences during the interaction, wherein each of the one or more of the newly generated sequences include a first state(s), an action (a), a reward score (r), and a second state (s′), the reward score being related to feedback made under guidance of the labeled data and a credibility of the action;

iteratively updating the experience pool by adding the newly generated sequences to the experience pool; and

updating parameters of the reinforcement learning model through continuous simulation and iteration using the updated experience pool for subsequent reinforcement learning iterations, thereby obtaining final training results;

wherein target information extracted from the labeled data is used as a training target for training the reinforcement learning model, and wherein the experience pool is generated based on training information extracted from the unlabeled data under the guidance of the target information extracted from the labeled data, and

wherein the credibility of the action which is determined based on an occurrence probability and specificity of the action in a key information set in a medical field, and wherein occurrence probability is an occurrence probability of the action in corresponding medical dialog data, and the specificity is a specificity of the action under a specific disease.

7 . The medical dialog system according to claim 6 , wherein the diagnostic results comprise at least one of: possible disease types, symptoms of the possible disease types, recommended drugs applicable for the possible disease types, symptoms targeted by the recommended drugs, and links to learn more about the recommended drugs.

8 . The medical dialog system according to claim 7 , wherein the diagnostic results are output and displayed on the user interface in the form of natural language.

9 . The medical dialog system according to claim 6 , wherein the question comprises a plurality of options, so that the user selects one of the plurality of options as the answer.

10 . The medical dialog system according to claim 6 , wherein the instructions further cause the processor to perform a named entity recognition process on the natural language input information so as to extract symptom information.

11 . The medical dialog system according to claim 6 , wherein the medical dialog system further generates the diagnostic results based on a knowledge graph, the knowledge graph comprises nodes of M diseases and N symptoms, and corresponding relationships between various diseases and various symptoms, where M and N are integers greater than or equal to 1, and the knowledge graph further comprises recommended drugs, preventive measures, treatment schemes, and etiology for each disease.