Extending question and answer samples
Implementations of the present specification provide a method and an apparatus for extending question and answer samples. According to the method, a random number is generated for each existing sample, a question is blurred for a sample whose random number belongs to sample extension random numbers, to generate an extended sample, so that an overall sample blurring extension rate can be effectively controlled. In addition, for a sample needing blurring extension, a question is extended by deleting a word with a predetermined part of speech in the corresponding question, and then an extended sample is generated based on an extended question, so that more question expression ways are compatible. As such, a question and answer model is trained by using a sample set to which extended samples are added, so that an answer can be provided to a user more effectively.
1. A computer-implemented method for extending question and answer samples, wherein the question and answer samples comprise a first sample, the first sample comprises a first question and a pre-marked first answer, and the method comprises:
performing syntax analysis on the first question to determine a part of speech corresponding to each word in the first question including a first word corresponding to a first part of speech;
generating a random number within a predetermined value range for the first sample, and detecting whether the generated random number belongs to selected sample extension random numbers, wherein the selected sample extension random numbers are values selected from the predetermined value range and are used to select a corresponding sample from the question and answer samples for extension;
responsive to detecting the generated random number belongs to the selected sample extension random numbers, deleting the first word corresponding to the first part of speech to generate one or more extended questions;
in response to generating the one or more extended questions, generating an extended sample based on the one or more extended questions and the pre-marked first answer;
training a question and answer model using the extended sample; and
generating output of the trained question and answer model to provide a response to a user question.
2. The computer-implemented method of claim 1 , wherein a proportion of a quantity of the selected sample extension random numbers to a quantity of values within the predetermined value range is predetermined.
3. The computer-implemented method of claim 1 , wherein deleting the first word corresponding to the first part of speech comprises:
deleting any one of words with the first part of speech from the first question.
4. The computer-implemented method of claim 1 , wherein the first part of speech comprises at least one of the following: a noun, a verb, a verb phrase, and an interrogative.
5. The computer-implemented method of claim 1 , wherein the first part of speech comprises one of a noun and an interrogative.
6. The computer-implemented method of claim 1 , wherein the generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
combining each extended question of the one or more extended questions with the pre-marked first answer to generate the extended sample.
7. The computer-implemented method of claim 1 , wherein questions extended from the first question comprise a first extended question, and generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
responsive to detecting an extended question within the question and answer samples that matches the first extended question, marking a first extended answer for the first extended question based on a predetermined template to generate a first extended sample.
8. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations for extending question and answer samples, wherein the question and answer samples comprise a first sample, the first sample comprises a first question and a pre-marked first answer, and the operations comprise:
performing syntax analysis on the first question to determine a part of speech corresponding to each word in the first question including a first word corresponding to a first part of speech;
generating a random number within a predetermined value range for the first sample, and detecting whether the generated random number belongs to selected sample extension random numbers, wherein the selected sample extension random numbers are values selected from the predetermined value range and are used to select a corresponding sample from the question and answer samples for extension;
responsive to detecting the generated random number belongs to the selected sample extension random numbers, deleting the first word corresponding to the first part of speech to generate one or more extended questions;
in response to generating the one or more extended questions, generating an extended sample based on the one or more extended questions and the pre-marked first answer;
training a question and answer model using the extended sample; and
generating output of the trained question and answer model to provide a response to a user question.
9. The non-transitory, computer-readable medium of claim 8 , wherein a proportion of a quantity of the selected sample extension random numbers to a quantity of values within the predetermined value range is predetermined.
10. The non-transitory, computer-readable medium of claim 8 , wherein deleting the first word corresponding to the first part of speech comprises:
deleting any one of words with the first part of speech from the first question.
11. The non-transitory, computer-readable medium of claim 8 , wherein the first part of speech comprises at least one of the following: a noun, a verb, a verb phrase, and an interrogative.
12. The non-transitory, computer-readable medium of claim 8 , wherein the first part of speech comprises one of a noun and an interrogative.
13. The non-transitory, computer-readable medium of claim 8 , wherein the generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
combining each extended question of the one or more extended questions with the pre-marked first answer to generate the extended sample.
14. The non-transitory, computer-readable medium of claim 8 , wherein questions extended from the first question comprise a first extended question, and generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
responsive to detecting an extended question within the question and answer samples that matches the first extended question, marking a first extended answer for the first extended question based on a predetermined template to generate a first extended sample.
15. A computer-implemented system for extending question and answer samples, wherein the question and answer samples comprise a first sample, the first sample comprises a first question and a pre-marked first answer, and the system comprises:
one or more computers; and
one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising:
performing syntax analysis on the first question to determine a part of speech corresponding to each word in the first question including a first word corresponding to a first part of speech;
generating a random number within a predetermined value range for the first sample, and detecting whether the generated random number belongs to selected sample extension random numbers, wherein the selected sample extension random numbers are values selected from the predetermined value range and are used to select a corresponding sample from the question and answer samples for extension;
responsive to detecting the generated random number belongs to the selected sample extension random numbers, deleting the first word corresponding to the first part of speech to generate one or more extended questions;
in response to generating the one or more extended questions, generating an extended sample based on the one or more extended questions and the pre-marked first answer;
training a question and answer model using the extended sample; and
generating output of the trained question and answer model to provide a response to a user question.
16. The computer-implemented system of claim 15 , wherein a proportion of a quantity of the selected sample extension random numbers to a quantity of values within the predetermined value range is predetermined.
17. The computer-implemented system of claim 15 , wherein deleting the first word corresponding to the first part of speech comprises:
deleting any one of words with the first part of speech from the first question.
18. The computer-implemented system of claim 15 , wherein the first part of speech comprises at least one of the following: a noun, a verb, a verb phrase, and an interrogative.
19. The computer-implemented system of claim 15 , wherein the generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
combining each extended question of the one or more extended questions with the pre-marked first answer to generate the extended sample.
20. The computer-implemented system of claim 15 , wherein questions extended from the first question comprise a first extended question, and generating the extended sample based on the one or more extended questions and the pre-marked first answer comprises:
responsive to detecting an extended question within the question and answer samples that matches the first extended question, marking a first extended answer for the first extended question based on a predetermined template to generate a first extended sample.