IP Library › Granted Patent US 12,008,319
Granted Patent B2
US 12,008,319 · App. 17/613,506 · Granted Jun 11, 2024

Method and apparatus for selecting answers to idiom fill-in-the-blank questions, and computer device

Inventors: Xiang Liu (Guangdong, CN); Xiuling Chen (Guangdong, CN)
Assignee: PING AN TECHNOLOGY (SHENZHEN) CO., LTD.
G06F40/289
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,008,319
App. No.
17/613,506
Granted
Jun 11, 2024
Kind
B2
Abstract

Disclosed are a method and apparatus for selecting answers to idiom fill-in-the-blank questions, a computer device, and a storage medium. The method includes: obtaining a question text of idiom fill-in-the-blank questions, the question text including a fill-in-the-blank text and n candidate idioms, and the fill-in-the-blank text including m fill-in-the-blanks to be filled in with the candidate idioms; obtaining an explanatory text of all the candidate idioms; obtaining, through an idiom selection fill-in-the-blank model, a confidence that each fill-in-the-blank is filled in with each candidate idiom; selecting m idioms from the n candidate idioms to form multiple groups of answers; calculating a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers; and obtaining a group of answers with the highest confidence sum as answers to the idiom fill-in-the-blank questions. The present application implements answers to idiom fill-in-the-blank questions with high accuracy.

Claims (84)

1. A method for selecting answers to idiom fill-in-the-blank questions, comprising the following steps:

obtaining a question text of idiom fill-in-the-blank questions, wherein the question text comprises a fill-in-the-blank text and n candidate idioms, and the fill-in-the-blank text comprises m fill-in-the-blanks to be filled in with the candidate idioms;

obtaining an explanatory text of all the candidate idioms;

inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms;

selecting m idioms from the n candidate idioms, and randomly arranging the m idioms to fill in the m fill-in-the-blanks to form a plurality of groups of answers, wherein in each group of answers, the candidate idioms can be selected to fill in the fill-in-the-blanks only once at most;

calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers; and

obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions.

2. The method for selecting answers to idiom fill-in-the-blank questions according to claim 1 , before the step of inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, comprising:

obtaining training samples, wherein the training samples comprise a plurality of corpus texts of idiom fill-in-the-blank questions with selected answers and an explanatory text of all idioms in an idiom library; and

inputting the training samples into a BERT language model-based natural language neural network for training to obtain the idiom selection fill-in-the-blank model, wherein the natural language neural network comprises a network output layer and a convolutional layer, the convolutional layer is formed by stacking a plurality of convolutional networks in sequence, an input layer of a latter convolutional network is connected to output layers of all former convolutional networks, an output layer of each of the convolutional networks is connected to an output layer of the BERT language model, and an output layer of the last convolutional network is connected to the network output layer.

3. The method for selecting answers to idiom fill-in-the-blank questions according to claim 2 , wherein the step of inputting the training samples into a BERT language model-based natural language neural network for training to obtain the idiom selection fill-in-the-blank model comprises:

inputting the training samples into the BERT language model, and extracting a feature matrix of the training samples based on the BERT language model; and

inputting the feature matrix into the natural language neural network for iterative training to obtain the idiom selection fill-in-the-blank model.

4. The method for selecting answers to idiom fill-in-the-blank questions according to claim 1 , before the step of calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers, comprising:

separately determining whether there is at least one candidate idiom in each group of answers for which the confidence that each of the fill-in-the-blanks is filled in with the at least one candidate idiom is less than a first threshold; and

if yes, eliminating corresponding answers, and skipping calculating a sum of the confidences that each of the fill-in-the-blanks is filled in with the at least one candidate idiom in the eliminated answers.

5. The method for selecting answers to idiom fill-in-the-blank questions according to claim 1 , after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, further comprising:

filling in the fill-in-the-blank text with the target answers to obtain an answer text;

obtaining a score for the answer text given by a user, and determining whether the score is greater than a second threshold, wherein the user scores the answer text based on the standard answers;

if the score is greater than the second threshold, forming a training set from the answer text and an explanatory text of all candidate idioms of the target answers, and inputting the training set into the idiom selection fill-in-the-blank model for retraining; and

saving the retrained idiom selection fill-in-the-blank model to a blockchain.

6. The method for selecting answers to idiom fill-in-the-blank questions according to claim 1 , wherein the idiom fill-in-the-blank questions are used to test a student; and

after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, the method further comprises:

obtaining a test text uploaded by a terminal of the student, wherein the test text is a text in which the student fills in the fill-in-the-blank text of the idiom fill-in-the-blank questions with student answers; and when the terminal of the student uploads the test text, sequentially splicing the student answers with which the idiom fill-in-the-blank questions are filled in to obtain a first idiom combination, performing hash calculation on the first idiom combination based on the blockchain to obtain a corresponding hash value, and adding the hash value as an identification code to the test text;

extracting all student answers in the test text, and sequentially splicing all the student answers to obtain a second idiom combination;

performing hash calculation on the second idiom combination based on the blockchain to obtain a corresponding verification hash value;

identifying the identification code in the test text, and verifying whether the verification hash value is consistent with the identification code; and

if yes, comparing the student answers with the target answers one by one to obtain accuracy of the student answers.

7. A computer device, comprising a memory and a processor, wherein a computer program is stored in the memory; and when the processor executes the computer program, the computer device implements a method for selecting answers to idiom fill-in-the-blank questions by:

obtaining a question text of idiom fill-in-the-blank questions, wherein the question text comprises a fill-in-the-blank text and n candidate idioms, and the fill-in-the-blank text comprises m fill-in-the-blanks to be filled in with the candidate idioms;

obtaining an explanatory text of all the candidate idioms;

inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms;

selecting m idioms from the n candidate idioms, and randomly arranging the m idioms to fill in the m fill-in-the-blanks to form a plurality of groups of answers, wherein in each group of answers, the candidate idioms can be selected to fill in the fill-in-the-blanks only once at most;

calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers; and

obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions.

8. The computer device according to claim 7 , wherein before the step of inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, the computer device is configured to:

obtain training samples, wherein the training samples comprise a plurality of corpus texts of idiom fill-in-the-blank questions with selected answers and an explanatory text of all idioms in an idiom library; and

input the training samples into a BERT language model-based natural language neural network for training to obtain the idiom selection fill-in-the-blank model, wherein the natural language neural network comprises a network output layer and a convolutional layer, the convolutional layer is formed by stacking a plurality of convolutional networks in sequence, an input layer of a latter convolutional network is connected to output layers of all former convolutional networks, an output layer of each of the convolutional networks is connected to an output layer of the BERT language model, and an output layer of the last convolutional network is connected to the network output layer.

9. The computer device according to claim 8 , wherein the computer device is specifically configured to:

input the training samples into the BERT language model, and extract a feature matrix of the training samples based on the BERT language model; and

input the feature matrix into the natural language neural network for iterative training to obtain the idiom selection fill-in-the-blank model.

10. The computer device according to claim 7 , wherein before the step of calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers, the computer device is configured to:

separately determine whether there is at least one candidate idiom in each group of answers for which the confidence that each of the fill-in-the-blanks is filled in with the at least one candidate idiom is less than a first threshold; and

if yes, eliminate corresponding answers, and skip calculating a sum of the confidences that each of the fill-in-the-blanks is filled in with the at least one candidate idiom in the eliminated answers.

11. The computer device according to claim 7 , wherein after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, the computer device is further configured to:

fill in the fill-in-the-blank text with the target answers to obtain an answer text;

obtain a score for the answer text given by a user, and determine whether the score is greater than a second threshold, wherein the user scores the answer text based on the standard answers;

if the score is greater than the second threshold, form a training set from the answer text and an explanatory text of all candidate idioms of the target answers, and input the training set into the idiom selection fill-in-the-blank model for retraining; and

save the retrained idiom selection fill-in-the-blank model to a blockchain.

12. The computer device according to claim 7 , wherein the idiom fill-in-the-blank questions are used to test a student; and

after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, the computer device is further configured to:

obtain a test text uploaded by a terminal of the student, wherein the test text is a text in which the student fills in the fill-in-the-blank text of the idiom fill-in-the-blank questions with student answers; and when the terminal of the student uploads the test text, sequentially splice the student answers with which the idiom fill-in-the-blank questions are filled in to obtain a first idiom combination, perform hash calculation on the first idiom combination based on the blockchain to obtain a corresponding hash value, and add the hash value as an identification code to the test text;

extract all student answers in the test text, and sequentially splice all the student answers to obtain a second idiom combination;

perform hash calculation on the second idiom combination based on the blockchain to obtain a corresponding verification hash value;

identify the identification code in the test text, and verify whether the verification hash value is consistent with the identification code; and

if yes, compare the student answers with the target answers one by one to obtain accuracy of the student answers.

13. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, a method for selecting answers to idiom fill-in-the-blank questions is implemented, and the method comprises the following steps:

obtaining a question text of idiom fill-in-the-blank questions, wherein the question text comprises a fill-in-the-blank text and n candidate idioms, and the fill-in-the-blank text comprises m fill-in-the-blanks to be filled in with the candidate idioms;

obtaining an explanatory text of all the candidate idioms;

inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms;

selecting m idioms from the n candidate idioms, and randomly arranging the m idioms to fill in the m fill-in-the-blanks to form a plurality of groups of answers, wherein in each group of answers, the candidate idioms can be selected to fill in the fill-in-the-blanks only once at most;

calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers; and

obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions.

14. The non-transitory computer-readable storage medium according to claim 13 , wherein before the step of inputting the fill-in-the-blank text and the explanatory text into a pre-trained idiom selection fill-in-the-blank model, to obtain a confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, the method comprises:

obtaining training samples, wherein the training samples comprise a plurality of corpus texts of idiom fill-in-the-blank questions with selected answers and an explanatory text of all idioms in an idiom library; and

inputting the training samples into a BERT language model-based natural language neural network for training to obtain the idiom selection fill-in-the-blank model, wherein the natural language neural network comprises a network output layer and a convolutional layer, the convolutional layer is formed by stacking a plurality of convolutional networks in sequence, an input layer of a latter convolutional network is connected to output layers of all former convolutional networks, an output layer of each of the convolutional networks is connected to an output layer of the BERT language model, and an output layer of the last convolutional network is connected to the network output layer.

15. The non-transitory computer-readable storage medium according to claim 14 , wherein the step of inputting the training samples into a BERT language model-based natural language neural network for training to obtain the idiom selection fill-in-the-blank model comprises:

inputting the training samples into the BERT language model, and extracting a feature matrix of the training samples based on the BERT language model; and

inputting the feature matrix into the natural language neural network for iterative training to obtain the idiom selection fill-in-the-blank model.

16. The non-transitory computer-readable storage medium according to claim 13 , wherein before the step of calculating, based on a KM algorithm and the confidence that each of the fill-in-the-blanks is filled in with each of the candidate idioms, a sum of the confidences that the fill-in-the-blanks are filled in with the candidate idioms in each group of answers, the method comprises:

separately determining whether there is at least one candidate idiom in each group of answers for which the confidence that each of the fill-in-the-blanks is filled in with the at least one candidate idiom is less than a first threshold; and

if yes, eliminating corresponding answers, and skipping calculating a sum of the confidences that each of the fill-in-the-blanks is filled in with the at least one candidate idiom in the eliminated answers.

17. The non-transitory computer-readable storage medium according to claim 13 , wherein after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, the method further comprises:

filling in the fill-in-the-blank text with the target answers to obtain an answer text;

obtaining a score for the answer text given by a user, and determining whether the score is greater than a second threshold, wherein the user scores the answer text based on the standard answers;

if the score is greater than the second threshold, forming a training set from the answer text and an explanatory text of all candidate idioms of the target answers, and inputting the training set into the idiom selection fill-in-the-blank model for retraining; and

saving the retrained idiom selection fill-in-the-blank model to a blockchain.

18. The non-transitory computer-readable storage medium according to claim 13 , wherein the idiom fill-in-the-blank questions are used to test a student; and

after the step of obtaining a group of answers with the highest confidence sum as target answers, and outputting the target answers as answers to the idiom fill-in-the-blank questions, the method further comprises:

obtaining a test text uploaded by a terminal of the student, wherein the test text is a text in which the student fills in the fill-in-the-blank text of the idiom fill-in-the-blank questions with student answers; and when the terminal of the student uploads the test text, sequentially splicing the student answers with which the idiom fill-in-the-blank questions are filled in to obtain a first idiom combination, performing hash calculation on the first idiom combination based on the blockchain to obtain a corresponding hash value, and adding the hash value as an identification code to the test text;

extracting all student answers in the test text, and sequentially splicing all the student answers to obtain a second idiom combination;

performing hash calculation on the second idiom combination based on the blockchain to obtain a corresponding verification hash value;

identifying the identification code in the test text, and verifying whether the verification hash value is consistent with the identification code; and

if yes, comparing the student answers with the target answers one by one to obtain accuracy of the student answers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 23, 2021
From: LIU, XIANG; CHEN, XIULING
To: PING AN TECHNOLOGY (SHENZHEN) CO., LTD.
Reel/Frame 058200/0706 →
Priority Claims (1)
CN 202010923909.X · Sep 4, 2020 · national
Continuity (1)
Related Publication 20220261546A1 · Aug 18, 2022