Method of evaluating data, training method, electronic device, and storage medium
A method of evaluating data, a method of training an evaluation model, an electronic device, and a storage medium are provided, and relate to a field of a computer technology, in particular to fields of intelligent search and deep learning technologies. The method of evaluating data includes: acquiring, in response to a request for identifying a quality of index data to be identified, target association data of a target webpage corresponding to the index data to be identified, wherein the target webpage is a webpage having an unknown web content, and the target association data indicates a quality of the target webpage corresponding to the index data to be identified; and obtaining, based on the target association data, a quality evaluation result for the index data to be identified.
1 . A computer-implemented method of evaluating data, comprising:
acquiring, in response to a request for identifying a quality of index data to be identified, target association data of a target webpage corresponding to the index data to be identified, wherein the target webpage is a webpage having an unknown web content, and the target association data indicates a quality of the target webpage corresponding to the index data to be identified;
inputting the target association data into an evaluation model to obtain a quality evaluation result for the index data to be identified; and
retaining the index data to be identified in an index database or deleting the index data to be identified from an index database based on the quality evaluation result, so as to improve quality of webpages presented by a search engine,
wherein the computer-implemented method further comprises: before inputting the target association data into an evaluation model to obtain a quality evaluation result for the index data to be identified:
determining, from the target association data, target association data of a target type; and
extracting, from the target association data of the target type, a target association feature of the target type as the target association data, and
wherein the target association data of the target type comprises user feedback data for the target webpage, and the extracting, from the target association data of the target type, a target association feature of the target type comprises:
extracting a user feedback feature from the user feedback data for the target webpage by using a feedback feature extraction model, wherein the feedback feature extraction model comprises an encoding and decoding module, a first fully connected layer, a Long Short-Term Memory network module, and a second fully connected layer connected in sequence.
2 . The method according to claim 1 , wherein the target association data of the target type further comprises link-related text data of the webpage, and the extracting, from the target association data of the target type, a target association feature of the target type comprises:
extracting a text feature from the link-related text data of the webpage by using a text feature extraction model, wherein the text feature extraction model comprises a tokenization module and a convolutional neural network module connected in sequence.
3 . The method according to claim 1 , wherein the target association data further comprises at least one selected from: attribute data related to the target webpage or attribute data of a website related to the target webpage.
4 . The method according to claim 3 , wherein,
the attribute data related to the target webpage comprises at least one selected from: a number of link to an outside of the webpage, a number of link to an inside of the webpage, or link-related text data of the webpage;
the attribute data of the website related to the target webpage comprises at least one selected from: a number of link to an inside of the website, a number of link to an outside of the website, or user feedback data for the website related to the target webpage; and
the user feedback data for the target webpage comprises at least one selected from: add-to-favorites behavior data, thumb-up behavior data, sharing behavior data, or comment data.
5 . A method of training an evaluation model implemented in the method according to claim 1 , comprising:
determining a plurality of quality problem types for index data;
acquiring, for each quality problem type of the plurality of quality problem types, training data matched with each quality problem type and a label corresponding to the training data, wherein the training data comprises target association data of a sample webpage corresponding to sample index data, the sample webpage is a webpage having an unknown web content, and the label indicates a quality of the sample index data; and
training the evaluation model by using the training data and the label so as to obtain a trained evaluation model.
6 . The method according to claim 5 , wherein the training the evaluation model by using the training data and the label so as to obtain a trained evaluation model comprises:
determining, from the training data, training data of a target type;
extracting a training feature of the target type from the training data of the target type; and
training the evaluation model by using the training feature of the target type and the label, so as to obtain the trained evaluation model.
7 . An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to implement the method of training the evaluation module according to claim 5 .
8 . The electronic device according to claim 7 , wherein the instructions are further configured to cause the at least one processor to at least:
determine, from the training data, training data of a target type;
extract a training feature of the target type from the training data of the target type; and
train the evaluation model by using the training feature of the target type and the label, so as to obtain the trained evaluation model.
9 . A non-transitory computer-readable storage medium having computer instructions therein, wherein the computer instructions are configured to cause a computer to implement the method of training the evaluation module according to claim 8 .
10 . The non-transitory computer-readable storage medium according to claim 9 , wherein the instructions are further configured to cause the computer to at least:
determine, from the training data, training data of a target type;
extract a training feature of the target type from the training data of the target type; and
train the evaluation model by using the training feature of the target type and the label, so as to obtain the trained evaluation model.
11 . An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to implement the method of evaluating data according to claim 1 .
12 . The electronic device according to claim 11 , wherein the target association data of the target type further comprises link-related text data of the webpage, and wherein the instructions are further configured to cause the at least one processor to at least:
extract a text feature from the link-related text data of the webpage by using a text feature extraction model, wherein the text feature extraction model comprises a tokenization module and a convolutional neural network module connected in sequence.
13 . A non-transitory computer-readable storage medium having computer instructions therein, wherein the computer instructions are configured to cause a computer to implement the method of evaluating data according to claim 1 .