Structured query language evaluation method, electronic device and storage medium

1 . A structured query language evaluation method, comprising:

obtaining a query question input by a user;

generating a predicted structured query language (SQL) corresponding to the query question based on the query question by using a large language model;

determining whether there is a SQL corresponding to the query question, wherein the SQL is pre-defined and directly corresponds to the query question;

in presence of the SQL corresponding to the query question:

obtaining a first execution result corresponding to the predicted SQL;

obtaining a second execution result corresponding to the SQL;

comparing the predicted SQL and the SQL to check consistency of the predicted SQL and the SQL in terms of structure and syntax rule, to obtain a first comparison result;

verifying whether the first execution result and the second execution result return same or logically equivalent data, to obtain a second comparison result;

obtaining first preliminary evaluation results based on the first comparison result and the second comparison result, the first preliminary evaluation results comprehensively evaluating syntax correctness of the predicted SQL and consistency of execution results of the predicted SQL and SQL;

obtaining a quantity of correct evaluations in the first preliminary evaluation results;

calculating a ratio between the quantity of the correct evaluations and a total quantity of the first preliminary evaluation results; and

taking the ratio as the evaluation result corresponding to the predicted SQL; and

in absence of the SQL:

obtaining a third execution result corresponding to the predicted SQL;

performing semantic analysis on the predicted SQL to obtain a semantic analysis result; and

evaluating an accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL.

2 . The method of claim 1 , wherein evaluating the accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL comprises:

obtaining target semantic features needed by the user;

matching semantic features in the semantic analysis result with the target semantic features, and performing evaluation by using the third execution result, to obtain second preliminary evaluation results; and

obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results.

3 . The method of claim 2 , wherein obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results comprises:

obtaining a quantity of correct evaluations in the second preliminary evaluation results; and

calculating a ratio between the quantity of the correct evaluations and a total quantity of the second preliminary evaluation results, and taking the ratio as the evaluation result corresponding to the predicted SQL.

4 . An electronic device, comprising:

at least one processor; and

a memory connected in communication with the at least one processor;

wherein the memory stores an instruction executable by the at least one processor, and the instruction, when executed by the at least one processor, enables the at least one processor to execute:

obtaining a query question input by a user;

generating a predicted structured query language (SQL) corresponding to the query question based on the query question by using a large language model;

determining whether there is a SQL corresponding to the query question, wherein the SQL is pre-defined and directly corresponds to the query question;

in presence of the SQL corresponding to the query question:

obtaining a first execution result corresponding to the predicted SQL;

obtaining a second execution result corresponding to the SQL;

comparing the predicted SQL and the SQL to check consistency of the predicted SQL and the SQL in terms of structure and syntax rule, to obtain a first comparison result;

verifying whether the first execution result and the second execution result return same or logically equivalent data, to obtain a second comparison result;

obtaining first preliminary evaluation results based on the first comparison result and the second comparison result, the first preliminary evaluation results comprehensively evaluating syntax correctness of the predicted SQL and consistency of execution results of the predicted SQL and SQL;

obtaining a quantity of correct evaluations in the first preliminary evaluation results;

calculating a ratio between the quantity of the correct evaluations and a total quantity of the first preliminary evaluation results; and

taking the ratio as the evaluation result corresponding to the predicted SQL; and

in absence of the SQL:

obtaining a third execution result corresponding to the predicted SQL;

performing semantic analysis on the predicted SQL to obtain a semantic analysis result; and

evaluating the accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL.

5 . The electronic device of claim 4 , wherein evaluating the accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL comprises:

obtaining target semantic features needed by the user;

matching semantic features in the semantic analysis result with the target semantic features, and performing evaluation by using the third execution result, to obtain second preliminary evaluation results; and

obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results.

6 . The electronic device of claim 5 , wherein obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results comprises:

obtaining a quantity of correct evaluations in the second preliminary evaluation results; and

calculating a ratio between the quantity of the correct evaluations and a total quantity of the second preliminary evaluation results, and taking the ratio as the evaluation result corresponding to the predicted SQL.

7 . A non-transitory computer-readable storage medium storing a computer instruction thereon, wherein the computer instruction is used to cause a computer to execute:

obtaining a query question input by a user;

generating a predicted structured query language (SQL) corresponding to the query question based on the query question by using a large language model;

determining whether there is a SQL corresponding to the query question, wherein the SQL is pre-defined and directly corresponds to the query question;

in presence of the SQL corresponding to the query question:

obtaining a first execution result corresponding to the predicted SQL;

obtaining a second execution result corresponding to the SQL;

comparing the predicted SQL and the SQL to check consistency of the predicted SQL and the SQL in terms of structure and syntax rule, to obtain a first comparison result;

verifying whether the first execution result and the second execution result return same or logically equivalent data, to obtain a second comparison result;

obtaining first preliminary evaluation results based on the first comparison result and the second comparison result, the first preliminary evaluation results comprehensively evaluating syntax correctness of the predicted SQL and consistency of execution results of the predicted SQL and SQL;

obtaining a quantity of correct evaluations in the first preliminary evaluation results;

calculating a ratio between the quantity of the correct evaluations and a total quantity of the first preliminary evaluation results; and

taking the ratio as the evaluation result corresponding to the predicted SQL; and

in absence of the SQL:

obtaining a third execution result corresponding to the predicted SQL;

performing semantic analysis on the predicted SQL to obtain a semantic analysis result; and

evaluating the accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL.

8 . The non-transitory computer-readable storage medium of claim 7 , wherein evaluating the accuracy of the predicted SQL based on the third execution result and the semantic analysis result to obtain the evaluation result corresponding to the predicted SQL comprises:

obtaining target semantic features needed by the user;

matching semantic features in the semantic analysis result with the target semantic features, and performing evaluation by using the third execution result, to obtain second preliminary evaluation results; and

obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results.

9 . The non-transitory computer-readable storage medium of claim 8 , wherein obtaining the evaluation result corresponding to the predicted SQL based on the second preliminary evaluation results comprises:

obtaining a quantity of correct evaluations in the second preliminary evaluation results; and

calculating a ratio between the quantity of the correct evaluations and a total quantity of the second preliminary evaluation results, and taking the ratio as the evaluation result corresponding to the predicted SQL.