Retrieval optimization method based on multi-level knowledge base, medium, and system
Provided are a retrieval optimization method based on a multi-level knowledge base, a medium, and a system. The method includes: hierarchically managing knowledge data to construct a multi-level knowledge base; retrieving the multi-level knowledge base according to a user query to generate an initial vector chunk set; then performing first query rewriting on the query and retrieving the multi-level knowledge base again to generate an updated vector chunk set. The strong and weak direction positioning of this retrieval can generate query questions that are both professional and comprehensive.
1 . A retrieval optimization method based on a multi-level knowledge base, comprising:
S 1 : hierarchically managing knowledge data to construct a multi-level knowledge base;
S 2 : retrieving the multi-level knowledge base according to a user query to generate an initial vector chunk set; then performing first query rewriting on the query and retrieving the multi-level knowledge base again to generate an updated vector chunk set;
S 3 : determining retrieval strength and weakness directions according to statistical results of sources of the initial vector chunk set and the updated vector chunk set in a multi-level data chunk;
S 4 : performing second query rewriting on the query according to the retrieval strength and weakness directions, and outputting a query question; wherein
the step S 3 comprises:
S 31 : determining source positions of vector chunks from the initial vector chunk set and the updated vector chunk set in the multi-level knowledge base;
S 32 : counting a number of vector chunks comprised at the source positions;
S 33 : sorting statistical results to determine the retrieval strength and weakness directions;
the step S 33 comprises:
S 331 : determining a position coefficient at each of the source positions according to the number of vector chunks comprised at the source positions;
S 332 : if the position coefficient is greater than a first set threshold or a ranking of the position coefficient is within a first set rank, determining the position coefficient as a strong retrieval direction; otherwise, determining the position coefficient as a weak retrieval direction; and
the step S 332 further comprises:
determining whether a position coefficient at a source position determined to be in the weak direction is less than a second set threshold or whether a ranking of the position coefficient is greater than a second set rank, if yes, screening out a source position corresponding to the position coefficient; wherein the second set threshold is less than the first set threshold; and the second set rank is greater than the first set rank.
2 . The retrieval optimization method according to claim 1 , wherein the step S 31 further comprises:
S 31 a : comparing the initial vector chunk set with the updated vector chunk set to determine an identical vector chunk and different vector chunks;
S 31 b : configuring a higher number coefficient for the identical vector chunk than for the different vector chunks; and
the step S 32 specifically comprises: counting a number of vector chunks comprised in the source positions based on a weight according to the number coefficient.
3 . The retrieval optimization method according to claim 2 , wherein the step S 331 comprises:
S 3311 : in the multi-level knowledge base, determining a position coefficient at each of the source positions on a first level according to a number of vector chunks comprised at the source positions on the first level; and
S 3312 : for each of source positions at a level other than the first level, determining a position coefficient at the source position at the level other than the first level according to a number of vector chunks comprised at the source position and position coefficients of superior levels of the source position.
4 . The retrieval optimization method according to claim 3 , wherein the step S 3312 further comprises:
S 33121 : for the source position at the level other than the first level, counting a sum of a number of vector chunks comprised in the same level;
S 33122 : determining a level coefficient at the source position according to a ratio of the number of vector chunks comprised at the source position to the sum of the number of vector chunks comprised at the same level; and
S 33123 : determining the position coefficient at the source position at the level other than the first level according to the number of vector chunks comprised in the source position and the position coefficients of superior levels of the source position as well as the level coefficient at the source position.
5 . The retrieval optimization method according to claim 1 , comprising:
P 1 : optimizing the query question by adopting the retrieval optimization method;
P 2 : determining a weight of the multi-level knowledge base according to the retrieval strength and weakness directions determined in the retrieval optimization method; and
P 3 : generating a final vector chunk set according to the optimized query question and the multi-level knowledge base set by the weight.
6 . A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to claim 1 is implemented.
7 . An electronic system, comprising: a memory, a processor, and a computer program that is stored in the memory and that can be run on the processor, wherein when executing the computer program, the processor implements the method according to claim 1 .