Method and system for evaluating effectiveness of prompt

1 . A computer-implemented method comprising:

receiving input data for generating a prompt for receiving a response using a large language model (LLM);

generating, based on the input data, a plurality of sub-prompts, wherein each sub-prompt of the plurality of sub-prompts corresponds with the prompt for receiving the response using the LLM; and

performing a series of operations for a sub-prompt of the plurality of sub-prompts, the series of operations including:

generating a sub-response corresponding to the sub-prompt using the LLM;

converting the sub-response to a respective matrix of a plurality of matrices, based on Latent Semantic Analysis (LSA);

evaluating each evaluation aspect of a plurality of evaluation aspects corresponding to the sub-response based on the respective matrix of the plurality of matrices to measure an effectiveness of the sub-prompt in generating the response, wherein evaluation of each evaluation aspect of the plurality of evaluation aspects comprises:

computing a score value corresponding to each respective matrix of the plurality of matrices;

computing, based on the score value corresponding to each respective matrix of the plurality of matrices, a combined score value corresponding to each evaluation aspect of the plurality evaluation aspects, wherein computing the combined score value corresponding to each evaluation aspect comprises:

computing, based on the sub-response and the input data, a performance improvement factor value for the sub-prompt, wherein the performance improvement factor value is based upon a keyword score value and a style similarity score value;

computing, based on the respective score value computed corresponding to each evaluation aspect and the performance improvement factor value, the combined score value corresponding to each evaluation aspect;

wherein computing the performance improvement factor value for the sub-prompt comprises:

 analyzing the sub-response to determine:

 presence of a plurality of preferred keywords, wherein the preferred keywords are specified by a user in the input data; and

 similarity of the sub-response to a preferred response style, wherein the preferred response style is specified by the user in the input data; and

 generating, based upon the analyzing, the keyword score value and the style similarity score value;

computing, based on the combined score value, an optimal score value corresponding to the sub-response corresponding to the evaluation aspect; and

determining whether the optimal score value corresponding to the sub-response exceeds a predetermined threshold value;

determining, based upon the optimal score exceeding the predetermined threshold value, the sub-prompt as an effective prompt for generating the response using the LLM corresponding to the input data; and

generating, based upon the optimal score is less than the predetermined threshold value, feedback to refine the sub-prompt of the plurality of sub-prompts, wherein the sub-prompt is refined by refining coverage, structure, and occurrence of the plurality of preferred keywords based on the feedback generated for the sub-prompt.

2 . The computer-implemented method of claim 1 , further comprising repeating the series of operations for another sub-prompt of the plurality of sub-prompts upon determining the optimal score value fails to exceed the predetermined threshold value.

3 . The computer-implemented method of claim 1 , wherein the keyword score value is generated based upon a respective weightage assigned to each preferred keyword of the plurality of preferred keywords.

4 . The computer-implemented method of claim 1 , wherein the preferred response style includes one of a professional response style, a creative response style, or a friendly response style.

5 . The computer-implemented method of claim 1 , wherein the plurality of evaluation aspects comprises clarity, relevance, hallucination, domain knowledge, and/or consistency.

6 . The computer-implemented method of claim 1 , wherein the input data includes a plurality of preferred keywords, a respective weightage for each preferred keyword of the plurality of preferred keywords, and query information.

7 . The computer-implemented method of claim 6 , wherein the input data further includes a preferred response style.

8 . A system comprising:

at least one memory configured to store machine executable instructions; and

at least one processor communicatively coupled with the at least one memory, and configured to execute the machine executable instructions to:

receive input data for generating a prompt for receiving a response using a large language model (LLM);

generate, based on the input data, a plurality of sub-prompts, wherein each sub-prompt of the plurality of sub-prompts corresponds with the prompt for receiving the response using the LLM; and

perform a series of operations for a sub-prompt of the plurality of sub-prompts, the series of operations including:

generating a sub-response corresponding to the sub-prompt using the LLM;

converting the sub-response to a respective matrix of a plurality of matrices, based on Latent Semantic Analysis (LSA);

evaluating each evaluation aspect of a plurality of evaluation aspects corresponding to the sub-response based on the respective matrix of the plurality of matrices to measure an effectiveness of the sub-prompt in generating the response, wherein evaluation of each evaluation aspect of the plurality of evaluation aspects comprises:

computing a score value corresponding to each respective matrix of the plurality of matrices;

computing, based on the score value corresponding to each respective matrix of the plurality of matrices, a combined score value corresponding to each evaluation aspect of the plurality of evaluation aspects, wherein computing the combined score value corresponding to each evaluation aspect comprises:

 computing, based on the sub-response and the input data, a performance improvement factor value for the sub-prompt, wherein the performance improvement factor value is based upon a keyword score value and a style similarity score value;

 computing, based on the respective score value computed corresponding to each evaluation aspect and the performance improvement factor value, the combined score value corresponding to each evaluation aspect, wherein computing the performance improvement factor value for the sub-prompt comprises:

 analyzing the sub-response to determine:

 presence of a plurality of preferred keywords, wherein the preferred keywords are specified by a user in the input data; and

 similarity of the sub-response to a preferred response style, wherein the preferred response style is specified by the user in the input data; and

 generating, based upon the analyzing, the keyword score value and the style similarity score value;

computing, based on the combined score value, an optimal score value corresponding to the sub-response corresponding to the evaluation aspect; and

determining whether the optimal score value corresponding to the sub-response exceeds a predetermined threshold value;

determining, based upon the optimal score exceeding the predetermined threshold value, the sub-prompt as an effective prompt for generating the response using the LLM corresponding to the input data; and

generating, based upon the optimal score is less than the predetermined threshold value, feedback to refine the sub-prompt of the plurality of sub-prompts, wherein the sub-prompt is refined by refining coverage, structure, and occurrence of the plurality of preferred keywords based on the feedback generated for the sub-prompt.

9 . The system of claim 8 , wherein the at least one processor is further configured to execute the machine executable instructions to repeat the series of operations for another sub-prompt of the plurality of sub-prompts upon determining the optimal score value fails to exceed the predetermined threshold value.

10 . The system of claim 8 , wherein the keyword score value is generated based upon a respective weightage assigned to each preferred keyword of the plurality of preferred keywords.

11 . The system of claim 8 , wherein the preferred response style includes one of a professional response style, a creative response style, or a friendly response style; and wherein the plurality of evaluation aspects comprises clarity, relevance, hallucination, domain knowledge, and/or consistency.

12 . The system of claim 8 , wherein the input data includes a plurality of preferred keywords, a respective weightage for each preferred keyword of the plurality of preferred keywords, and a query information.

13 . The system of claim 12 , wherein the input data further includes a preferred response style.

14 . A non-transitory computer-readable media (CRM) for storing machine executable instructions stored thereon, which, when executable by at least one processor of a computing device, cause the computing device to execute the machine executable instructions to:

receive input data for generating a prompt for receiving a response using a large language model (LLM);

generate, based on the input data, a plurality of sub-prompts, wherein each sub-prompt of the plurality of sub-prompts corresponds with the prompt for receiving the response using the LLM; and

perform a series of operations for a sub-prompt of the plurality of sub-prompts, the series of operations including:

generating a sub-response corresponding to the sub-prompt using the LLM;

converting the sub-response to a respective matrix of a plurality of matrices, based on Latent Semantic Analysis (LSA);

evaluating each evaluation aspect of a plurality of evaluation aspects corresponding to the sub-response based on the respective matrix of the plurality of matrices to measure an effectiveness of the sub-prompt in generating the response, wherein evaluation of each evaluation aspect of the plurality of evaluation aspects comprises:

computing a score value corresponding to each respective matrix of the plurality of matrices;

computing, based on the score value corresponding to each respective matrix of the plurality of matrices, a combined score value corresponding to of each evaluation aspect of the plurality of evaluation aspects, wherein computing the combined score value corresponding to each evaluation aspect comprises:

computing, based on the sub-response and the input data, a performance improvement factor value for the sub-prompt, wherein the performance improvement factor value is based upon a keyword score value and a style similarity score value;

computing, based on the respective score value computed corresponding to each evaluation aspect and the performance improvement factor value, the combined score value corresponding to each evaluation aspect, wherein computing the performance improvement factor value for the sub-prompt comprises:

 analyzing the sub-response to determine:

 presence of a plurality of preferred keywords, wherein the preferred keywords are specified by a user in the input data; and

 similarity of the sub-response to a preferred response style, wherein the preferred response style is specified by the user in the input data; and

 generating, based upon the analyzing, the keyword score value and the style similarity score value;

computing, based on the combined score value for evaluation of each evaluation aspect, an optimal score value corresponding to the sub-response; and

determining whether the optimal score value corresponding to the sub-response exceeds a predetermined threshold value;

determining, based upon the optimal score exceeding the predetermined threshold value, the sub-prompt as an effective prompt for generating the response using the LLM corresponding to the input data; and

generating, based upon the optimal score is less than the predetermined threshold value, feedback to refine the sub-prompt of the plurality of sub-prompts, wherein the sub-prompt is refined by refining coverage, structure, and occurrence of the plurality of preferred keywords based on the feedback generated for the sub-prompt.