Confidence level for agent-generated search analytics

1 . A computer-implemented method, including:

inducing an agent-under-test (AUT) to disclose respective outputs in response to processing a target input probe, wherein the AUT comprises an encoder and a decoder, wherein the encoder receives the target input probe and encodes the target input probe into a context vector, and wherein the decoder, conditioned on the context vector generated by the encoder, generates the respective outputs;

analyzing the respective outputs, generated by the decoder conditioned on the context vector, to determine whether the respective outputs represent a sample for the target input probe, and to determine a confidence level of the respective outputs being the sample for the target input probe; and

analyzing the respective outputs, generated by the decoder conditioned on the context vector, to generate analytics for the sample.

2 . The computer-implemented method of claim 1 , wherein the AUT is an artificial intelligence (AI) system.

3 . The computer-implemented method of claim 2 , wherein the AUT is a large language model (LLM).

4 . The computer-implemented method of claim 1 , wherein the confidence level reflects a reliability index indicating distributional completeness of the respective outputs through one or more statistical models.

5 . The computer-implemented method of claim 1 , further including inducing the AUT at periodic intervals.

6 . The computer-implemented method of claim 5 , further including for each of the periodic intervals, determining whether the respective outputs represent the sample for the target input probe, and/or determining the confidence level of the respective outputs representing the sample for the target input probe.

7 . The computer-implemented method of claim 5 , wherein the periodic intervals are second-wise.

8 . The computer-implemented method of claim 5 , wherein the periodic intervals are minute-wise, hour-wise, day-wise, week-wise, month-wise, and/or year-wise.

9 . The computer-implemented method of claim 5 , wherein the periodic intervals are retrospective and apply to the respective outputs disclosed in prior time periods.

10 . The computer-implemented method of claim 1 , wherein an inducing agent sends the target input probe to the AUT.

11 . The computer-implemented method of claim 1 , wherein the target input probe is a prompt to the AUT.

12 . The computer-implemented method of claim 11 , wherein the respective outputs are answers to the target input probe.

13 . The computer-implemented method of claim 1 , wherein the analyzing the respective outputs to generate the analytics for the sample further includes determining a share of voice percentage of features associated with the target input probe, a share of voice percentile of the features associated with the target input probe, and a share of voice proportion of the features associated with the target input probe.

14 . The computer-implemented method of claim 1 , wherein the analyzing the respective outputs to generate the analytics for the sample further includes performing sentiment analysis, bias detection, topics identification, source identification, and/or inaccuracy detection.

15 . The computer-implemented method of claim 1 , further including displaying the respective outputs for the target input probe.

16 . The computer-implemented method of claim 1 , further including displaying the determined confidence level of the respective outputs being the sample for the target input probe.

17 . The computer-implemented method of claim 1 , further including determining the confidence level for the generated analytics.

18 . The computer-implemented method of claim 17 , further including displaying the determined confidence level for the generated analytics.