Granted Patent B1

US 12,585,709 · App. 19/195,369 · Granted Mar 24, 2026

Inventors: Sumangal Mandal (Bangalore, IN); Vaishali Gupta (Bangalore, IN); Amit Kaushal (Bangalore, IN)

Assignee: INTUIT INC.

G06F16/93G06F16/2237G06F40/131

Abstract

Claims (90)

1 . A method for preemptive entropy reduction in vector embeddings to enhance semantic search, comprising:

receiving, by at least one processor, documents and a set of validation prompts;

segmenting, by the at least one processor, the documents into chunks;

embedding, by the at least one processor, the chunks into a vector space;

querying, by the at least one processor, a large language model (LLM) with the validation prompts;

intercepting, by the at least one processor, retrieved chunks of the chunks from the vector space in response to the validation prompts;

mapping, by the at least one processor, the retrieved chunks to their positions within the documents;

calculating, by the at least one processor, document coverage for the retrieved chunks, the document coverage indicating an amount of each document covered by the retrieved chunks;

calculating, by the at least one processor, individual document entropies with respect to the chunks used in response to the validation prompts based on the document coverage;

aggregating, by the at least one processor, the individual document entropies to calculate a total system entropy;

generating, by the at least one processor, a report compiling coverage analysis and entropy calculations; and

iteratively refining, by the at least one processor, at least one of the validation prompts, document segmentation, or embedding process in response to the document coverage and the entropy in the report to reduce the entropy in the vector space until the total system entropy reaches a predetermined acceptable threshold prior to deployment of the semantic search.

2 . The method of claim 1 , wherein segmenting the documents into the chunks comprises:

analyzing document structure and content;

defining an optimal chunk size for the document structure and the content; and

preserving metadata linking chunks to a document of the documents from which the metadata linking chunks were extracted.

3 . The method of claim 1 , wherein embedding the chunks into the vector space comprises:

selecting an embedding model;

processing each chunk through the embedding model to generate vector representations; and

normalizing the vector representations.

4 . The method of claim 1 , wherein querying the LLM with the validation prompts comprises:

processing the validation prompts through the LLM to generate queries; and

sending the generated queries to a vector database to retrieve relevant chunks.

5 . The method of claim 1 , wherein calculating the document coverage comprises:

determining a percentage of each document covered by the retrieved chunks;

identifying retrieval frequency of the retrieved chunks; and

analyzing a distribution of the retrieved chunks across the documents.

6 . The method of claim 1 , wherein generating the report comprises:

generating visualizations of metrics; and

creating actionable recommendations for system refinement.

7 . The method of claim 6 , further comprising:

presenting, by the at least one processor, the generated report to domain experts;

gathering, by the at least one processor, feedback on recommendations and potential adjustments; and

identifying, by the at least one processor, cross-domain impacts and opportunities for collaboration.

8 . The method of claim 7 , wherein refining at least one of the validation prompts, the document segmentation, or the embedding process comprises:

implementing expert-approved changes to validation prompts;

adjusting document segmentation strategies; and

modifying the embedding process for gathered insights.

9 . The method of claim 8 , further comprising:

re-running the method with refined inputs;

comparing, by the at least one processor, new results with previous iterations;

assessing, by the at least one processor, improvement in entropy and coverage metrics; and

identifying, by the at least one processor, areas for further optimization.

10 . A system for preemptive entropy reduction in vector embeddings to enhance semantic search, comprising:

an LLM server;

a database server;

a processing server configured to:

receive documents and a set of validation prompts,

segment the documents into chunks,

embed the chunks into a vector space,

transmit the validation prompts to the LLM server configured to process the validation prompts and generate queries for retrieving the chunks from the database server,

intercept retrieved chunks of the chunks from the database server in response to the LLM server queries,

map the retrieved chunks to their positions within the documents,

calculate document coverage for the retrieved chunks, the document coverage indicating an amount of each document covered by the retrieved chunks,

calculate individual document entropies with respect to the chunks used in response to the validation prompts based on the document coverage,

aggregate the individual document entropies to calculate a total system entropy,

generate a report compiling coverage analysis and entropy calculations, and

iteratively refine at least one of the validation prompts, document segmentation, or embedding process in response to the document coverage and the entropy in the report to reduce the entropy in the vector space until the total system entropy reaches a predetermined acceptable threshold prior to deployment of the semantic search; and

a user interface configured to display the report and receive refinement inputs.

11 . The system of claim 10 , wherein to segment the documents into the chunks, the processing server is further configured to:

analyze document structure and content;

define an optimal chunk size for the document structure and the content; and

preserve metadata linking the chunks to a document of the documents from which the metadata linking chunks were extracted.

12 . The system of claim 10 , wherein to embed the chunks into the vector space, the processing server is further configured to:

select an embedding model;

process each chunk through the embedding model to generate vector representations; and

normalize the vector representations.

13 . The system of claim 10 , wherein the LLM server is configured to:

process the validation prompts to generate the queries; and

send the generated queries to the database server to retrieve relevant chunks.

14 . The system of claim 10 , wherein to calculate the document coverage, the processing server is further configured to:

determine a percentage of each document covered by the retrieved chunks;

identify retrieval frequency of the retrieved chunks; and

analyze a distribution of the retrieved chunks across the documents.

15 . The system of claim 10 , wherein to generate the report, the processing server is further configured to:

generate visualizations of metrics; and

create actionable recommendations for system refinement for display on the user interface.

16 . The system of claim 15 , wherein the user interface is configured to:

present the generated report to domain experts;

gather feedback on recommendations and potential adjustments; and

identify cross-domain impacts and opportunities for collaboration.

17 . The system of claim 16 , wherein to refine at least one of the validation prompts, the document segmentation, or the embedding process, the processing server is further configured to:

implement expert-approved changes to prompts received through the user interface;

adjust document segmentation strategies; and

modify the embedding process for gathered insights.

18 . The system of claim 17 , wherein the processing server is further configured to:

re-run the entropy reduction process with refined inputs;

compare new results with previous iterations;

assess improvement in entropy and coverage metrics; and

identify areas for further optimization for displaying on the user interface.

References Cited (9)