Multi-field semantic vector indexing for named entity retrieval in tenant-specific environments

1 . A compound artificial intelligence (AI) system, comprising:

a user device configured to display a query interface to receive a query from a user in a natural language string, the query for finding one or more named entities that are associated with domain specific data of a domain;

a data store storing a semantic index comprising a plurality of semantic embeddings that are associated with a plurality of named entities that are formerly analyzed by the domain, wherein the semantic index comprises a set of semantic understandings of the plurality of named entities that is represented as the semantic embeddings, the semantic understandings are extracted based on characteristics of the named entities that are identified from the domain specific data, and wherein the semantic index is generated by:

extracting relevant structured fields from a structured data source in the domain,

parsing unstructured documents associated with the same named entity,

combining extracted structured and unstructured information into a normalized data model, and

generating semantic descriptors for each semantic dimension from the normalized data model; and

a computing device comprising memory and one or more processors, wherein the memory stores computer code comprising executable instructions, wherein the instructions, when executed by the one or more processors, cause the one or more processors to:

invoke a transformer-based deep neural network (DNN) to decompose the natural language string into one or more semantic components, each semantic component comprising a version of the natural language string, each semantic component being in natural language;

convert the one or more semantic components into one or more query embeddings;

search for the semantic index;

identify, from the semantic index, one or more semantic embeddings based on the one or more query embeddings;

identify one or more named entities that are associated with the one or more semantic embeddings; and

generate a response to the query based on the one or more named entities, the response include the one or more named entities that are formerly analyzed by the domain and match the natural language string.

2 . The system of claim 1 , wherein the version of the natural language string is a portion of the natural language string or an interpreted understanding that is generated by the transformer-based DNN.

3 . The system of claim 1 , wherein the natural language string comprises a target named entity, and wherein the instruction to invoke the transformer-based DNN to decompose the natural language string into the one or more semantic components comprises instructions to:

detect a presence of the named entity in the natural language string;

extract the named entity from the string and matching it to a stored record in a domain-specific entity database;

retrieve stored semantic dimension values associated with the matched named entity; and

generate interpreted semantic components from the retrieved dimension values for use in similarity searching.

4 . The system of claim 3 , wherein the instruction to identify, from the semantic index, one or more semantic embeddings comprises instructions to:

embed each semantic dimension value retrieved from the target named entity;

search dimension-specific vector indexes of the semantic index with corresponding embeddings to produce nearest neighbor results in each dimension;

aggregate the nearest neighbor results across multiple dimensions into a unified candidate set; and

rank the candidate set based on aggregated semantic similarity scores.

5 . A computer-implemented method, comprising:

receiving a query from a user in a natural language string, the query for finding one or more named entities that are associated with domain specific data of a domain;

invoking a transformer-based deep neural network (DNN) to decompose the natural language string into one or more semantic components, each semantic component comprising a version of the natural language string, each semantic component being in natural language;

converting the one or more semantic components into one or more query embeddings;

searching for a semantic index that stores a plurality of semantic embeddings that are associated with a plurality of named entities that are formerly analyzed by the domain, wherein the semantic index comprises a set of semantic understandings of the plurality of named entities that is represented as the semantic embeddings, the semantic understandings are extracted based on characteristics of the named entities that are identified from the domain specific data, and wherein the semantic index is generated by:

extracting relevant structured fields from a structured data source in the domain,

parsing unstructured documents associated with the same named entity,

combining extracted structured and unstructured information into a normalized data model, and

generating semantic descriptors for each semantic dimension from the normalized data model;

identifying, from the semantic index, one or more semantic embeddings based on the one or more query embeddings;

identifying one or more named entities that are associated with the one or more semantic embeddings; and

generating a response to the query based on the one or more named entities, the response include the one or more named entities that are formerly analyzed by the domain and match the natural language string.

6 . The computer-implemented method of claim 5 , wherein the version of the natural language string is a portion of the natural language string or an interpreted understanding that is generated by the transformer-based DNN.

7 . The computer-implemented method of claim 5 , wherein invoking the transformer-based DNN to decompose the natural language string into the one or more semantic components comprises:

identifying portions of the natural language string that correspond to predefined semantic dimensions used in the semantic index;

extracting relevant natural language phrases for each identified semantic dimension;

normalizing the extracted phrases to remove extraneous linguistic content while preserving semantic meaning;

generating interpreted natural language versions that capture an intent of each phrase with respect to the semantic dimension; and

associating each interpreted phrase with a corresponding dimension label for embedding generation.

8 . The computer-implemented method of claim 5 , wherein the natural language string comprises a target named entity, and wherein invoking the transformer-based DNN to decompose the natural language string into the one or more semantic components comprises:

detecting a presence of the named entity in the natural language string;

extracting the named entity from the string and matching it to a stored record in a domain-specific entity database;

retrieving stored semantic dimension values associated with the matched named entity; and

generating interpreted semantic components from the retrieved dimension values for use in similarity searching.

9 . The computer-implemented method of claim 8 , wherein identifying, from the semantic index, one or more semantic embeddings comprises:

embedding each semantic dimension value retrieved from the target named entity;

searching dimension-specific vector indexes of the semantic index with corresponding embeddings to produce nearest neighbor results in each dimension;

aggregating the nearest neighbor results across multiple dimensions into a unified candidate set; and

ranking the candidate set based on aggregated semantic similarity scores.

10 . The computer-implemented method of claim 5 , wherein identifying the one or more named entities that are associated with the one or more semantic embeddings comprises:

retrieving the named entity identifier that is stored in association with each semantic embedding in the semantic index;

mapping each matching embedding to its corresponding named entity identifier; and

merging identifiers across dimension-specific searches to produce a consolidated list of matching named entities.

11 . The computer-implemented method of claim 5 , wherein identifying the one or more named entities that are associated with the one or more semantic embeddings comprises:

determining, for each named entity in the semantic index, a count of semantic embeddings;

selecting as candidate named entities those named entities having a highest count of semantic embeddings;

computing an aggregated similarity score for each candidate named entity based on matching its matching semantic embeddings to the query embeddings; and

ranking the candidate named entities in descending order of the aggregated similarity scores.

12 . The computer-implemented method of claim 5 , wherein the query is received from a user interface operated by a server and the semantic index resides on a tenant-side AI system that is operated within the domain.

13 . The computer-implemented method of claim 5 , wherein the semantic index comprises a plurality of separate vector indexes, each vector index corresponding to a semantic dimension used to characterize the named entities.

14 . The computer-implemented method of claim 5 , wherein the semantic index is populated by extracting data from both structured data sources and unstructured data sources within the domain.

15 . The computer-implemented method of claim 5 , wherein identifying, from the semantic index, the one or more semantic embeddings comprises:

applying a ranking algorithm to order the matching named entities based on aggregated similarity scores across multiple semantic dimensions.

16 . The computer-implemented method of claim 5 , wherein the semantic index supports hybrid retrieval in which semantic similarity search results are filtered or combined with keyword search results over associated documents for each named entity.

17 . The computer-implemented method of claim 16 , wherein a keyword search is performed over a document database that stores parsed content linked to named entities via a precomputed mapping.

18 . The computer-implemented method of claim 5 , wherein the transformer-based DNN is configured to dynamically create additional semantic components upon detecting attributes in the domain specific data.

19 . A non-transitory computer-readable medium configured to store code comprising instructions, wherein the instructions, when executed by one or more processors, cause the one or more processors to:

receive a query from a user in a natural language string, the query for finding one or more named entities that are associated with domain specific data of a domain;

invoke a transformer-based deep neural network (DNN) to decompose the natural language string into one or more semantic components, each semantic component comprising a version of the natural language string, each semantic component being in natural language;

convert the one or more semantic components into one or more query embeddings;

search for a semantic index that stores a plurality of semantic embeddings that are associated with a plurality of named entities that are formerly analyzed by the domain, wherein the semantic index comprises a set of semantic understandings of the plurality of named entities that is represented as the semantic embeddings, the semantic understandings are extracted based on characteristics of the named entities that are identified from the domain specific data, and wherein the semantic index is generated by:

extracting relevant structured fields from a structured data source in the domain,

parsing unstructured documents associated with the same named entity,

combining extracted structured and unstructured information into a normalized data model, and

generating semantic descriptors for each semantic dimension from the normalized data model;

identify, from the semantic index, one or more semantic embeddings based on the one or more query embeddings;

identify one or more named entities that are associated with the one or more semantic embeddings; and

generate a response to the query based on the one or more named entities, the response include the one or more named entities that are formerly analyzed by the domain and match the natural language string.