Granted Patent B1

US 12,242,503 · App. 18/898,502 · Granted Mar 4, 2025

Inventors: Elaine Kelsey (Corvallis, OR); Elliot Nicholas Robson (Seoul, KR); Sazzad Mahmud Nasir (Muncie, IN); Jeffrey Thomas Yarbro (Memphis, TN); Robert Oscar Robson (Corvallis, OR); Lauren Elizabeth Egerton (New York, NY); Spencer Thomas Ward (Kent, WA); Brendan Michael Kelly (Somerville, MA)

Assignee: THIA ST Co.

G06F16/258G06F16/2455

Abstract

Claims (202)

1. A method performed by a copilot comprising a network of microservices, the method comprising:

receiving client input from a first client;

at an expansion microservice:

receiving a first input comprising one or more language tokens, comprising or derived from the client input;

determining first output comprising one or more tokens associated with but distinct from the first input; and

transmitting the first output toward a retrieval microservice;

at the retrieval microservice:

receiving a second input comprising or derived from the first output;

performing retrieval augmented generation (RAG) based on the second input to retrieve third input, comprising one or more data objects related to the second input, from one or more data producers among the microservices;

determining and transmit third output, based on the second and third inputs, toward one or more core microservices including a given core microservice;

at the given core microservice:

receiving a fourth input comprising or derived from the third output;

applying at least one trained machine learning tool to the fourth input, to determine a fourth output; and

transmitting the fourth output toward an evaluation microservice;

at an evaluation microservice:

receiving fifth input comprising or derived from the fourth output; and

analyzing the fifth input;

determining whether to further invoke one or more of the expansion, retrieval, or core microservices based on the analyzing;

determining client output from the fourth output or from results of the further invoking; and

transmitting the client output to a second client.

2. The method of claim 1 , wherein the trained machine learning tool is a large language model (LLM).

3. The method of claim 1 , wherein a given one of the one or more data producers comprises an embedding microservice and a document microservice, the document microservice maintains documents in a document repository indexed by vector representations, and the RAG action is performed by:

invoking the embedding microservice;

at the embedding microservice:

receiving a sixth input from the retrieval microservice, the sixth input derived from the second input; and

determining and transmitting a sixth output comprising one or more vector embeddings representative of at least portions of the sixth input;

invoking the document microservice; and

at the document microservice:

receiving a seventh input comprising or derived from the sixth output; and

identifying and transmitting one or more of the documents having content similar to at least portions of the seventh input;

wherein the third input to the retrieval microservice comprises or is derived from the seventh output.

4. The method of claim 1 , wherein a given one of the one or more data producers comprises a database microservice, and the RAG action is performed by:

invoking the database microservice; and

at the database microservice:

receiving a eighth input from the retrieval microservice, the eighth input derived from the second input;

retrieving, from one or more databases, database objects relevant to the eighth input; and

determining and transmitting a eighth output based on the retrieved database objects;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

5. The method of claim 1 , wherein a given one of the one or more data producers comprises a messaging microservice, and the RAG action is performed by:

invoking the messaging microservice; and

at the messaging microservice:

receiving an ninth input from the retrieval microservice, the ninth input derived from the second input;

retrieving, from one or more message repositories, messages or metadata relevant to the ninth input; and

determining and transmitting an ninth output based on the retrieved messages or metadata;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

6. The method of claim 5 , wherein the one or more data objects comprise email, voicemail, text messages, instant messages, video messages, multi-mode messages, or attachments thereto.

7. The method of claim 1 , wherein the microservices further comprise a qualification microservice, and the method further comprises:

at the qualification microservice:

receiving a tenth input based on the third output;

comparing the tenth input with a graphical model of a knowledge corpus incorporated in the copilot;

determining whether the copilot is competent to act on the tenth input, wherein the copilot is determined to be competent in at least a first case; and

in the first case, determining and transmitting tenth output based on the tenth input, toward at least one of the core microservices;

wherein the fourth input comprises or is derived from the tenth output.

8. The method of claim 1 , wherein the one or more core microservices comprise an ensemble of at least two core microservices arranged to form at least one cycle, within which each core microservice other than the given core microservice is coupled to:

receive eleventh input from a neighboring first core microservice of the ensemble;

apply at least one of the trained ML tools to the eleventh input, to determine an eleventh output; and

transmit the eleventh output to a neighboring second core microservice distinct from the first core microservice.

9. The method of claim 1 , wherein the evaluation microservice is a first evaluation microservice, and the microservices further comprise second and third evaluation microservices, and the method further comprises:

at the second evaluation microservice:

receiving twelfth input derived from the third input received by the retrieval microservice; and

analyzing the twelfth input; and

determining whether to perform another RAG iteration based on the analyzing the twelfth input;

at the third evaluation microservice:

receiving thirteenth input derived from the fourth output generated by the given core microservice; and

analyzing the thirteenth input; and

determining whether to perform another RAG iteration based on the analyzing the thirteenth input.

10. One or more non-transitory computer-readable media storing instructions which, when executed by one or more hardware processors implementing a copilot comprising a network of microservices, cause the one or more hardware processors to perform operations comprising:

receiving client input from a first client;

at an expansion microservice:

receiving a first input comprising one or more language tokens, comprising or derived from the client input;

determining first output comprising one or more tokens associated with but distinct from the first input; and

transmitting the first output toward a retrieval microservice;

at the retrieval microservice:

receiving a second input comprising or derived from the first output;

performing retrieval augmented generation (RAG) based on the second input to retrieve third input, comprising one or more data objects related to the second input, from one or more data producers among the microservices;

determining and transmit third output, based on the second and third inputs, toward one or more core microservices including a given core microservice;

at the given core microservice:

receiving a fourth input comprising or derived from the third output;

applying at least one trained machine learning tool to the fourth input, to determine a fourth output; and

transmitting the fourth output toward an evaluation microservice;

at an evaluation microservice:

receiving fifth input comprising or derived from the fourth output; and

analyzing the fifth input;

determining whether to further invoke one or more of the expansion, retrieval, or core microservices based on the analyzing;

determining client output from the fourth output or from results of the further invoking operation; and

transmitting the client output to a second client.

11. The one or more computer-readable media of claim 10 , wherein the trained machine learning tool is a large language model (LLM).

12. The one or more computer-readable media of claim 10 , wherein a given one of the one or more data producers comprises an embedding microservice and a document microservice, the document microservice maintains documents in a document repository indexed by vector representations, and the RAG operation comprises further operations including:

invoking the embedding microservice;

at the embedding microservice:

receiving a sixth input from the retrieval microservice, the sixth input derived from the second input; and

determining and transmitting a sixth output comprising one or more vector embeddings representative of at least portions of the sixth input;

invoking the document microservice; and

at the document microservice:

receiving a seventh input comprising or derived from the sixth output; and

identifying and transmitting one or more of the documents having content similar to at least portions of the seventh input;

wherein the third input to the retrieval microservice comprises or is derived from the seventh output.

13. The one or more computer-readable media of claim 10 , wherein a given one of the one or more data producers comprises a database microservice, and the RAG operation comprises further operations including:

invoking the database microservice; and

at the database microservice:

receiving a eighth input from the retrieval microservice, the eighth input derived from the second input;

retrieving, from one or more databases, database objects relevant to the eighth input; and

determining and transmitting a eighth output based on the retrieved database objects;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

14. The one or more computer-readable media of claim 10 , wherein a given one of the one or more data producers comprises a messaging microservice, and the RAG operation comprises further operations including:

invoking the messaging microservice; and

at the messaging microservice:

receiving an ninth input from the retrieval microservice, the ninth input derived from the second input;

retrieving, from one or more message repositories, messages or metadata relevant to the ninth input; and

determining and transmitting an ninth output based on the retrieved messages or metadata;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

15. The one or more computer-readable media of claim 14 , wherein the one or more data objects comprise email, voicemail, text messages, instant messages, video messages, multi-mode messages, or attachments thereto.

16. The one or more computer-readable media of claim 10 , wherein the microservices further comprise a qualification microservice, and the operations further comprise:

at the qualification microservice:

receiving a tenth input based on the third output;

comparing the tenth input with a graphical model of a knowledge corpus incorporated in the copilot;

determining whether the copilot is competent to act on the tenth input, wherein the copilot is determined to be competent in at least a first case; and

in the first case, determining and transmitting tenth output based on the tenth input, toward at least one of the core microservices;

wherein the fourth input comprises or is derived from the tenth output.

17. The one or more computer-readable media of claim 10 , wherein the one or more core microservices comprise an ensemble of at least two core microservices arranged to form at least one cycle, and the operations further comprise:

by each core microservice other than the given core microservice:

receive eleventh input from a neighboring first core microservice of the ensemble;

apply at least one of the trained ML tools to the eleventh input, to determine an eleventh output; and

transmit the eleventh output to a neighboring second core microservice distinct from the first core microservice.

18. The one or more computer-readable media of claim 10 , wherein the evaluation microservice is a first evaluation microservice, and the microservices further comprise second and third evaluation microservices, and the operations further comprise:

at the second evaluation microservice:

receiving twelfth input derived from the third input received by the retrieval microservice; and

analyzing the twelfth input; and

determining whether to perform another RAG iteration based on the analyzing the twelfth input;

at the third evaluation microservice:

receiving thirteenth input derived from the fourth output generated by the given core microservice; and

analyzing the thirteenth input; and

determining whether to perform another RAG iteration based on the analyzing the thirteenth input.

19. A system, comprising:

a copilot, comprising a network of microservices, implemented on one or more hardware processors with memory coupled thereto:

one or more computer-readable media storing instructions which, when executed by the one or more hardware processors, cause the copilot to perform operations comprising:

receiving client input from a first client;

at an expansion microservice:

receiving a first input comprising one or more language tokens, comprising or derived from the client input;

determining first output comprising one or more tokens associated with but distinct from the first input; and

transmitting the first output toward a retrieval microservice;

at the retrieval microservice:

receiving a second input comprising or derived from the first output;

performing retrieval augmented generation (RAG) based on the second input to retrieve third input, comprising one or more data objects related to the second input, from one or more data producers among the microservices;

determining and transmit third output, based on the second and third inputs, toward one or more core microservices including a given core microservice;

at the given core microservice:

receiving a fourth input comprising or derived from the third output;

applying at least one trained machine learning tool to the fourth input, to determine a fourth output; and

transmitting the fourth output toward an evaluation microservice;

at an evaluation microservice:

receiving fifth input comprising or derived from the fourth output; and

analyzing the fifth input;

determining whether to further invoke one or more of the expansion, retrieval, or core microservices based on the analyzing;

determining client output from the fourth output or from results of the further invoking operation; and

transmitting the client output to a second client.

20. The system of claim 19 , wherein the trained machine learning tool is a large language model (LLM).

21. The system of claim 19 , wherein a given one of the one or more data producers comprises an embedding microservice and a document microservice, the document microservice maintains documents in a document repository indexed by vector representations, and the RAG operation comprises further operations including:

invoking the embedding microservice;

at the embedding microservice:

receiving a sixth input from the retrieval microservice, the sixth input derived from the second input; and

determining and transmitting a sixth output comprising one or more vector embeddings representative of at least portions of the sixth input;

invoking the document microservice; and

at the document microservice:

receiving a seventh input comprising or derived from the sixth output; and

identifying and transmitting one or more of the documents having content similar to at least portions of the seventh input;

wherein the third input to the retrieval microservice comprises or is derived from the seventh output.

22. The system of claim 19 , wherein a given one of the one or more data producers comprises a database microservice, and the RAG operation comprises further operations including:

invoking the database microservice; and

at the database microservice:

receiving a eighth input from the retrieval microservice, the eighth input derived from the second input;

retrieving, from one or more databases, database objects relevant to the eighth input; and

determining and transmitting a eighth output based on the retrieved database objects;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

23. The system of claim 19 , wherein a given one of the one or more data producers comprises a messaging microservice, and the RAG operation comprises further operations including:

invoking the messaging microservice; and

at the messaging microservice:

receiving an ninth input from the retrieval microservice, the ninth input derived from the second input;

retrieving, from one or more message repositories, messages or metadata relevant to the ninth input; and

determining and transmitting an ninth output based on the retrieved messages or metadata;

wherein the third input to the retrieval microservice comprises or is derived from the eighth output.

24. The system of claim 23 , wherein the one or more data objects comprise email, voicemail, text messages, instant messages, video messages, multi-mode messages, or attachments thereto.

25. The system of claim 19 , wherein the microservices further comprise a qualification microservice, and the operations further comprise:

at the qualification microservice:

receiving a tenth input based on the third output;

comparing the tenth input with a graphical model of a knowledge corpus incorporated in the copilot;

determining whether the copilot is competent to act on the tenth input, wherein the copilot is determined to be competent in at least a first case; and

in the first case, determining and transmitting tenth output based on the tenth input, toward at least one of the core microservices;

wherein the fourth input comprises or is derived from the tenth output.

26. The system of claim 19 , wherein the one or more core microservices comprise an ensemble of at least two core microservices arranged to form at least one cycle, and the operations further comprise:

by each core microservice other than the given core microservice:

receive eleventh input from a neighboring first core microservice of the ensemble;

apply at least one of the trained ML tools to the eleventh input, to determine an eleventh output; and

transmit the eleventh output to a neighboring second core microservice distinct from the first core microservice.

27. The system of claim 19 , wherein the evaluation microservice is a first evaluation microservice, and the microservices further comprise second and third evaluation microservices, and the operations further comprise:

at the second evaluation microservice:

receiving twelfth input derived from the third input received by the retrieval microservice; and

analyzing the twelfth input; and

determining whether to perform another RAG iteration based on the analyzing the twelfth input;

at the third evaluation microservice:

receiving thirteenth input derived from the fourth output generated by the given core microservice; and

analyzing the thirteenth input; and

determining whether to perform another RAG iteration based on the analyzing the thirteenth input.

Assignments (2)

Continuity (2)

References Cited (67)

Cited By (8)

US 12,405,977 US 12,405,978 US 12,405,979 US 12,475,151 US 12,493,638 US 12,602,412 US 12,711,164 US 12,711,308