Granted Patent B2

US 12,651,449 · App. 18/354,066 · Granted Jun 9, 2026

Inventors: Rory Donovan (Arroyo Grande, CA); Alan Salimov (Oakland, CA); Alexander Gluklick Braun (Oakland, MI); Kerim Doruk Karinca (San Francisco, CA)

Assignee: Virtuous AI, Inc.

G06V10/82G06V10/80G06V10/87

Abstract

Claims (37)

1 . A modular artificial intelligence (AI) platform comprises:

a network interface configured to communicate via a network;

at least one processor; and

a non-transitory machine-readable storage medium that stores operational instructions that, when executed by the at least one processor, cause the at least one processor to perform operations that include:

receiving, via the network interface, media input that includes image data and text data associated therewith;

generating encoded text data, based on the text data and via a text encoder module that includes first language processing AI;

generating encoded image data, based on the image data and via an image encoder module that includes a first plurality of neural networks and at least one first long short-term memory;

generating concept structure data, based on the encoded text data and the encoded image data and via a concept identification module that includes graph-based learning AI;

generating decoded text data, based on the concept structure data and the encoded text data and via a text decoder module that includes second language processing AI;

generating decoded image data, based on the concept structure data and the encoded image data and via an image decoder module that includes a second plurality of neural networks and at least one second long short-term memory; and

combining the decoded image data and the decoded text data to generate media output data.

2 . The modular AI platform of claim 1 , wherein the image decoder module and the text decoder module are trained based on the concept structure data.

3 . The modular AI platform of claim 1 , wherein the image encoder module and the text encoder module are trained based on the concept structure data.

4 . The modular AI platform of claim 1 , wherein the first language processing AI includes a Bidirectional Encoder Representations from Transformers (BERT) AI model.

5 . The modular AI platform of claim 1 , wherein the second language processing AI includes a Bidirectional Encoder Representations from Transformers (BERT) AI model.

6 . The modular AI platform of claim 1 , wherein the first plurality of neural networks includes k U-net models operating as k-experts.

7 . The modular AI platform of claim 6 , wherein the k-experts are trained independently on different subsets of the data, and outputs of k-experts are then combined using a gating mechanism that selects a most relevant one of the k-experts for each input image of the image data.

8 . The modular AI platform of claim 1 , wherein the concept identification module includes a third long short-term memory that processes the encoded image data for input to the graph-based learning AI.

9 . The modular AI platform of claim 1 , wherein the concept identification module includes a fourth long short-term memory that processes the encoded text data for input to the graph-based learning AI.

10 . The modular AI platform of claim 1 , wherein the graph-based learning AI operates based on a GraphSAGE model.

11 . A method comprises:

receiving, via a network interface, media input that includes image data and text data associated therewith;

generating encoded text data, based on the text data and via a text encoder module that includes first language processing AI;

generating encoded image data, based on the image data and via an image encoder module that includes a first plurality of neural networks and at least one first long short-term memory;

generating concept structure data, based on the encoded text data and the encoded image data and via a concept identification module that includes graph-based learning AI;

generating decoded text data, based on the concept structure data and the encoded text data and via a text decoder module that includes second language processing AI;

generating decoded image data, based on the concept structure data and the encoded image data and via an image decoder module that includes a second plurality of neural networks and at least one second long short-term memory; and

combining the decoded image data and the decoded text data to generate media output data.

12 . The method of claim 11 , wherein the image decoder module and the text decoder module are trained based on the concept structure data.

13 . The method of claim 11 , wherein the image encoder module and the text encoder module are trained based on the concept structure data.

14 . The method of claim 11 , wherein the first language processing AI includes a Bidirectional Encoder Representations from Transformers (BERT) AI model.

15 . The method of claim 11 , wherein the second language processing AI includes a Bidirectional Encoder Representations from Transformers (BERT) AI model.

16 . The method of claim 11 , wherein the first plurality of neural networks includes k U-net models operating as k-experts.

17 . The method of claim 16 , wherein the k-experts are trained independently on different subsets of the data, and outputs of k-experts are then combined using a gating mechanism that selects a most relevant one of the k-experts for each input image of the image data.

18 . The method of claim 11 , wherein the concept identification module includes a third long short-term memory that processes the encoded image data for input to the graph-based learning AI.

19 . The method of claim 11 , wherein the concept identification module includes a fourth long short-term memory that processes the encoded text data for input to the graph-based learning AI.

20 . The method of claim 11 , wherein the graph-based learning AI operates based on a GraphSAGE model.

Assignments (1)

Continuity (1)

References Cited (41)