Granted Patent B2

US 12,456,290 · App. 16/913,153 · Granted Oct 28, 2025

Inventors: Siddhant Pardeshi (Pune, IN); Pranit P. Kothari (Pune, IN); Vinayak Vilas Gaikwad (Pune, IN); Aditya Karra (Indore, IN); Travis Muhlestein (Redmond, WA)

Assignee: NVIDIA Corporation

G06V10/82G06F3/04815G06F8/38G06F8/76G06N3/08G06V10/454G06F8/34G06N20/00G06V20/20

Abstract

Claims (56)

1. One or more processors, comprising: circuitry to:

use one or more neural networks to transform different types of visual features of a first graphical user interface (GUI) for a first device into representations with a common structure;

receive one or more constraints associated with a second GUI being executed by a second device;

determine one or more relationships between the one or more constraints and the different types of visual features based, at least in part, on the representations; and

use the one or more neural networks to generate the second GUI that includes one or more of the different types of visual features of the first GUI that are modified based, at least in part, on the one or more relationships.

2. The one or more processors of claim 1 , wherein the circuitry is further to determine context corresponding to the different types of visual features to determine the one or more relationships.

3. The one or more processors of claim 1 , wherein the one or more neural networks comprise a generative adversarial network (GAN) to generate the second GUI.

4. The one or more processors of claim 1 , wherein the one or more neural networks comprise a portion that selects a screen or page for the second GUI.

5. The one or more processors of claim 1 , wherein the common structure is associated with a JavaScript Object Notation (JSON) schema.

6. The one or more processors of claim 1 , wherein different portions of the one or more neural networks identify the different types of visual features.

7. A system, comprising:

one or more processors to use one or more neural networks to transform different types of visual features of a first graphical user interface (GUI) for a first device into representations with a common structure;

receive one or more constraints associated with a second GUI being executed by a second device;

determine one or more relationships between the one or more constraints and the different types of visual features based, at least in part, on the representations; and

use the one or more neural networks to generate the second GUI that includes one or more of the different types of visual features of the first GUI that are modified based, at least in part, on the one or more relationships.

8. The system of claim 7 , wherein the one or more neural networks comprise at least one of: convolutional neural network, variational encoder, or recurrent neural network to identify at least one of the different types of visual features.

9. The system of claim 7 , wherein the one or more processors are further to determine context corresponding to the different types of visual features to determine the one or more relationships.

10. The system of claim 7 , wherein the one or more neural networks generate a latent space that indicates the one or more relationships.

11. The system of claim 7 , wherein the one or more neural networks comprise a portion that selects a screen or page for the second GUI.

12. The system of claim 7 , wherein the common structure is associated with a JavaScript Object Notation (JSON) schema.

13. A method comprising:

transforming, using one or more neural networks, different types of visual features of a first graphical user interface (GUI) for a first device into representations with a common structure;

receiving one or more constraints associated with a second GUI being executed by a second device;

determining one or more relationships between the one or more constraints and the different types of visual features based, at least in part, on the representations; and

using the one or more neural networks to generate the second GUI that includes one or more of the different types of visual features of the first GUI that are modified based, at least in part, on the one or more relationships.

14. The method of claim 13 , further comprising:

identifying the different types of visual features using different portions of the one or more neural networks.

15. The method of claim 13 , further comprising:

determining context corresponding to the different types of visual features to determine the one or more relationships.

16. The method of claim 13 , wherein the one or more neural networks generate a latent space that indicates the one or more relationships.

17. The method of claim 13 , wherein the one or more neural networks comprise a portion that selects a screen or page for the second GUI.

18. The method of claim 13 , wherein the common structure is associated with a JavaScript Object Notation (JSON) schema.

19. A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:

use one or more neural networks to transform different types of visual features of a first graphical user interface (GUI) for a first device into representations with a common structure;

receive one or more constraints associated with a second GUI being executed by a second device;

determine one or more relationships between the one or more constraints and the different types of visual features based, at least in part, on the representations; and

use the one or more neural networks to generate the second GUI that includes one or more of the different types of visual features of the first GUI that are modified based, at least in part, on the one or more relationships.

20. The non-transitory machine-readable medium of claim 19 , wherein the instructions if performed further cause the one or more processors to:

identify the different types of visual features using different portions of the one or more neural networks.

21. The non-transitory machine-readable medium of claim 19 , wherein the instructions if performed further cause the one or more processors to:

determine context corresponding to the different types of visual features to determine the one or more relationships.

22. The non-transitory machine-readable medium of claim 19 , wherein the one or more neural networks generate a latent space that indicates the one or more relationships.

23. The non-transitory machine-readable medium of claim 19 , wherein the one or more neural networks comprise a portion that selects a screen or page for the second GUI.

24. The non-transitory machine-readable medium of claim 19 , wherein the common structure is associated with a JavaScript Object Notation (JSON) schema.

25. An interface translation system, comprising:

one or more processors to:

use one or more neural networks to transform different types of visual features of a first graphical user interface (GUI) for a first device into representations with a common structure;

receive one or more constraints associated with a second GUI being executed by a second device;

determine one or more relationships between the one or more constraints and the different types of visual features based, at least in part, on the representations; and

use the one or more neural networks to generate the second GUI that includes one or more of visual features of the first GUI that are modified based, at least in part, on the one or more relationships; and

memory for storing network parameters for the one or more neural networks.

26. The interface translation system of claim 25 , wherein the one or more processors are further to identify the different types of the visual features using different portions of the one or more neural networks.

27. The interface translation system of claim 25 , wherein the one or more processors are further to determine context corresponding to the different types of visual features to determine the one or more relationships.

28. The interface translation system of claim 25 , wherein the one or more first neural networks comprise a portion that selects a screen or page for the second GUI.

29. The interface translation system of claim 25 , wherein the one or more neural networks generate a latent space that indicates the one or more relationships.

30. The interface translation system of claim 25 , wherein the common structure is associated with a JavaScript Object Notation (JSON) schema.

Assignments (1)

Continuity (1)

References Cited (29)