Automatic user interface data generation
Techniques are disclosed relating to automatically synthesizing user interface (UI) component instances. In disclosed techniques a computer system receives a set of existing UI elements and a set of design rules for the set of existing elements, where design rules in the set of design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements. The one or more allowed states may correspond to one or more visual characteristics. Using the set of existing UI elements, the computer system may then automatically generate a plurality of UI component instances based on the set of design rules, where a respective UI component instance includes a first UI element in a first allowed state. The computer system may then train, using the plurality of UI component instances, a machine learning model operable to automatically generate UI designs.
1. A method, comprising:
receiving, by a computer system, a set of existing user interface (UI) elements and a set of design rules for the set of existing UI elements, wherein design rules in the set of design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements, and wherein the one or more allowed states correspond to one or more visual characteristics;
automatically generating, by the computer system using the set of existing UI elements, a plurality of UI component instances, wherein the automatically generating is performed based on the set of design rules, and wherein a respective UI component instance includes a first UI element in a first allowed state; and
training, by the computer system using the plurality of UI component instances, a machine learning model, wherein the trained machine learning model is operable to automatically generate UI designs.
2. The method of claim 1 , wherein the automatically generating includes:
generating, based on the one or more allowed states for respective UI elements in the set of existing UI elements, a tree structure, wherein levels of the tree structure correspond to respective UI elements, and wherein branches of the tree structure at a respective level correspond to different allowed states of UI element corresponding to that respective level.
3. The method of claim 2 , wherein the generating the tree structure includes traversing the tree structure to identify the plurality of UI component instances, wherein leaf nodes of the tree structure represent different ones of the plurality of UI component instances.
4. The method of claim 1 , wherein an allowed state for the first UI element is conditionally based on an allowed state for a second, different UI element.
5. The method of claim 1 , wherein the respective UI component instance includes the first UI element in a second allowed state and a second UI element in a first allowed state.
6. The method of claim 5 , wherein the set of design rules specify that the first allowed state of the first UI element includes a first color and that the second allowed state of the first UI element includes a second, different color.
7. The method of claim 1 , wherein the set of design rules specify, for UI elements in the set of existing UI elements, compatibility between the UI elements and visual characteristics of the UI elements.
8. The method of claim 1 , further comprising:
automatically generating, by the computer system based on the plurality of UI component instances, a plurality of UI design instances; and
training, by the computer system using the plurality of UI design instances, the machine learning model trained using the plurality of UI component instances.
9. The method of claim 1 , wherein the set of design rules and the set of existing UI elements are received from an entity requesting automatic generation of UIs, and wherein the set of design rules are associated with a plurality of design requirements of the entity.
10. A non-transitory computer-readable medium having instructions stored thereon that are capable of causing a computing device to implement operations comprising:
receiving a set of existing user interface (UI) elements and a set of design rules for the set of existing UI elements, wherein the design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements, wherein the one or more allowed states correspond to one or more visual characteristics;
automatically generating, using the set of existing UI elements, a plurality of UI component instances, wherein the automatically generating is performed based on the set of design rules, and wherein a respective UI component instance includes a first UI element in a first allowed state and a second UI element in a second allowed state; and
training, using the plurality of UI component instances, a machine learning model operable to automatically generate UI designs.
11. The non-transitory computer-readable medium of claim 10 , wherein the automatically generating includes:
generating, based on the one or more allowed states for respective UI elements in the set of existing UI elements, a tree structure, wherein levels of the tree structure correspond to respective UI elements, and wherein branches of the tree structure at a respective level correspond to different allowed states of a UI element corresponding to that respective level.
12. The non-transitory computer-readable medium of claim 11 , wherein the generating the tree structure includes traversing the tree structure to identify the plurality of UI component instances, wherein leaf nodes of the tree structure represent different ones of the plurality of UI component instances.
13. The non-transitory computer-readable medium of claim 10 , wherein the set of design rules specify, for UI elements in the set of existing UI elements, compatibility between the UI elements and visual characteristics of the UI elements.
14. The non-transitory computer-readable medium of claim 10 , wherein an allowed state for the first UI element is conditionally based on an allowed state for the second UI element.
15. The non-transitory computer-readable medium of claim 10 , wherein the set of design rules include design labels assigned by a designer to respective UI elements in the set of existing UI elements, wherein the design labels specify the one or more allowed states for the respective UI elements.
16. A method, comprising:
automatically generating, by a computing system using a machine learning model trained using a plurality of synthetic UI component instances, one or more user interface (UI) designs, wherein the plurality of synthetic UI component instances are automatically generated by:
receiving a set of existing user interface (UI) elements and a set of design rules for the set of existing UI elements, wherein the design rules indicate one or more allowed states for respective UI elements in the set of existing UI elements, wherein the one or more allowed states correspond to a respective visual characteristic; and
automatically generating, using the set of existing UI elements, the plurality of synthetic UI component instances, wherein the automatically generating is performed based on the set of design rules, and wherein a respective UI component instance includes a first UI element in a first allowed state.
17. The method of claim 16 , wherein the automatically generating includes:
generating, based on the one or more allowed states for respective UI elements in the set of existing UI elements, a tree structure, wherein levels of the tree structure correspond to respective UI elements, and wherein branches of the tree structure at a respective level correspond to different allowed states of the UI element corresponding to that respective level.
18. The method of claim 17 , wherein the generating the tree structure includes traversing the tree structure to identify the plurality of synthetic UI component instances, wherein leaf nodes of the tree structure represent different ones of the plurality of synthetic UI component instances.
19. The method of claim 16 , wherein an allowed state for the first UI element is conditionally based on an allowed state for a second, different UI element.
20. The method of claim 16 , wherein the set of design rules specify one or more visual characteristics of the following visual characteristics: color, shape, size, opacity, font, orientation, and position.