IP Library › Granted Patent US 12,061,882
Granted Patent B2
US 12,061,882 · App. 16/886,372 · Granted Aug 13, 2024

Machine-learning assisted autocomplete in visual programming

Inventor: Siu Yee Man (Vancouver, CA)
Assignee: Electronic Arts Inc.
G06F8/34G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,061,882
App. No.
16/886,372
Granted
Aug 13, 2024
Kind
B2
Abstract

A visual-programming tool accesses a datastore of existing program data, which includes program code associated with a plurality of programs, (e.g., games), stored as multi-layer trees of nodes with logical connections signifying relationships between nodes of adjacent layers. The tool uses machine learning to determine multi-layer trees of predictions including at least one chain of nodes from the multi-layer tree of nodes. The tool provides one or more of the multi-layer trees of predictions including chains of nodes for presentation via a graphical-user interface and receives a gliding gesture input indicating a decision between nodes of the one or more layers. The input indicates a decision associated with selecting a chain of nodes when the input traverses more than one layer. The visual-programming tool provides further layers from the one or more multi-layer trees of predictions based on the received input without requiring acceptance of individual nodes at each layer.

Claims (99)

1. A computer-implemented method implementing a visual-programming system, the computer-implemented method comprising:

storing, in cloud storage, shared program code accessible via a visual-programming tool incorporating machine-learning assisted autocomplete as a multi-layer tree of nodes with a logical connection signifying a relationship between at least one node of a first layer of the multi-layer tree of nodes and at least one node of a second layer of the multi-layer tree of nodes;

determining, via the visual-programming system and based at least on a machine-learning component of the visual-programming tool operating on heuristics of the multi-layer tree of nodes, a plurality of multi-layer trees of predictions based on the multi-layer tree of nodes accessed via the cloud storage, wherein individual multi-layer trees of predictions of the plurality of multi-layer trees of predictions include at least one chain of nodes formed with logical connections between layers of the individual multi-layer tree of predictions;

receiving, via a graphical user interface (GUI) associated with the visual-programming tool, a first indication corresponding to a first decision in development program code;

determining, based on a machine learning model associated with the visual-programming tool and the visual-programming system, a first multi-layer tree of code predictions of the plurality of multi-layer trees of predictions based on the first indication of the first decision received at the GUI;

providing, via the GUI associated with the visual-programming tool, at least a first layer of nodes of the first multi-layer tree of code predictions based on the first indication of the first decision;

determining, via the visual-programming tool, at least a first node of the first layer of nodes associated with the first indication;

providing, via the GUI associated with the visual-programming tool, at least a first visual marking of a presentation of at least one node including the first node of the first layer of nodes that corresponds to the first decision;

receiving, via the GUI associated with the visual-programming tool, a second indication corresponding to a second decision in the development program code, the second decision being made from among the nodes of the first layer of the first multi-layer tree of code predictions;

providing, via the GUI associated with the visual-programming tool, at least a second layer of nodes of the first multi-layer tree of code predictions based on the second indication of the second decision;

determining, via the visual-programming tool, at least a second node of the second layer of nodes associated with the second indication;

providing, via the GUI associated with the visual-programming tool, at least a second visual marking of the presentation of at least one node including the second node of the second layer of nodes that corresponds to the second decision; and

wherein:

the at least one chain of nodes includes a first chain of nodes that includes the first node and the second node; and

responsive to receiving an indication of acceptance of the at least one chain of nodes, the GUI is further configured to cause presentation of one or more alternative nodes in subsequent layers of the multi-layer tree of predictions in the GUI, the presentation of the one or more alternative nodes including at least one of:

brightened according to an increasing level of certainty that the one or more alternative nodes can be a part of the chain; or

dimmed according to a decreasing level of certainty that the one or more alternative nodes will be a part of the chain.

2. The computer-implemented method of claim 1 , further comprising:

identifying, by the machine-learning component of the visual-programming tool, a type of graph associated with the multi-layer tree of nodes; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the type of graph.

3. The computer-implemented method of claim 1 , wherein at least part of the development program code is associated with programs including games, the computer-implemented method further comprising:

identifying, by the machine-learning component of the visual-programming tool, a plurality of games having a similarity in game play;

determining, by the machine-learning component of the visual-programming tool, that the development program code has the similarity; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the similarity.

4. The computer-implemented method of claim 1 , wherein at least part of the development program code is associated with programs including games, the computer-implemented method further comprising:

identifying, by the machine-learning component of the visual-programming tool, a game genre associated with the development program code; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the game genre.

5. The computer-implemented method of claim 1 , further comprising:

identifying, by the machine-learning component of the visual-programming tool, a role of a developer associated with the first indication; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the role of the developer.

6. The visual-programming tool configured to perform a method as claim 1 recites, the visual-programming tool comprising:

the GUI configured to present the multi-layer tree of predictions corresponding to at least a part of the multi-layer tree of nodes;

an input interface configured to receive input from one or more users of the visual-programming tool;

responsive to receiving input associated with the first decision in the development code via the input interface, the GUI configured to present the multi-layer tree of predictions in the GUI of the visual-programming tool; and

the input interface configured to receive an indication of focus on the chain of nodes among nodes of the multi-layer tree of predictions.

7. The visual-programming tool of claim 6 , wherein the GUI is further configured to change presentation of the multi-layer tree of predictions responsive to receiving the indication of focus on the chain of nodes.

8. The visual-programming tool of claim 6 , wherein:

the input interface is further configured to receive an indication of acceptance of a chain of nodes of the multi-layer tree of predictions via an input associated with the GUI; and

responsive to receiving the indication of acceptance of the chain of nodes, the GUI is further configured to cause presentation of one or more alternative nodes in subsequent layers of the multi-layer tree of predictions in the GUI, the presentation of the one or more alternative nodes including at least one of:

designated valid according to a determination that the one or more alternative nodes can be a part of the chain; or

designated invalid according to a determination that the one or more alternative nodes cannot be a part of the chain.

9. The visual-programming tool of claim 6 , wherein the input interface is further configured to recognize focus on the chain of nodes being indicated by maintenance of a mouse-button press while a mouse is dragged over the chain of nodes.

10. The visual-programming tool of claim 9 , wherein the input interface is further configured to recognize acceptance of the chain of nodes being indicated by finishing the mouse drag with release of the mouse-button press over a selected node of the chain of nodes.

11. The computer-implemented method of claim 1 , wherein the machine-learning component of the visual programming tool operates on program code features including at least one of input parameter type or output result type.

12. The computer-implemented method of claim 1 , further comprising:

determining, via the visual-programming tool, a first connecting line linking the first node with the second node in the chain of nodes;

providing, via the GUI associated with the visual-programming tool, a visual marking of the first connecting line;

receiving, via the GUI associated with the visual-programming tool, a third indication corresponding to a selection of the first chain of nodes; and

changing the visual marking of the first connecting line provided via the GUI to represent the selection.

13. A system comprising:

one or more processors;

one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations comprising:

storing, in cloud storage, shared program code accessible via a visual-programming tool incorporating machine-learning assisted autocomplete as a multi-layer tree of nodes with a logical connection signifying a relationship between at least one node of a first layer of the multi-layer tree of nodes and at least one node of a second layer of the multi-layer tree of nodes;

determining, at a visual-programming system and based at least on a machine-learning component of the visual-programming tool operating on heuristics of the multi-layer tree of nodes, a plurality of multi-layer trees of predictions based on the multi-layer tree of nodes, wherein individual multi-layer trees of predictions of the plurality of multi-layer trees of predictions include at least one chain of nodes formed with logical connections between layers of the individual multi-layer tree of predictions;

receiving, via a graphical user interface (GUI) associated with the visual-programming tool:

a first indication corresponding to a first decision in development program code;

determining, based on a machine learning model associated with the visual-programming tool and the visual-programming system, a first multi-layer tree of code predictions of the plurality of multi-layer trees of predictions based on the first indication of the first decision received at the GUI;

providing, via the GUI associated with the visual-programming tool, at least a first layer of nodes of the first multi-layer tree of code predictions based on the first indication of the first decision;

determining, via the machine-learning component, at least a first subset of the first layer of nodes not associated with the first indication;

providing, via the GUI associated with the visual-programming tool, at least a first visual marking of a presentation of at least one node of the first layer of nodes that corresponds to the first decision;

receiving, via the GUI associated with the visual-programming tool, a second indication corresponding to a second decision in the development program code, the second decision being made from among the nodes of the first layer of the first multi-layer tree of code predictions;

providing, via the GUI associated with the visual-programming tool, at least a first subset of the second layer of nodes of the first multi-layer tree of code predictions based on the second indication of the second decision, wherein the second indication represents focus on a second part of the chain of nodes from the first multi-layer tree of predictions;

determining, via the machine-learning component, at least a second subset of the second layer of nodes not associated with the second indication; and

providing, via the GUI associated with the visual-programming tool, at least a second visual marking of the presentation of at least one node of the second layer of nodes that corresponds to the second decision; and

wherein:

the at least one chain of nodes includes a first chain of nodes that includes the first node and the second node; and

responsive to receiving the second indication, the GUI is further configured to cause presentation of one or more alternative nodes in subsequent layers of the multi-layer tree of predictions in the GUI, the presentation of the one or more alternative nodes including at least one of:

brightened according to an increasing level of certainty that the one or more alternative nodes can be a part of the chain; or

dimmed according to a decreasing level of certainty that the one or more alternative nodes will be a part of the chain.

14. The system of claim 13 , the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations further comprising:

identifying, by the machine-learning component of the visual-programming tool, a type of graph associated with the multi-layer tree of nodes; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the type of graph.

15. The system of claim 13 , wherein at least part of the development program code is associated with programs including games, the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations further comprising:

identifying, by the machine-learning component of the visual-programming tool, a plurality of games having a similarity in game play;

determining, by the machine-learning component of the visual-programming tool, that the development program code shares the similarity; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the similarity.

16. The system of claim 13 , wherein at least part of the development program code is associated with programs including games, the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations further comprising:

identifying, by the machine-learning component of the visual-programming tool, a game genre associated with the development program code; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the game genre.

17. The system of claim 13 , the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations further comprising:

identifying, by the machine-learning component of the visual-programming tool, a role of a developer associated with the first indication; and

wherein the machine-learning component of the visual-programming tool associates the first multi-layer tree of code predictions with the role of the developer.

18. The system of claim 13 , wherein the first indication represents focus on a node from the first layer of nodes as part of a chain of nodes from the first multi-layer tree of code predictions;

the GUI is configured to change presentation of the first multi-layer tree of predictions responsive to receiving the indication of focus on the node from the first layer of nodes as part of the chain of nodes; and

the second indication includes representation of acceptance of the chain of nodes including the node from the first layer of nodes and a node from the second layer of nodes from the first multi-layer tree of predictions.

19. The system of claim 13 , the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations further comprising:

determining, via the visual-programming tool, a first connecting line linking the first node with the second node in the chain of nodes;

providing, via the GUI associated with the visual-programming tool, a visual marking of the first connecting line;

receiving, via the GUI associated with the visual-programming tool, a third indication corresponding to a selection of the first chain of nodes; and

changing the visual marking of the first connecting line provided via the GUI to represent the selection.

20. The system of claim 13 , the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to perform operations wherein:

the GUI is configured to present the first multi-layer tree of predictions corresponding to at least a part of the multi-layer tree of nodes;

an input interface is configured to receive input from one or more users of the visual-programming tool;

responsive to receiving input associated with the first decision in the development code via the input interface, the GUI is configured to present the first multi-layer tree of predictions in the GUI of the visual-programming tool;

the input interface is configured to receive an indication of focus on the chain of nodes among nodes of the first multi-layer tree of predictions;

the input interface is further configured to receive an indication of acceptance of a chain of nodes of the first multi-layer tree of predictions via an input associated with the GUI; and

responsive to receiving the indication of acceptance of the chain of nodes, the GUI is further configured to cause presentation of one or more alternative nodes in subsequent layers of the first multi-layer tree of predictions in the GUI, the presentation of the one or more alternative nodes including at least one of:

designated valid according to a determination that the one or more alternative nodes can be a part of the chain; or

designated invalid according to a determination that the one or more alternative nodes cannot be a part of the chain.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 28, 2020
From: MAN, SIU YEE
To: ELECTRONIC ARTS INC.
Reel/Frame 052779/0852 →
Continuity (1)
Related Publication 20210373857A1 · Dec 2, 2021