IP Library › Granted Patent US 12,277,407
Granted Patent B2
US 12,277,407 · App. 18/125,263 · Granted Apr 15, 2025

Programming aiding method in a visual programming interface

Inventors: Tatsuya Hasebe (Tokyo, JP); Erika Katayama (Tokyo, JP); Makoto Onodera (Tokyo, JP)
Assignee: Hitachi, Ltd.
G06F8/34
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,277,407
App. No.
18/125,263
Filed
Mar 23, 2023
Granted
Apr 15, 2025
Kind
B2
Art Unit
2192
USPC
717/105
Abstract

A programming aiding apparatus that includes a visual program storage unit performs a programming aiding method. In the apparatus, a training data generating unit edits the visual program. A graph learning unit trains a graph learning model that receives a feature of a node and a connection of the node in the visual program as an input, among the input data, and outputs a likelihood of a feature of a node to be added to in the visual program. An added node selecting unit calculates a likelihood of a feature of a node by using data of a visual program as input data input to the learning model trained by the graph learning unit, and selects a node to be added based on the likelihood.

Claims (32)

1. A programming aiding apparatus that aids visual programming, the programming aiding apparatus comprising:

a visual program storage unit that stores therein a visual program created in past;

a training data generating unit that edits the visual program stored in the visual program storage unit to generate training data including a pair of input data that is input to a learning model and target data, the pair of input data including information of a directed graph of the visual program, and the target data including node features serving as ground truth for training;

a graph learning unit that trains a graph learning model that receives the feature of a node and a connection of the node in the visual program as an input, among the pair of input data, and outputs a likelihood of the feature of the node to be added to in the visual program; and

an added node selecting unit that receives the likelihood output by the graph learning unit, and selects a node to be added based on the likelihood.

2. The programming aiding apparatus according to claim 1 ,

further comprising a node feature calculating unit that converts node data including a type of the node, a name of the node, and a property value of the node into the feature, wherein

the node feature calculating unit calculates node feature data from the pair of input data and outputs the node feature data to the graph learning unit, and

the graph learning unit trains the graph learning model using data including the node feature as an input.

3. The programming aiding apparatus according to claim 1 ,

further comprising an edge feature calculating unit that converts data including handle information of a target and a source of an edge, and an edge type into features, the handle information and the edge type being included in data of an edge that connects nodes in the visual program, wherein

the edge feature calculating unit calculates an edge feature data from the pair of input data, and outputs the edge feature data to the graph learning unit, and

the graph learning unit trains the graph learning model using data including the edge feature as an input.

4. The programming aiding apparatus according to claim 1 ,

further comprising a metadata feature calculating unit that converts metadata including tag information including a name and a sequence of keyword character strings that are associated with the visual program into the feature, wherein

the metadata feature calculating unit calculates a metadata feature from the pair of input data, and outputs the metadata feature to the graph learning unit, and

the graph learning unit trains the graph learning model using data including the metadata feature as an input.

5. The programming aiding apparatus according to claim 1 , wherein the training data generating unit sets a sub graph of a part of the visual program stored in the visual program storage unit as input data, and sets some of nodes and edges connected to the sub graph as target data.

6. The programming aiding apparatus according to claim 1 , wherein

the training data generating unit sets the visual program currently being created at a certain point of time as the input data, and sets the node and an edge having been added to the visual program currently being created as the target data, based on a creation history of the visual program stored in the visual program storage unit.

7. A programming aiding method for aiding visual programming, the programming aiding method comprising:

(a) collecting and storing a visual program created in past;

(b) generating training data including a pair of input data that is input to a learning model and target data, the pair of input data including information of a directed graph of the visual program, and the target data including the node features serving as ground truth for training;

(c) training a graph learning model that receives the feature of the node and a connection of the node in the visual program as an input, among the pair of input data, and outputs a likelihood of the feature of the node to be added to in the visual program;

(d) causing the trained graph learning model to calculate, using the graph learning model having been trained, to output the likelihood of the feature of the node, by using data of the visual program currently being created by a user as input data; and

(e) outputting a node to be added based on the likelihood.

8. The programming aiding method according to claim 7 , wherein

in (d), when the user makes an operation of adding a node to the visual program, the likelihood of the feature of the node is calculated using data of the visual program at the time when the operation is made, as input data.

9. The programming aiding method according to claim 8 ,

further comprising (f) sorting information based on the likelihood calculated by the graph learning model, selecting one or more nodes having a high likelihood that is higher than a threshold value, and presenting the one or more nodes having the high likelihood to the user as candidates as the node to be added.

10. The programming aiding method according to claim 9 , wherein

in (f), a degree of match between the directed graph of the visual program resultant of adding a presented candidate node to the visual program and the visual program created in past is calculated, and when there is a past visual program matching by a degree higher a threshold, one or more matched visual programs are presented to a user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2023
From: HASEBE, TATSUYA; KATAYAMA, ERIKA; ONODERA, MAKOTO
To: HITACHI, LTD.
Reel/Frame 063200/0263 →
Priority Claims (1)
JP 2022-072965 · Apr 27, 2022 · national
Continuity (1)
Related Publication 20230350647A1 · Nov 2, 2023
References Cited (6)
US 20170316114A1 · Bourhani · 2017 [cited by examiner]
US 20180357152A1 · Browne · 2018 [cited by examiner]
US 20200264847A1 · Nasu · 2020 [cited by examiner]
US 20210373857A1 · Man · 2021 [cited by examiner]
JP 201924935A · 2019 [cited by applicant]
Feng Xia et al., Graph Learning: A Survey, 2021 IEEE, [Retrieved on Dec. 26, 2024]. Retrieved from the internet: <URL: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9416834> 19 Pages (109-127) (Year: 2021). [cited by examiner]