IP Library Granted Patent US 12,561,901
Granted Patent B2
US 12,561,901 · App. 18/041,837 · Granted Feb 24, 2026

Virtual reality based programming support system and method

Inventors: Akihiro Hori (Tokyo, JP); Masumi Kawakami (Tokyo, JP)
Assignee: Hitachi, Ltd.
G06T17/00G06T7/20G06V10/761G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,561,901
App. No.
18/041,837
Granted
Feb 24, 2026
Kind
B2
Abstract

To support creation of an editable information expression of a predetermined language in a virtual reality space, an information expression creation support device is provided. The device supports creation of an editable information expression source code of the predetermined language in the virtual reality space by tracking a movement trajectory of an action of a creator and selecting an information expression from candidates of the information expression, which are prepared in advance, based on an acquired movement trajectory of an action of the creator, and the selected information expression is arranged at a predetermined position.

Claims (47)

1 . A source code creation support device for supporting creation of a source code in a virtual reality space, comprising:

a processor;

a display that displays the source code; and

a storage device, wherein

the storage device stores a first table that contains, for each of a plurality of predetermined movement trajectories, a corresponding motion name and a second table that contains, for each motion name contained in the first table, a corresponding code content, and the processor:

acquires a movement trajectory of an action of a creator,

refers to the first table to determine if the movement trajectory of the action of the creator matches one of the plurality of predetermined movement trajectories and identifies the corresponding motion name contained in the first table,

refers to the second table to select the corresponding code content contained in the second table for the identified corresponding motion name,

arranges the selected corresponding code content at a predetermined position in the source code, wherein the predetermined position is determined by

analyzing the movement trajectory of the action of the creator and the source code already input within an approximate position designated by the creator based on the movement trajectory of the action of the creator, or

acquiring an input by the creator specifying where to arrange the selected corresponding code content in the source code already input, or

analyzing the source code already input, wherein

the processor further displays the source code with the selected corresponding code content arranged at the predetermined position in the source code to the display.

2 . The source code creation support device according to claim 1 , wherein,

the processor displays a virtual operation panel in a virtual reality space, wherein,

when the second table does not contain the corresponding code content for the motion name contained in the first table, the processor accepts an input of the code content based on the creator's voice or a controller operation.

3 . The source code creation support device according to claim 1 , wherein

the processor analyzes the source code already input, and when a source code candidate from a plurality of source code candidates, whose possibility of being input next is less than or equal to a predetermined value in terms of grammar of a predetermined language, the processor excludes the source code candidate with the low possibility of being input next from the plurality of source code candidates.

4 . The source code creation support device according to claim 1 , wherein

the processor gives a priority to a source code candidate whose possibility of being input next is greater than or equal to another source code candidate based on an analysis result of the source code already input and an analysis result of a source code created in the past.

5 . The source code creation support device according to claim 1 , wherein

the processor sets at least one coordinate axis for comparison based on a temporal change in coordinates on the at least one set coordinate axis in order to determine whether the movement trajectory of the action of the creator matches one of the predetermined movement trajectories having the corresponding motion name stored in the first table.

6 . The source code creation support device according to claim 1 , wherein

the processor outputs to the display a calculation result of determining whether the movement trajectory of the action of the creator matches one of the predetermined movement trajectories having the corresponding motion name stored in the first table.

7 . A source code creation support method to be executed by a computer to support creation of a source code of a predetermined language in a virtual reality space, the method comprising:

the computer:

acquiring a movement trajectory of an action of a creator,

referring to a first table that contains for each of a plurality of predetermined movement trajectories, a corresponding motion name, to determine if the movement trajectory of the action of the creator match one of the plurality of predetermined movement trajectories and identifies the corresponding motion name contained in the first table,

referring to a second table that contains for each motion name contained in the first table, a corresponding code content, to select the corresponding code content for the identified motion name,

arranging the selected corresponding code content at a predetermined position in the source code, wherein the predetermined position is determined by

analyzing the movement trajectory of the action of the creator and the source code already input within an approximate position designated by the creator based on the movement trajectory of the action of the creator, or

acquiring input by the creator specifying where to arrange the selected corresponding code content in the source code already input, or

analyzing the source code already input wherein,

the method further comprising the computer displaying the source code with the selected corresponding code content arranged at the predetermined position in the source code to a display.

8 . A storage device that stores a computer program, a first table, and a second table, wherein, the first table, the second table, and the computer program stored in the storage device may be transferred to and from a non-temporary storage medium, and

the computer program causes a computer;

to support creation of a source code in a virtual reality space,

the storage device storing the first table that contains for each of a plurality of predetermined movement trajectories, a corresponding motion name and the second table that contains for each motion name contained in the first table, a corresponding code content, and

the computer program causes a processor of the computer to execute:

a step of acquiring a movement trajectory of an action of a creator,

a step of referring to the first table to determine if the movement trajectory of the action of the creator matches one of the plurality of predetermined movement trajectories and identifies the corresponding motion name contained in the first table,

a step of referring to the second table to select the corresponding code content for the identified corresponding motion name,

a step of arranging the selected corresponding code content at a predetermined position in the source code, wherein the predetermined position is determined by

analyzing the movement trajectory of the action of the creator and a source code already input within an approximate position designated by the creator based on the movement trajectory of the action of the creator, or

acquiring input by the creator specifying where to arrange the selected corresponding code content in the source code already input, or

analyzing the source code already input, wherein,

the computer program further causes the processor of the computer to execute a step of displaying the source code with the selected corresponding code content arranged at the predetermined position in the source code to a display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2024
From: HORI, AKIHIRO; KAWAKAMI, MASUMI
To: HITACHI, LTD.
Reel/Frame 066448/0043 →
Priority Claims (1)
JP 2020-138277 · Aug 18, 2020 · national
Continuity (1)
Related Publication 20240013481A1 · Jan 11, 2024
References Cited (18)
US 5481625A · Suzuki · 1996 [cited by applicant]
US 10801841B1 · Rintoul · 2020 [cited by examiner]
US 20070139370A1 · Lu · 2007 [cited by examiner]
US 20150220265A1 · Takahashi · 2015 [cited by applicant]
US 20170242581A1 · Couelier et al. · 2017 [cited by applicant]
US 20180136721A1 · Alleaume et al. · 2018 [cited by applicant]
US 20190094534A1 · Yamaura et al. · 2019 [cited by applicant]
JP H05290221A · 1993 [cited by applicant]
JP 2012160204A · 2012 [cited by applicant]
JP 2013114431A · 2013 [cited by applicant]
JP 2015148946A · 2015 [cited by applicant]
JP 2018081694A · 2018 [cited by applicant]
JP 2019061590A · 2019 [cited by applicant]
JP 2019512765A · 2019 [cited by applicant]
WO WO2008018274A1 · 2008 [cited by applicant]
International Search Report, mailed May 18, 2021, for International Application No. PCT/JP2021/008008. [cited by applicant]
Notice of Reasons for Refusal, mailed Jan. 9, 2024, for Japanese Application No. 2020-138277 (with English translation). [cited by applicant]
Onishi, et al., Operation Interface in Immersive VR Programming Environment, The Institute of Electronics, Information and Communication Engineers, IEICE Technical Report, pp. 85-88 (2019) (with English translation). [cited by applicant]