IP Library › Granted Patent US 12,242,823
Granted Patent B2
US 12,242,823 · App. 17/812,588 · Granted Mar 4, 2025

User interface controls for visual software modification

Inventors: Alvin Wang (Cambridge, MA); Jonathan Wang (Cambridge, MA); Mark Kalinich (Cambridge, MA)
Assignee: Watershed Informatics, Inc.
G06F8/38G06F3/0484G06F8/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,823
App. No.
17/812,588
Filed
Jul 14, 2022
Granted
Mar 4, 2025
Kind
B2
Art Unit
2171
USPC
715/763
Abstract

Techniques for visual software development are disclosed, including: generating, by a visual code editor, a data model of code elements in a set of source code; based on the data model, generating an adjustable graphical user interface (GUI) control corresponding to a subset of the set of source code; presenting the adjustable GUI control in a GUI of the visual code editor; and responsive to user input in the GUI to adjust the adjustable GUI control, modifying the set of source code according to the user input and executing the modified set of source code.

Claims (83)

1. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating, by a visual code editor, (1) a data model of a plurality of code elements in a set of source code and (2) a runtime code environment (RCE) for execution of the set of source code;

based on the set of source code, the data model, and the RCE, generating a first adjustable graphical user interface (GUI) form control;

presenting the first adjustable GUI form control in alignment with the set of source code in a GUI of the visual code editor; and

responsive to first user input in the GUI to adjust the first adjustable GUI form control:

modifying the set of source code and the data model according to the first user input;

updating the presentation of the set of source code in the GUI to reflect the modified set of source code; and

executing the modified set of source code, thereby updating the RCE;

wherein presenting the first adjustable GUI form control in alignment with the set of source code in the GUI of the visual code editor includes displaying the first adjustable GUI form control and a second adjustable GUI form control horizontally adjacent to each other in the GUI, the first and second adjustable GUI controls both being displayed vertically adjacent to the set of the source code.

2. The one or more non-transitory computer-readable media of claim 1 , wherein generating the data model of the plurality of code elements comprises:

parsing the set of source code into an abstract syntax tree (AST), a plurality of nodes of the AST corresponding to respective code elements in the plurality of code elements.

3. The one or more non-transitory computer-readable media of claim 1 , wherein:

the GUI comprises a web page,

the first adjustable GUI form control is a hypertext markup language (HTML) form control, and

generating the first adjustable GUI form control is performed by a browser configured to render the web page.

4. The one or more non-transitory computer-readable media of claim 1 , the operations further comprising:

generating a serialized object comprising a state of the first adjustable GUI form control during a first session of the visual code editor; and

based on the serialized object, restoring the state of the first adjustable GUI form control in a second session of the visual code editor.

5. The one or more non-transitory computer-readable media of claim 1 , the operations further comprising:

while presenting the first adjustable GUI form control in the GUI of the visual code editor, detecting a change in the set of source code; and

adjusting the first adjustable GUI form control according to the change in the set of source code.

6. The one or more non-transitory computer-readable media of claim 1 wherein executing the modified set of source code is performed in response to selection of a code execution element in the GUI.

7. The one or more non-transitory computer-readable media of claim 1 wherein generating the first adjustable GUI form control includes:

requesting variable introspection from the RCE with reference to the data model; and

in response to obtaining updated variable values from the RCE, updating the data model and using the updated variable values in the first adjustable GUI form control.

8. The one or more non-transitory computer-readable media of claim 1 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code includes displaying the set of source code in the GUI with the modifications highlighted.

9. The one or more non-transitory computer-readable media of claim 7 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code further includes fading out the highlighting a predetermined amount of time after displaying the set of source code in the GUI with the modifications highlighted.

10. A system comprising:

at least one device including a hardware processor;

the system being configured to perform operations comprising:

generating, by a visual code editor, (1) a data model of a plurality of code elements in a set of source code and (2) a runtime code environment (RCE) for execution of the set of source code;

based on the set of source code, the data model, and the RCE, generating a first adjustable graphical user interface (GUI) form control;

presenting the first adjustable GUI form control in alignment with the set of source code in a GUI of the visual code editor; and

responsive to first user input in the GUI to adjust the first adjustable GUI form control:

modifying the set of source code and the data model according to the first user input;

updating the presentation of the set of source code in the GUI to reflect the modified set of source code; and

executing the modified set of source code, thereby updating the RCE;

wherein presenting the first adjustable GUI form control in alignment with the set of source code in the GUI of the visual code editor includes displaying the first adjustable GUI form control and a second adjustable GUI form control horizontally adjacent to each other in the GUI, the first and second adjustable GUI controls both being displayed vertically adjacent to the set of the source code.

11. The system of claim 10 , wherein generating the data model of the plurality of code elements comprises:

parsing the set of source code into an abstract syntax tree (AST), a plurality of nodes of the AST corresponding to respective code elements in the plurality of code elements.

12. The system of claim 10 , wherein:

the GUI comprises a web page,

the first adjustable GUI form control is a hypertext markup language (HTML) form control, and

generating the first adjustable GUI form control is performed by a browser configured to render the webpage.

13. The system of claim 10 , the operations further comprising:

generating a serialized object comprising a state of the first adjustable GUI form control during a first session of the visual code editor; and

based on the serialized object, restoring the state of the first adjustable GUI form control in a second session of the visual code editor.

14. The system of claim 10 , the operations further comprising:

while presenting the first adjustable GUI form control in the GUI of the visual code editor, detecting a change in the set of source code; and

adjusting the first adjustable GUI form control according to the change in the set of source code.

15. The system of claim 10 wherein executing the modified set of source code is performed in response to selection of a code execution element in the GUI.

16. The system of claim 10 wherein generating the first adjustable GUI form control includes:

requesting variable introspection from the RCE with reference to the data model; and

in response to obtaining updated variable values from the RCE, updating the data model and using the updated variable values in the first adjustable GUI form control.

17. The system of claim 10 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code includes displaying the set of source code in the GUI with the modifications highlighted.

18. The system of claim 10 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code further includes fading out the highlighting a predetermined amount of time after displaying the set of source code in the GUI with the modifications highlighted.

19. A method comprising:

generating, by a visual code editor, (1) a data model of a plurality of code elements in a set of source code and (2) a runtime code environment (RCE) for execution of the set of source code;

based on the set of source code, the data model, and the RCE, generating a first adjustable graphical user interface (GUI) form control;

presenting the first adjustable GUI form control in alignment with the set of source code in a GUI of the visual code editor; and

responsive to first user input in the GUI to adjust the first adjustable GUI form control:

modifying the set of source code and the data model according to the first user input;

updating the presentation of the set of source code in the GUI to reflect the modified set of source code; and

executing the modified set of source code, thereby updating the RCE;

wherein presenting the first adjustable GUI form control in alignment with the set of source code in the GUI of the visual code editor includes displaying the first adjustable GUI form control and a second adjustable GUI form control horizontally adjacent to each other in the GUI, the first and second adjustable GUI controls both being displayed vertically adjacent to the set of the source code.

20. The method of claim 19 , wherein generating the data model of the plurality of code elements comprises:

parsing the set of source code into an abstract syntax tree (AST), a plurality of nodes of the AST corresponding to respective code elements in the plurality of code elements.

21. The method of claim 19 , wherein:

the GUI comprises a web page,

the first adjustable GUI form control is a hypertext markup language (HTML) form control, and

generating the first adjustable GUI form control is performed by a browser configured to render the webpage.

22. The method of claim 19 , further comprising:

generating a serialized object comprising a state of the first adjustable GUI form control during a first session of the visual code editor; and

based on the serialized object, restoring the state of the first adjustable GUI form control in a second session of the visual code editor.

23. The method of claim 19 , further comprising:

while presenting the first adjustable GUI form control in the GUI of the visual code editor, detecting a change in the set of source code; and

adjusting the first adjustable GUI form control according to the change in the set of source code.

24. The method of claim 19 wherein executing the modified set of source code is performed in response to selection of a code execution element in the GUI.

25. The method of claim 19 wherein generating the first adjustable GUI form control includes:

requesting variable introspection from the RCE with reference to the data model; and

in response to obtaining updated variable values from the RCE, updating the data model and using the updated variable values in the first adjustable GUI form control.

26. The method of claim 19 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code includes displaying the set of source code in the GUI with the modifications highlighted.

27. The method of claim 19 wherein updating the presentation of the set of source code in the GUI to reflect the modified set of source code further includes fading out the highlighting a predetermined amount of time after displaying the set of source code in the GUI with the modifications highlighted.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 14, 2022
From: WANG, ALVIN; WANG, JONATHAN; KALINICH, MARK
To: WATERSHED INFORMATICS, INC.
Reel/Frame 060509/0239 →
Continuity (2)
Provisional Application 63222011 · Jul 15, 2021
Related Publication 20230019933A1 · Jan 19, 2023
References Cited (22)
US 7316003B1 · Dulepet et al. · 2008 [cited by applicant]
US 11003308B1 · Dryer · 2021 [cited by examiner]
US 11513778B1 · Graves · 2022 [cited by examiner]
US 20070055966A1 · Waddington et al. · 2007 [cited by applicant]
US 20090077091A1 · Khen · 2009 [cited by examiner]
US 20090125877A1 · Kuzsma, Jr. · 2009 [cited by examiner]
US 20120324423A1 · Khan · 2012 [cited by examiner]
US 20130014093A1 · Lee · 2013 [cited by examiner]
US 20130239087A1 · Fox et al. · 2013 [cited by applicant]
US 20140047413A1 · Sheive et al. · 2014 [cited by applicant]
US 20150347094A1 · Lattner · 2015 [cited by examiner]
US 20170123763A1 · Childs · 2017 [cited by examiner]
US 20180024731A1 · Sanches et al. · 2018 [cited by applicant]
US 20180107478A1 · Codato · 2018 [cited by examiner]
US 20190213102A1 · Raviv et al. · 2019 [cited by applicant]
US 20190324728A1 · Evans et al. · 2019 [cited by applicant]
US 20190369969A1 · Donohoe · 2019 [cited by examiner]
US 20220156062A1 · Kroth · 2022 [cited by examiner]
US 20220317978A1 · Barik · 2022 [cited by examiner]
US 20230289148A1 · Wang et al. · 2023 [cited by applicant]
Subset, Wikipedia (https://en.wikipedia.org/wiki/Subset). [cited by examiner]
International Search Report and Written Opinion from corresponding International Application No. PCT/US2022/073786 dated Nov. 10, 2022. [cited by applicant]