IP Library › Granted Patent US 12,688,112
Granted Patent B2
US 12,688,112 · App. 18/194,523 · Granted Jul 21, 2026

Quality analysis of visual programming scripting language using machine learning techniques

Inventors: Alexander Senchenko (Langley, CA); Milan Culibrk (Coquitlam, CA)
Assignee: Electronic Arts Inc.
G06F11/3616G06F8/427G06F8/71G06F11/3608
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,688,112
App. No.
18/194,523
Filed
Mar 31, 2023
Granted
Jul 21, 2026
Kind
B2
Art Unit
2199
USPC
717/126
Abstract

A quality analysis tool for visual-programming scripting languages uses machine learning to process changes from visual-programming environments. The quality analysis tool can receive data associated with a code submission via a visual-programming scripting language, process the data to identify features in the data that correspond to previously identified defects, apply a pattern matching algorithm to the identified features, determine a risk prediction based on a learned pattern recognition model associated with a pattern in the features, and transmit a notice of predicted risk. The quality analysis tool can train models for use with visual-programming scripting languages and visual-programming environments.

Claims (74)

1 . A system comprising:

one or more processors;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to:

receive data associated with a code submission formatted in a visual-programming scripting language used to create game logic using visual-scripting cards, wherein the visual-scripting cards are disconnected from textual code;

process the data to identify features in the data that correspond to previously identified defects, wherein processing the data includes at least parsing at least one debugging (DBX) file received from a DBX-type tool into at least one schematic representation, the at least one schematic representation including at least an anchor point that pins connections to locations, a proxied node that can be referenced by connections to connect a node that is elsewhere in a graph, and a group that groups nodes together;

apply a pattern-matching algorithm to the identified features;

identify defects associated with the identified features;

determine a context of one or more patterns associated with the defects based at least in part on a longest simplified path in a graph structure of the visual-programming scripting language, the longest simplified path representing a number of hops affecting a plurality of nodes, wherein nodes having higher connection counts are flagged with greater risk;

employ target labels to communicate risk associated with the one or more patterns associated with the defects;

determine predicted risk based on a learned pattern recognition model associated with a pattern in the identified features and the one or more patterns associated with the defects; and

transmit a notice of the predicted risk to at least one of a quality assurance device or a developer device, wherein the notice includes advice or recommendations related to the identified defects.

2 . The system of claim 1 , wherein the data includes at least one of:

code changes;

reported defects; or

feature briefs.

3 . The system of claim 1 , wherein the processing includes at least one of:

cleaning the data;

normalizing the data; or

transforming the data.

4 . The system of claim 1 , wherein the pattern-matching algorithm includes a sparse-sensitive algorithm including a Śliwerski, Zimmerman, and Zeller (SZZ) analysis.

5 . The system of claim 1 , wherein a source of the data includes at least one of:

a version control system;

an issue tracking tool; or

a semantic data source.

6 . The system of claim 5 , wherein the processing further includes dynamically evaluating the data by applying an algorithm to identify a source of visual programming associated with the predicted risk.

7 . The system of claim 6 , wherein the algorithm to identify the source of visual programming associated with the predicted risk is configured to:

determine the predicted risk further based at least in part on version control history; and

create a blended predicted risk by combining the predicted risk from version control and the predicted risk based on the learned pattern recognition model.

8 . The system of claim 1 , the one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, configure the system to:

train a machine learning model as the learned pattern recognition model using the identified features.

9 . A computer-implemented method comprising:

receiving data associated with a code submission formatted in a visual-programming scripting language used to create game logic using visual-scripting cards, wherein the visual-scripting cards are disconnected from textual code;

processing the data to identify features in the data that represent defects, wherein processing the data includes at least parsing at least one debugging (DBX) file received from a DBX-type tool into at least one schematic representation, the at least one schematic representation including at least a shortcut;

extracting the identified features from the processed data that have been identified as representing defects;

identifying defects associated with the identified features;

determining a context of one or more patterns associated with the defects based at least in part on a longest simplified path in a graph structure of the visual-programming scripting language, the longest simplified path representing a number of hops affecting a plurality of nodes, wherein nodes having higher connection counts are flagged with greater risk;

employing target labels to communicate risk associated with the one or more patterns associated with the defects;

determining a predicted risk based on a learned model associated with the identified features and the one or more patterns associated with the defects; and

transmitting a notice of the predicted risk to at least one of a quality assurance device or a developer device, wherein the notice includes advice or recommendations related to the identified defects.

10 . The computer-implemented method of claim 9 , wherein the data includes at least one of:

code changes;

reported defects; or

feature briefs.

11 . The computer-implemented method of claim 9 , wherein the processing includes at least one of:

cleaning the data;

normalizing the data; or

transforming the data.

12 . The computer-implemented method of claim 9 , wherein a source of the data includes at least one of:

a version control system;

an issue tracking tool; or

a semantic data source.

13 . The computer-implemented method of claim 12 , wherein the processing further includes dynamically evaluating the data by applying an algorithm to identify a source of visual programming associated with the predicted risk.

14 . The computer-implemented method of claim 13 , wherein the algorithm is configured to identify the predicted risk from version control history.

15 . The computer-implemented method of claim 9 , further comprising:

training a machine learning model using the extracted identified features.

16 . The computer-implemented method of claim 9 , further comprising:

applying a pattern-matching algorithm to the identified features; and

wherein determining the predicted risk is further based on a pattern from the pattern-matching algorithm.

17 . The computer-implemented method of claim 16 , wherein the pattern-matching algorithm includes a sparse-sensitive algorithm including a Śliwerski, Zimmerman, and Zeller (SZZ) analysis.

18 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving data associated with a code submission formatted in a visual-programming scripting language used to create game logic using visual-scripting cards, wherein the visual-scripting cards are disconnected from textual code;

processing the data to identify features in the data, wherein processing the data includes at least parsing at least one debugging (DBX) file received from a DBX-type tool into at least one schematic representation, the at least one schematic representation including at least an annotation;

extracting identified features from the processed data;

identifying defects associated with the identified features;

determining a context of one or more patterns associated with the defects based at least in part on a longest simplified path in a graph structure of the visual-programming scripting language, the longest simplified path representing a number of hops affecting a plurality of nodes, wherein nodes having higher connection counts are flagged with greater risk;

employing target labels to communicate risk associated with the one or more patterns associated with the defects;

determining a risk prediction based on a learned model associated with the identified features and the one or more patterns associated with the defects; and

transmitting a notice of the predicted risk to at least one of a quality assurance device or a developer device, wherein the notice includes advice or recommendations related to the identified defects.

19 . The one or more non-transitory computer-readable media of claim 18 , wherein the processing includes at least one of:

parsing the data;

transforming the data; or

dynamically evaluating the data by applying an algorithm to identify a source of visual programming associated with the predicted risk.

20 . The one or more non-transitory computer-readable media of claim 18 , the operations further comprising:

training a machine learning model using the extracted identified features and the target labels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 19, 2023
From: SENCHENKO, ALEXANDER; CULIBRK, MILAN
To: ELECTRONIC ARTS INC.
Reel/Frame 063372/0173 →
Continuity (1)
Related Publication 20240330149A1 · Oct 3, 2024
References Cited (30)
US 6199199B1 · Johnston · 2001 [cited by examiner]
US 8166459B2 · Suenbuel · 2012 [cited by examiner]
US 10740216B1 · Parent · 2020 [cited by examiner]
US 11573772B2 · Kogelman · 2023 [cited by examiner]
US 11880293B1 · Hendrix · 2024 [cited by examiner]
US 12079689B1 · Rao · 2024 [cited by examiner]
US 20160291944A1 · Jefferson · 2016 [cited by examiner]
US 20180150742A1 · Woulfe · 2018 [cited by examiner]
US 20210349910A1 · Chadha · 2021 [cited by examiner]
US 20210374229A1 · Kumar · 2021 [cited by examiner]
US 20230057720A1 · Aradhya · 2023 [cited by examiner]
US 20230214192A1 · Makhija · 2023 [cited by examiner]
US 20230297489A1 · Mano · 2023 [cited by examiner]
US 20230305942A1 · Thome · 2023 [cited by examiner]
US 20230350647A1 · Hasebe · 2023 [cited by examiner]
US 20230385042A1 · Obando Chacon · 2023 [cited by examiner]
US 20240193072A1 · Kanta · 2024 [cited by examiner]
US 20240256425A1 · Thome · 2024 [cited by examiner]
Qingkai Shi, et al. 2021. Path-sensitive sparse analysis without path conditions. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI 2021). Associatio… [cited by examiner]
Jacek Śliwerski, Thomas Zimmermann, and Andreas Zeller. 2005. When do changes induce fixes? SIGSOFT Softw. Eng. Notes 30, 4 (Jul. 2005), 1-5. (Year: 2005). [cited by examiner]
K. Howland, J. Good and J. Robertson, “Script Cards: A Visual Programming Language for Games Authoring by Young People,” Visual Languages and Human-Centric Computing (VL/HCC'06), Brighton, UK, 2006, pp. 181-186 (Year: 2… [cited by examiner]
Chen, et al., “Applications of Psychological Science for Actionable Analytics,” Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Eng… [cited by applicant]
Kondo, et al., “The Impact of Context Metrics on Just-In-Time Defect Prediction,” Empirical Software Engineering, No. 25, pp. 890-393. [cited by applicant]
McIntosh & Kamei, “Are Fix-Inducing Changes a Moving Target? A Longitudinal Case Study of Just-In-Time Defect Prediction,” Proceedings of the 40th International Conference on Software Engineering, 2018, 1 pg. [cited by applicant]
Menzies, et al., “Implications of Ceiling Effects in Defect Predictors,” Proceedings of the 4th International Workshop on Predictor Models in Software Engineering, 2008, pp. 47-54. [cited by applicant]
Parnin & Orso, “Are Automated Debugging Techniques Actually Helping Programmers?,” Proceedings of the 2011 International Symposium on Software Testing and Analysis, 2011, pp. 199-209. [cited by applicant]
Shrikanth, et al., “Early Life Cycle Software Defect Prediction. Why? How?,” 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021, pp. 448-459. [cited by applicant]
Tantithamthavorn, et al., “The Impact of Automated Parameter Optimization on Defect Prediction Models,” IEEE Transactions on Software Engineering, vol. 45, No. 7, 2018, pp. 683-711. [cited by applicant]
Tantithamthavorn, et al., “The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models,” IEEE Transactions on Software Engineering, vol. 46, No. 11, 2008, pp. 1200-1219. [cited by applicant]
Wood, T., “Precision and Recall,” Retrieved from DeepAI website on Aug. 11, 2020 at <<https://deepai.org/machine-learning-glossary-and-terms/precision-and-recall>>, 14 pgs. [cited by applicant]