IP Library Granted Patent US 12,260,306
Granted Patent B2
US 12,260,306 · App. 17/891,350 · Granted Mar 25, 2025

Textual explanations for abstract syntax trees with scored nodes

Inventors: Kenyu Kobayashi (Lausanne, CH); Arno Schneuwly (Effretikon, CH); Renata Khasanova (Zurich, CH); Matteo Casserini (Zurich, CH); Felix Schmidt (Baden-Dattwil, CH)
Assignee: Oracle International Corporation
G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,306
App. No.
17/891,350
Granted
Mar 25, 2025
Kind
B2
Abstract

Herein is a machine learning (ML) explainability (MLX) approach in which a natural language explanation is generated based on analysis of a parse tree such as for a suspicious database query or web browser JavaScript. In an embodiment, a computer selects, based on a respective relevance score for each non-leaf node in a parse tree of a statement, a relevant subset of non-leaf nodes. The non-leaf nodes are grouped in the parse tree into groups that represent respective portions of the statement. Based on a relevant subset of the groups that contain at least one non-leaf node in the relevant subset of non-leaf nodes, a natural language explanation of why the statement is anomalous is generated.

Claims (72)

1. A method comprising:

selecting, based on a respective relevance score for each non-leaf node in a parse tree of a semantically valid logic statement, a relevant subset of non-leaf nodes;

grouping non-leaf nodes in the parse tree into a plurality of groups that represent respective portions of the semantically valid logic statement;

generating, from the parse tree, one or more disconnected trees by:

replacing each group of a relevant subset of the plurality of groups that contain at least one non-leaf node in the relevant subset of non-leaf nodes with a group node, and

retaining only the group nodes; and

generating, based on the one or more disconnected trees, an explanation of why the semantically valid logic statement is anomalous;

wherein the method is performed by one or more computers.

2. The method of claim 1 wherein:

a grammar comprises a plurality of distinct nonterminal symbols;

each non-leaf node in the parse tree corresponds to a respective nonterminal symbol in the plurality of distinct nonterminal symbols;

each nonterminal symbol in the plurality of distinct nonterminal symbols corresponds to exactly one respective class in a plurality of distinct classes;

each class in the plurality of distinct classes corresponds to one or more nonterminal symbols in the plurality of distinct nonterminal symbols;

at least one class in the plurality of distinct classes corresponds to multiple nonterminal symbols in the plurality of distinct nonterminal symbols;

said grouping the non-leaf nodes in the parse tree is based on the plurality of distinct classes.

3. The method of claim 2 wherein:

the plurality of distinct classes comprises of one or more major classes and one or more minor classes;

said generating the explanation comprises printing portions of the semantically valid logic statement that correspond to the one or more minor classes.

4. The method of claim 2 wherein:

said non-leaf nodes in the parse tree comprise a parent non-leaf node and a child non-leaf node;

said grouping the non-leaf nodes in the parse tree comprises:

detecting that the child non-leaf node is adjacent to the parent non-leaf node, and

detecting that the child non-leaf node and the parent non-leaf node correspond to respective nonterminal symbols that correspond to a same class in the plurality of distinct classes.

5. The method of claim 4 wherein said generating the explanation comprises for each tree in the one or more disconnected trees, generating, for each branch node in the one or more disconnected trees, a respective distinct natural language sentence.

6. The method of claim 5 wherein the natural language sentence of a particular disconnected tree of the one or more disconnected trees identifies exactly three classes of the plurality of distinct classes.

7. The method of claim 6 further comprising visually coloring, in the natural language sentence, each class of the exactly three classes with a respective distinct color.

8. The method of claim 6 wherein:

the particular disconnected tree contains, at distinct respective depths, three or more group nodes;

the plurality of distinct classes consists of one or more major classes and one or more minor classes;

the exactly three classes are:

a first class of the group node at the least depth of the three or more group nodes,

a second class of the three or more group nodes that is in the one or more major classes, and

a third class of the group node at the most depth of the three or more group nodes.

9. The method of claim 8 wherein the natural language sentence identifies the exactly three classes in an ordering that is the first class, the second class, and the third class.

10. The method of claim 8 wherein:

the exactly three classes correspond to group nodes in a particular path in the particular disconnected tree;

the particular path contains a particular group node that corresponds to a class that is not one of the exactly three classes;

the particular group node is selected from a group consisting of:

a class of the one or more major classes, and

a class of the one or more minor classes.

11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:

selecting, based on a respective relevance score for each non-leaf node in a parse tree of a semantically valid logic statement, a relevant subset of non-leaf nodes;

grouping non-leaf nodes in the parse tree into a plurality of groups that represent respective portions of the semantically valid logic statement;

generating, from the parse tree, one or more disconnected trees by:

replacing each group of a relevant subset of the plurality of groups that contain at least one non-leaf node in the relevant subset of non-leaf nodes with a group node, and

retaining only the group nodes; and

generating, based on the one or more disconnected trees, an explanation of why the semantically valid logic statement is anomalous.

12. The one or more non-transitory computer-readable media of claim 11 wherein:

a grammar comprises a plurality of distinct nonterminal symbols;

each non-leaf node in the parse tree corresponds to a respective nonterminal symbol in the plurality of distinct nonterminal symbols;

each nonterminal symbol in the plurality of distinct nonterminal symbols corresponds to exactly one respective class in a plurality of distinct classes;

each class in the plurality of distinct classes corresponds to one or more nonterminal symbols in the plurality of distinct nonterminal symbols;

at least one class in the plurality of distinct classes corresponds to multiple nonterminal symbols in the plurality of distinct nonterminal symbols;

said grouping the non-leaf nodes in the parse tree is based on the plurality of distinct classes.

13. The one or more non-transitory computer-readable media of claim 12 wherein:

the plurality of distinct classes comprises of one or more major classes and one or more minor classes;

said generating the explanation comprises printing portions of the semantically valid logic statement that correspond to the one or more minor classes.

14. The one or more non-transitory computer-readable media of claim 12 wherein:

said non-leaf nodes in the parse tree comprise a parent non-leaf node and a child non-leaf node;

said grouping the non-leaf nodes in the parse tree comprises:

detecting that the child non-leaf node is adjacent to the parent non-leaf node, and

detecting that the child non-leaf node and the parent non-leaf node correspond to respective nonterminal symbols that correspond to a same class in the plurality of distinct classes.

15. The one or more non-transitory computer-readable media of claim 14 wherein said generating the explanation comprises for each tree in the one or more disconnected trees, generating, for each branch node in the one or more disconnected trees, a respective distinct natural language sentence.

16. The one or more non-transitory computer-readable media of claim 15 wherein the natural language sentence of a particular disconnected tree of the one or more disconnected trees identifies exactly three classes of the plurality of distinct classes.

17. The one or more non-transitory computer-readable media of claim 16 wherein:

the particular disconnected tree contains, at distinct respective depths, three or more group nodes;

the plurality of distinct classes consists of one or more major classes and one or more minor classes;

the exactly three classes are:

a first class of the group node at the least depth of the three or more group nodes,

a second class of the three or more group nodes that is in the one or more major classes, and

a third class of the group node at the most depth of the three or more group nodes.

18. The one or more non-transitory computer-readable media of claim 16 wherein the instructions further cause visually coloring, in the natural language sentence, each class of the exactly three classes with a respective distinct color.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2022
From: KOBAYASHI, KENYU; SCHNEUWLY, ARNO; KHASANOVA, RENATA; CASSERINI, MATTEO; SCHMIDT, FELIX
To: ORACLE INTERNATIONAL CORPORATION
Reel/Frame 061234/0176 →
Continuity (1)
Related Publication 20240061997A1 · Feb 22, 2024
References Cited (27)
US 8341096B2 · Fisher · 2012 [cited by examiner]
US 20150046492A1 · Balachandran · 2015 [cited by examiner]
US 20150222730A1 · Gower · 2015 [cited by examiner]
US 20150248462A1 · Theeten · 2015 [cited by examiner]
US 20160110421A1 · Gallé · 2016 [cited by examiner]
US 20180096144A1 · Pan · 2018 [cited by applicant]
US 20190188212A1 · Miller · 2019 [cited by applicant]
US 20210365456A1 · Kondiles · 2021 [cited by examiner]
US 20220198294A1 · Schneuwly et al. · 2022 [cited by applicant]
Kandanaarachchi et al., “On normalization and algorithm selection for unsupervised outlier detection”, Data Mining and Knowledge Discovery, vol. 34, Mar. 2020, pp. 309-354. [cited by applicant]
Apel et al., “Learning SQL for Database Intrusion Detection using Context-sensitive Modelling”, dated 2009, 33 pages. [cited by applicant]
Baldassarre et al., “Explainability Techniques for Graph Convolutional Networks”, dated May 2019, 21 pages. [cited by applicant]
Bockermann et al., “Learning SQL for Database Intrusion Detection Using Context-Sensitive Modelling”, DIMVA dated 2009, LNCS 5587, 10 pages. [cited by applicant]
Cai et al., “An Abstract Syntax Tree Encoding Method for Cross-Project Defect Prediction”, IEEE, dated Nov. 15, 2019, 10 pages. [cited by applicant]
Chomsky, “Three Models for the Description of Language,” in IRE Transactions on Information Theory, vol. 2, No. 3, Sep. 1956, 12 pages. [cited by applicant]
Follenfant et al., “SQL Abstract Syntax Trees Vocabulary”, available: https://ns.inria.fr/ast/sql/index.html, Jan. 2014, 26 pages. [cited by applicant]
Alon et al., “A General Path-Based Representation for Predicting Program Properties” dated Apr. 22, 2018, 16 pages. [cited by applicant]
Jimenez et al., “On the Impact of Tokenizer and Parameters on N-Gram Based Code Analysis”, dated 2018, 12 pages. [cited by applicant]
Zhang et al., “A Novel Neural Source Code Representation based on Abstract Syntax Tree”, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), 2019, 12 pages. [cited by applicant]
Linardatos et al., “Explainable AI: A Review of Machine Learning Interpretability Methods”, Entropy, vol. 23, No. 1, 2021, 45 pages. [cited by applicant]
Mou et al., “Building Program Vector Representations for Deep Learning”, dated Sep. 11, 2014, 11 pages. [cited by applicant]
Ndichu et al., “A machine learning approach to detection of JavaScript-based attacks using AST features and paragraph vectors”, Applied Soft Computing 84, 2019, 11 pages. [cited by applicant]
Rabinovich et al., “Abstract Syntax Networks for Code Generation and Semantic Parsing”, dated 2017, 11 pages. [cited by applicant]
Sharma et al., “A Survey on Machine Learning Techniques for Source Code Analysis”, dated 2021, 59 pages. [cited by applicant]
Widenius et al., “MySQL Reference Manual”, Chapter 6, Jan. 2002, 98 pages. [cited by applicant]
Yamaguchi et al., “Generalized Vulnerability Extrapolation using Abstract Syntax Trees”, ACSAC '12 Dec. 3-7, 2012, Orlando, Florida USA, 10 pages. [cited by applicant]
Harshvardhan et al., “A Comprehensive Survey and Analysis of Generative Models in Machine Learning”, Computer Science Review, vol. 38, 2020, 48 pages. [cited by applicant]