IP Library › Granted Patent US 12,462,105
Granted Patent B2
US 12,462,105 · App. 17/926,999 · Granted Nov 4, 2025

Clause based semantic parsing

Inventors: Zeqi Lin (Redmond, WA); JianGuang Lou (Redmond, WA); Dongmei Zhang (Beijing, CN)
Assignee: Microsoft Technology Licensing, LLC
G06F40/30G06F40/211G06F40/289G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,462,105
App. No.
17/926,999
Granted
Nov 4, 2025
Kind
B2
Abstract

According to implementations of the subject matter described herein, a clause-based semantic parsing solution is provided. In the solution, a first clause with independent semantics is determined from a target statement. The target statement is converted to a first intermediate statement based on a first logical representation corresponding to the first clause. Subsequently, at least one logical representation corresponding to at least part of semantics of the first intermediate statement is determined. The first logical representation and the at least one logical representation may be used to determine a target logical representation corresponding to semantics of the target statement. Therefore, more accurate semantic parsing can be achieved.

Claims (82)

1 . A computer-implemented method, comprising:

determining, by a clause recognition model trained using a plurality of training statements and position information of independent clauses, a first clause with independent semantics from a target statement, wherein the position information indicates start and end positions of independent clauses in the plurality of training statements;

converting, using a logical representation parsing model jointly trained with the clause recognition model based on a same training dataset, the target statement to a first intermediate statement based on a first logical representation corresponding to the first clause;

determining at least one logical representation corresponding to at least part of semantics of the first intermediate statement by;

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of a second clause when the first intermediate statement comprises the second clause with independent semantics, and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement;

determining a target logical representation corresponding to semantics of the target statement by combining the first logical representation and the at least one logical representation; and

outputting the target logical representation.

2 . The method of claim 1 , wherein determining the at least one logical representation comprises:

if it is determined that the first intermediate statement comprises a second clause with independent semantics,

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of the second clause; and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement; and

determining the at least one logical representation based on at least the second logical representation and the third logical representation.

3 . The method of claim 1 , wherein determining the first clause comprises:

identifying a first clause from the target statement using a clause recognition model, wherein the clause recognition model is trained based on a plurality of training statements and position information of an independent clause with independent semantics in each training statement.

4 . The method of claim 3 , wherein the position information indicates a start position and an end position of the independent clause in the training statement.

5 . The method of claim 3 , further comprising:

determining a statement logical representation corresponding to the training statement;

determining a clause logical representation of a candidate clause in the training statement; and

if it is determined that the statement logical representation contains the clause logical representation, determining the candidate clause as the independent clause.

6 . The method of claim 3 , further comprising:

determining a statement logical representation corresponding to the training statement;

determining a clause logical representation of a candidate clause in the training statement; and

if it is determined that the statement logical representation contains the clause logical representation,

converting the training statement to an intermediate training statement based on the clause logical representation;

determining a first intermediate logical representation of the intermediate training statement;

converting the statement logical representation to a second intermediate logical representation based on the clause logical representation; and

determining the candidate clause as the independent clause based on a difference between the first intermediate logical representation and the second intermediate logical representation.

7 . The method of claim 3 , wherein the first logical representation is determined using the logical representation parsing model.

8 . The method of claim 1 , wherein converting the target statement to a first intermediate statement comprises:

replacing a first clause in the target statement with a simplified representation of the first logical representation to obtain the first intermediate statement.

9 . The method of claim 1 , wherein determining a target logical representation of the target statement comprises:

combining the first logical representation and the at least one logical representation to determine the target logical representation.

10 . A device, comprising:

a processing unit; and

a memory coupled to the processing unit and having instructions stored thereon which, when executed by the processing unit, cause the device to perform acts comprising:

determining, by a clause recognition model trained using a plurality of training statements and position information of independent clauses, a first clause with independent semantics from a target statement, wherein the position information indicates start and end positions of independent clauses in the plurality of training statements;

converting, using a logical representation parsing model jointly trained with the clause recognition model based on a same training dataset, the target statement to a first intermediate statement based on a first logical representation corresponding to the first clause;

determining at least one logical representation corresponding to at least part of semantics of the first intermediate statement by:

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of a second clause when the first intermediate statement comprises the second clause with independent semantics, and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement;

determining a target logical representation corresponding to semantics of the target statement by combining the first logical representation and the at least one logical representation; and

outputting the target logical representation.

11 . The device of claim 10 , wherein determining the at least one logical representation comprises:

if it is determined that the first intermediate statement comprises a second clause with independent semantics,

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of the second clause; and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement; and

determining the at least one logical representation based on at least the second logical representation and the third logical representation.

12 . The device of claim 10 , wherein determining the first clause comprises:

identifying a first clause from the target statement using a clause recognition model, wherein the clause recognition model is trained based on a plurality of training statements and position information of an independent clause with independent semantics in each training statement.

13 . The device of claim 12 , wherein the position information indicates a start position and an end position of the independent clause in the training statement.

14 . The device of claim 12 , the acts further comprising:

determining a statement logical representation corresponding to the training statement;

determining a clause logical representation of a candidate clause in the training statement; and

if it is determined that the statement logical representation contains the clause logical representation, determining the candidate clause as the independent clause.

15 . At least one non-transitory computer storage medium comprising machine-executable instructions which, when executed by a device, cause the device to perform acts comprising:

determining, by a clause recognition model trained using a plurality of training statements and position information of independent clauses, a first clause with independent semantics from a target statement, wherein the position information indicates start and end positions of independent clauses in the plurality of training statements;

converting, using a logical representation parsing model jointly trained with the clause recognition model based on a same training dataset, the target statement to a first intermediate statement based on a first logical representation corresponding to the first clause;

determining at least one logical representation corresponding to at least part of semantics of the first intermediate statement by:

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of a second clause when the first intermediate statement comprises the second clause with independent semantics, and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement;

determining a target logical representation corresponding to semantics of the target statement by combining the first logical representation and the at least one logical representation; and

outputting the target logical representation.

16 . The at least one non-transitory computer storage medium of claim 15 , wherein determining the at least one logical representation comprises:

if it is determined that the first intermediate statement comprises a second clause with independent semantics,

converting the first intermediate statement to a second intermediate statement based on a second logical representation corresponding to semantics of the second clause; and

determining a third logical representation corresponding to at least part of semantics of the second intermediate statement; and

determining the at least one logical representation based on at least the second logical representation and the third logical representation.

17 . The at least one non-transitory computer storage medium of claim 15 , wherein determining the first clause comprises:

identifying a first clause from the target statement using a clause recognition model, wherein the clause recognition model is trained based on a plurality of training statements and position information of an independent clause with independent semantics in each training statement.

18 . The at least one non-transitory computer storage medium of claim 17 , wherein the position information indicates a start position and an end position of the independent clause in the training statement.

19 . The at least one non-transitory computer storage medium of claim 17 , the acts further comprising:

determining a statement logical representation corresponding to the training statement;

determining a clause logical representation of a candidate clause in the training statement; and

if it is determined that the statement logical representation contains the clause logical representation, determining the candidate clause as the independent clause.

20 . The at least one non-transitory computer storage medium of claim 17 , further comprising:

determining a statement logical representation corresponding to the training statement;

determining a clause logical representation of a candidate clause in the training statement; and

if it is determined that the statement logical representation contains the clause logical representation, converting the training statement to an intermediate training statement based on the clause logical representation;

determining a first intermediate logical representation of the intermediate training statement;

converting the statement logical representation to a second intermediate logical representation based on the clause logical representation; and

determining the candidate clause as the independent clause based on a difference between the first intermediate logical representation and the second intermediate logical representation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 2, 2022
From: LIN, ZEQI; LOU, JIANGUANG; ZHANG, DONGMEI
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 061961/0803 →
Priority Claims (1)
CN 202010609575.9 · Jun 29, 2020 · national
Continuity (1)
Related Publication 20230214597A1 · Jul 6, 2023
References Cited (66)
US 5093788A · Shiotani et al. · 1992 [cited by applicant]
US 8301436B2 · Wang · 2012 [cited by applicant]
US 8731905B1 · Tsang et al. · 2014 [cited by applicant]
US 9830315B1 · Xiao · 2017 [cited by examiner]
US 20080320031A1 · Denoual · 2008 [cited by examiner]
US 20150227528A1 · Kang · 2015 [cited by applicant]
US 20150261744A1 · Suenbuel · 2015 [cited by examiner]
US 20150331850A1 · Ramish · 2015 [cited by examiner]
US 20170069315A1 · Chung · 2017 [cited by applicant]
US 20180189269A1 · Quirk · 2018 [cited by examiner]
US 20180322188A1 · Zhou · 2018 [cited by examiner]
US 20190294676A1 · Sapugay et al. · 2019 [cited by applicant]
US 20200073983A1 · Sen · 2020 [cited by applicant]
CN 103250149A · 2013 [cited by applicant]
CN 109670024A · 2019 [cited by applicant]
CN 109753658A · 2019 [cited by applicant]
CN 109933772A · 2019 [cited by applicant]
CN 110427629A · 2019 [cited by applicant]
CN 110705316A · 2020 [cited by applicant]
CN 110727839A · 2020 [cited by applicant]
EP 0895396A2 · 1999 [cited by applicant]
WO 2020005601A1 · 2020 [cited by applicant]
Office Action Received for Chinese Application No. 202010609575.9, mailed on May 24, 2024, 18 pages (English Translation Provided). [cited by applicant]
Second Office Action Received for Chinese Application No. 202010609575.9, mailed on Oct. 31, 2024, 14 pages (English Translation Provided). [cited by applicant]
Third Office Action Received for Chinese Application No. 202010609575.9, mailed on Mar. 19, 2025, 15 pages (English Translation Provided). [cited by applicant]
Aho, et al., “Syntax directed translations and the pushdown assembler”, In Journal of Computer and System Sciences, vol. 3, Issue 1, Feb. 1, 1969, pp. 37-56. [cited by applicant]
Berant, et al., “Imitation learning of agenda-based semantic parsers”, In Transactions of the Association for Computational Linguistics, vol. 3, Oct. 2015, pp. 545-558. [cited by applicant]
Berant, et al., “Semantic parsing on Freebase from question-answer pairs”, In Proceedings of the conference on empirical methods in natural language processing, Oct. 18, 2013, pp. 1533-1544. [cited by applicant]
Bojanowski, et al., “Enriching word vectors with subword information”, In Journal of Computing Research Repository, Jul. 2016, pp. 135-146. [cited by applicant]
Bollacker, et al., “Freebase: a collaboratively created graph database for structuring human knowledge”, In Proceedings of the ACM SIGMOD international conference on Management of data, Jun. 9, 2008, pp. 1247-1250. [cited by applicant]
Cheng, et al., “Learning Structured Natural Language Representations for Semantic Parsing”, In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Jul. 30, 2017, pp. 44-55. [cited by applicant]
Cho, et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation”, In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Jun. 2014, pp. … [cited by applicant]
Devlin, et al., “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human … [cited by applicant]
Dong, et al., “Coarse-to-Fine Decoding for Neural Semantic Parsing”, In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), Jul. 15, 2018, pp. 731-742. [cited by applicant]
Dong, et al., “Language to Logical Form with Neural Attention”, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Aug. 7, 2016, pp. 33-43. [cited by applicant]
Furuse, et al., “Splitting Long or Ill-formed Input for Robust Spoken-language Translation”, In Proceedings of 17th International Conference on Computational Linguistics, vol. 1, Aug. 10, 1998, pp. 421-427. [cited by applicant]
Gu, et al., “Incorporating Copying Mechanism in Sequence-to-Sequence Learning”, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, Aug. 7, 2016, pp. 1631-1640. [cited by applicant]
Guo, et al., “Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation”, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 28, 2019, pp. 4524-4… [cited by applicant]
Jia, et al., “Data recombination for neural semantic parsing”, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Aug. 7, 2016, pp. 12-22. [cited by applicant]
Kalyanpur, et al., “Fact-based question decomposition in DeepQA”, In IBM Journal of Research and Development, vol. 56, Issue 3, May 1, 2012, 11 Pages. [cited by applicant]
Kate, “Learning to Transform Natural to Formal Languages”, In Proceedings of the Twentieth National Conference on Artificial Intelligence, Jul. 2005, pp. 1062-1068. [cited by applicant]
Krishnamurthy, et al., “Neural Semantic Parsing with Type Constraints for Semi-Structured Tables”, In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Sep. 7, 2017, pp. 1516-1526. [cited by applicant]
Liang, et al., “Learning Dependency-Based Compositional Semantics”, In Journal of Computational Linguistics, vol. 39, Issue 2, May 2, 2013, pp. 389-446. [cited by applicant]
Ling, et al., “Latent predictor networks for code generation”, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Aug. 7, 2016, pp. 599-609. [cited by applicant]
Luong, et al., “Effective Approaches to Attention-Based Neural Machine Translation”, In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Sep. 17, 2015, pp. 1412-1421. [cited by applicant]
Manning, et al., “The Stanford CoreNLP Natural Language Processing Toolkit”, In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Jun. 22, 2014, pp. 55-60. [cited by applicant]
Min, et al., “Multi-hop Reading Comprehension through Question Decomposition and Rescoring”, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 28, 2019, pp. 6097-6109. [cited by applicant]
Pasupat, et al., “Compositional Semantic Parsing on Semi-Structured Tables”, In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natur… [cited by applicant]
“International Search Report and Written Opinion Issued in PCT Application No. PCT/US21/034655”, Mailed Date: Sep. 24, 2021, 9 Pages. [cited by applicant]
Pelletier, Francisj, “The Principle of Semantic Compositionality”, In Journal of Topoi, vol. 13, Issue 1, Mar. 1994, 48 Pages. [cited by applicant]
Qi, et al., “Answering Complex Open-domain Questions Through Iterative Query Generation”, In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Nov. 3, 2019, pp. 2590-2602. [cited by applicant]
Rabinovich, et al., “Abstract Syntax Networks for Code Generation and Semantic Parsing”, In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Jul. 30, 2017, pp. 1139-1149. [cited by applicant]
See, et al., “Get to the point: Summarization with pointer-generator networks”, In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), Jul. 30, 2017, pp. 1073-1… [cited by applicant]
Shaw, et al., “Generating Logical Forms from Graph Representations of Text and Entities”, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 28, 2019, pp. 95-106. [cited by applicant]
Talmor, et al., “The Web as a Knowledge-Base for Answering Complex Questions”, In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol… [cited by applicant]
Treviso, et al., “Sentence Segmentation in Narrative Transcripts from Neuropsychological Tests using Recurrent Convolutional Neural Networks”, In Proceedings of the 15th Conference of the European Chapter of the Associa… [cited by applicant]
Vaswani, et al., “Attention Is All You Need”, In Proceedings of 31st Conference on Neural Information Processing Systems, Dec. 4, 2017, 11 Pages. [cited by applicant]
Xiao, et al., “Sequence-based Structured Prediction for Semantic Parsing”, In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, vol. 1, Aug. 2016, pp. 1341-1350. [cited by applicant]
Yin, et al., “A Syntactic Neural Model for General-Purpose Code Generation”, In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), Jul. 30, 2017, pp. 440-450. [cited by applicant]
Zelle, et al., “Learning to parse database queries using inductive logic programming”, In Proceedings of the thirteenth national conference on Artificial intelligence, vol. 2, Aug. 4, 1996, pp. 1050-1055. [cited by applicant]
Zettlemoyer, et al., “Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars”, In Proceedings of Twenty-First Conference on Uncertainty in Artificial Intelligence, Ju… [cited by applicant]
Zettlemoyer, et al., “Online Learning of Relaxed CCG Grammars for Parsing to Logical Form”, In Proceedings of 13th Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language … [cited by applicant]
Zhang, et al., “Complex Question Decomposition for Semantic Parsing”, In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 2019, pp. 4477-4486. [cited by applicant]
Zhao, et al., “Type-Driven Incremental Semantic Parsing with Polymorphism”, In the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, May 31, 2015, pp… [cited by applicant]
Kate, et al., “Using String-Kernels for Learning Semantic Parsers”, Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics,… [cited by applicant]
Notice of Allowance Received for Chinese Application No. 202010609575.9, mailed on Jun. 13, 2025, 15 pages (English Translation Provided). [cited by applicant]