IP Library › Granted Patent US 12,393,875
Granted Patent B2
US 12,393,875 · App. 18/386,015 · Granted Aug 19, 2025

Training encoder model and/or using trained encoder model to determine responsive action(s) for natural language input

Inventors: Brian Strope (Palo Alto, CA); Yun-Hsuan Sung (Mountain View, CA); Wangqing Yuan (Wilmington, MA)
Assignee: GOOGLE LLC
G06N20/00G06F16/3329G06F16/3344G06F16/3346G06F16/35G06N5/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,393,875
App. No.
18/386,015
Granted
Aug 19, 2025
Kind
B2
Abstract

Systems, methods, and computer readable media related to: training an encoder model that can be utilized to determine semantic similarity of a natural language textual string to each of one or more additional natural language textual strings (directly and/or indirectly); and/or using a trained encoder model to determine one or more responsive actions to perform in response to a natural language query. The encoder model is a machine learning model, such as a neural network model. In some implementations of training the encoder model, the encoder model is trained as part of a larger network architecture trained based on one or more tasks that are distinct from a “semantic textual similarity” task for which the encoder model can be used.

Claims (39)

1. A method implemented by one or more processors, comprising:

simultaneously training an encoder model based on a plurality of first positive training instances tailored to a first task and based on a plurality of second positive training instances tailored to a second task,

wherein the first task is distinct from the second task, wherein the plurality of first positive training instances are distinct from the plurality of second positive training instances, and wherein the first task and the second task are distinct from a semantic textual similarity task; and

after training the encoder model:

using the trained encoder model to determine semantic textual similarity of two textual segments, wherein using the trained encoder model to determine semantic textual similarity of two textual segments comprises:

receiving a query directed to an automated assistant;

generating a query encoding based on processing the query using the trained encoder model;

comparing the query encoding to a plurality of pre-determined query encodings each stored in association with one or more corresponding actions;

determining, based on the comparing, a given predetermined query encoding to which the query encoding is most similar; and

in response to the query and based on the given predetermined query encoding being most similar to the query encoding, causing the automated assistant to perform the one or more corresponding actions that are stored in association with the given predetermined query encoding, wherein the one or more corresponding actions the automated assistant is caused to perform comprise causing a graphical and/or natural language response to the query to be presented, interfacing with a third party agent, causing output to be streamed at one or more connected devices, and/or controlling the one or more connected devices.

2. The method of claim 1 , wherein the query is not explicitly mapped, by the automated assistant, to the one or more corresponding actions.

3. The method of claim 1 , wherein the query is based on user input received at a first computing device, and wherein the one or more corresponding actions comprise controlling the one or more connected devices.

4. The method of claim 1 , wherein comparing the query encoding to the plurality of pre-determined query encodings comprises:

generating a plurality of scalar values, each based on a corresponding dot product of the query encoding and a corresponding one of the predetermined query encodings; and wherein determining, based on the comparing, the predetermined query encoding to which the query encoding is most similar comprises: selecting the predetermined query encoding based on the scalar value, that is based on the dot product of the query encoding and the predetermined query encoding, being the minimal of the generated plurality of scalar values.

5. The method of claim 1 , wherein using the trained encoder model to determine semantic textual similarity of two textual segments comprises: determining that a distance in embedding space, between encodings corresponding with the two textual segments, satisfies a closeness threshold.

6. The method of claim 1 , wherein the encoder model comprises one or more weights, and wherein simultaneously training the encoder model comprises updating the one or more weights based on a first subset of the plurality of first positive training instances, then updating the one or more weights based on a second subset of the plurality of second positive training instances, then updating the one or more weights based on a third subset of the plurality of first positive training instances.

7. The method of claim 1 , wherein simultaneously training the encoder model comprises utilizing multiple independent workers in the simultaneously training, wherein the independent workers include a first independent worker that trains the encoder model utilizing only the plurality of first positive training instances and a second independent worker that trains the encoder model utilizing only the plurality of second positive training instances.

8. The method of claim 1 , wherein the first task is a true response task.

9. The method of claim 1 , wherein the first task is a natural language inference task.

10. A system comprising:

one or more computers comprising one or more processors, and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

simultaneously training an encoder model based on a plurality of first positive training instances tailored to a first task and based on a plurality of second positive training instances tailored to a second task,

wherein the first task is distinct from the second task, wherein the plurality of first positive training instances are distinct from the plurality of second positive training instances, and wherein the first task and the second task are distinct from a semantic textual similarity task; and

after training the encoder model:

using the trained encoder model to determine semantic textual similarity of two textual segments, wherein using the trained encoder model to determine semantic textual similarity of two textual segments comprises:

receiving a query directed to an automated assistant;

generating a query encoding based on processing the query using the trained encoder model;

comparing the query encoding to a plurality of pre-determined query encodings each stored in association with one or more corresponding actions;

determining, based on the comparing, a given predetermined query encoding to which the query encoding is most similar; and

in response to the query and based on the given predetermined query encoding being most similar to the query encoding, causing the automated assistant to perform the one or more corresponding actions that are stored in association with the given predetermined query encoding, wherein the one or more corresponding actions the automated assistant is caused to perform comprise causing a graphical and/or natural language response to the query to be presented, interfacing with a third party agent, causing output to be streamed at one or more connected devices, and/or controlling the one or more connected devices.

11. The system of claim 10 , wherein the query is not explicitly mapped, by the automated assistant, to the one or more corresponding actions.

12. The system of claim 10 , wherein the query is based on user input received at a first computing device, and wherein the one or more corresponding actions comprise controlling the one or more connected devices.

13. The system of claim 10 , wherein comparing the query encoding to the plurality of pre-determined query encodings comprises:

generating a plurality of scalar values, each based on a corresponding dot product of the query encoding and a corresponding one of the predetermined query encodings; and wherein determining, based on the comparing, the predetermined query encoding to which the query encoding is most similar comprises: selecting the predetermined query encoding based on the scalar value, that is based on the dot product of the query encoding and the predetermined query encoding, being the minimal of the generated plurality of scalar values.

14. The system of claim 10 , wherein using the trained encoder model to determine semantic textual similarity of two textual segments comprises: determining that a distance in embedding space, between encodings corresponding with the two textual segments, satisfies a closeness threshold.

15. The system of claim 10 , wherein the encoder model comprises one or more weights, and wherein simultaneously training the encoder model comprises updating the one or more weights based on a first subset of the plurality of first positive training instances, then updating the one or more weights based on a second subset of the plurality of second positive training instances, then updating the one or more weights based on a third subset of the plurality of first positive training instances.

16. The system of claim 10 , wherein simultaneously training the encoder model comprises utilizing multiple independent workers in the simultaneously training, wherein the independent workers include a first independent worker that trains the encoder model utilizing only the plurality of first positive training instances and a second independent worker that trains the encoder model utilizing only the plurality of second positive training instances.

17. The system of claim 10 , wherein the first task is a true response task.

18. The system of claim 10 , wherein the first task is a natural language inference task.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: STROPE, BRIAN; SUNG, YUN-HSUAN; YUAN, WANGQING
To: GOOGLE LLC
Reel/Frame 065670/0056 →
Continuity (4)
Continuation 16995149 · Aug 17, 2020
Continuation 16611725
Provisional Application 62599550 · Dec 15, 2017
Related Publication 20240062111A1 · Feb 22, 2024
References Cited (66)
US 10170107B1 · Dreyer · 2019 [cited by examiner]
US 10388274B1 · Hoffmeister · 2019 [cited by examiner]
US 10783456B2 · Strope et al. · 2020 [cited by applicant]
US 20020129015A1 · Caudill · 2002 [cited by examiner]
US 20040002860A1 · Deisher · 2004 [cited by applicant]
US 20100235341A1 · Bennett · 2010 [cited by examiner]
US 20150074027A1 · Huang et al. · 2015 [cited by applicant]
US 20150127583A1 · Allgaier · 2015 [cited by examiner]
US 20150293976A1 · Guo et al. · 2015 [cited by applicant]
US 20160350655A1 · Weiss · 2016 [cited by applicant]
US 20170323636A1 · Xiao et al. · 2017 [cited by applicant]
US 20170345420A1 · Barnett, Jr. · 2017 [cited by examiner]
US 20190005021A1 · Miller · 2019 [cited by examiner]
US 20190108282A1 · Zeng · 2019 [cited by examiner]
US 20190130904A1 · Homma · 2019 [cited by examiner]
US 20200380418A1 · Strope et al. · 2020 [cited by applicant]
CN 106653030 · 2017 [cited by applicant]
CN 107293296 · 2017 [cited by applicant]
EP 3128439 · 2017 [cited by applicant]
WO 2018226960 · 2018 [cited by applicant]
Ye et al., Learning Question Similarity with Recurrent Neural Networks, 2017 IEEE International Conference on Big Knowledge. (Previously supplied). (Year: 2017). [cited by examiner]
Chen, “Case-Based Reasoning System and Artificial Neural Networks: A Review”, Neural Comput & Applic. (2001)10:264-276. (Previously supplied). (Year: 2001). [cited by examiner]
Ahmed, “A Case-Based Reasoning System for Knowledge and Experience Reuse”, in the proceedings of the 24th annual workshop of the Swedish Artificial Intelligence Society, p. 70-80, Borås, Sweden, Editor(s): Löfström et a… [cited by examiner]
Zhang, “Case-Based Reasoning Adaptation for High Dimensional Solution Space”, ICCBR 2007, LNAI 4626, pp. 149-163, 2007. (Previously supplied). (Year: 2007). [cited by examiner]
European Patent Office, Intention to Grant issue in Application No. 20169141.7; 48 pages; dated Jan. 18, 2024. [cited by applicant]
European Patent Office, Communication issued in Application No. 24173537.2; 8 pages; dated Jun. 17, 2024. [cited by applicant]
Ahmad, W.U.; Multi-Task Learning for Document Ranking and Query Suggestion; ICLR Conference; 14 pages; dated May 3, 2018. [cited by applicant]
Liu, X et al.; Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval; 10 pages; dated May 1, 2015. [cited by applicant]
China National Intellectual Property Administration; Notification of Second Office Action issued in Application No. 201880073730; 11 pages; dated Sep. 23, 2023. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action issued in Application No. 201880073730; 17 pages; dated Feb. 27, 2023. [cited by applicant]
European Patent Office; Communication pursuant to Article 94(3) issued in Application No. 20169141.7, 5 pages, dated Mar. 24, 2022. [cited by applicant]
European Patent Office; Communication issue in Application No. 20169141.7; 7 pages; dated Jun. 5, 2020. [cited by applicant]
Ye, B. et al.; “Learning Question Similarity with Recurrent Neural Networks;” 2017 IEEE International Conference on Big Knowledge; 8 pages; 2017. [cited by applicant]
European Patent Office; Intention to Grant for European Application No. 18830624.5; dated Dec. 18, 2019. [cited by applicant]
European Patent Office; Invitation to Pay Additional Fees; Application No. PCT/US2018/065727; 12 pages; dated Mar. 29, 2019. [cited by applicant]
Zhou, X. et al., “An Auto-Encoder for Learning Conversation Representation Using LSTM”; Proc. Int. Conf. Adv. Biometrics (ICB); [Lecture Notes in Computer Science]; pp. 310-317; Nov. 12, 2015. [cited by applicant]
European Patent Office; International Search Report and Written Opinion of Ser. No. PCT/US2018/065727; 15 pages; dated May 21, 2019. [cited by applicant]
Agirre, E. et al. “SemEval-2012 Task 6: A Pilot On Semantic Textual Similarity.” In Proceedings of the First Joint Conference on Lexical and Computational Semantics—vol. 1: Proceedings of the main conference and the sha… [cited by applicant]
Al-Rfou, R. et al. “Conversational Contextual Cues: The Case Of Personalization And History For Response Ranking.” Cornell University; www.arXiv.org; arXiv:1606.00372; 10 pages; Jun. 1, 2016. [cited by applicant]
Arora, S. et al. “A Simple But Tough-To-Beat Baseline For Sentence Embeddings.” In 5th International Conference on Learning Representations (ICLR); 16 pages; Nov. 4, 2016. [cited by applicant]
Bär, D. et al. “UKP: Computing Semantic Textual Similarity By Combining Multiple Content Similarity Measures.” In Proceedings of the First Joint Conference on Lexical and Computational Semantics—vol. 1: Proceedings of t… [cited by applicant]
Bowman, S. et al. “A Large Annotated Corpus For Learning Natural Language Inference.” In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Ling… [cited by applicant]
Cer, D. “SemEval-2017 Task 1: Semantic Textual Similarity-Multilingual And Cross-Lingual Focused Evaluation.” Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval 2017); https://www.aclweb.org/… [cited by applicant]
Charlet, D. et al. “SimBow at SemEval-2017 Task 3: Soft-Cosine Semantic Similarity Between Questions For Community Question Answering.” In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2… [cited by applicant]
Chen, Q. et al. “Natural Language Inference With External Knowledge.” Cornell University; www.arXiv.org; arXiv preprint arXiv:1711.04289; 11 pages; Nov. 16, 2017. [cited by applicant]
Conneau, A. “Supervised Learning Of Universal Sentence Representations From Natural Language Inference Data.” In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 11 pages, Copenhag… [cited by applicant]
Filice, S. et al. “KeLP at SemEval-2017 Task 3: Learning Pairwise Patterns In Community Question Answering.” In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), https://www.aclweb.or… [cited by applicant]
Henderson, M. et al. “Efficient Natural Language Response Suggestion For Smart Reply.” Cornell University; arXiv.org; arXiv:1705.00652; 15 pages; May 1, 2017. [cited by applicant]
Iyyer, M. et al. “Deep Unordered Composition Rivals Syntactic Methods For Text Classification.” In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint … [cited by applicant]
Jimenez, S. et al. “Soft Cardinality: A Parameterized Similarity Function For Text Comparison.” In Proceedings of the First Joint Conference on Lexical and Computational Semantics—vol. 1: Proceedings of the main confere… [cited by applicant]
Kannan, A. et al. “Smart Reply: Automated Response Suggestion For Email.” In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; pp. 955-964; Aug. 2016. [cited by applicant]
Kiros, R. et al. “Skip-Thought Vectors.” Proceedings of the 28th International Conference on Neural Information Processing Systems; 9 pages; Dec. 2015, Retrieved date of Dec. 2015 from Google Scholar. [cited by applicant]
Kruszewski, G. et al. “Jointly Optimizing Word Representations For Lexical And Sentential Tasks With The C-Phrase Model.” In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and th… [cited by applicant]
Lau, J. et al. “An Empirical Evaluation Of Doc2Vec With Practical Insights Into Document Embedding Generation.” In Processings of ACL Workshop on Representation Learning for NLP, https://www.aclweb.org/anthology/W16-160… [cited by applicant]
Nakov, P. et al. “SemEval-2017 Task 3: Community Question Answering.” In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval '17, Vancouver, Canada. Association for Computational Linguistics; … [cited by applicant]
Pagliardini, M. “Unsupervised Learning Of Sentence Embeddings Using Compositional n-Gram Features.” Cornell University, arXiv.org; arXiv preprint arXiv:1703.02507; 9 pages; Mar. 7, 2017. [cited by applicant]
Shao, Y. “HCTI at SemEval-2017 Task 1: Use Convolutional Neural Network To Evaluate Semantic Textual Similarity.” In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), https://www.aclw… [cited by applicant]
Tai, K. et al. “Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks.” Cornell University; arXiv.org; arXiv preprint arXiv:1503.00075v3; 11 pages; May 30, 2015. [cited by applicant]
Tian et al. “ECNU at SemEval-2017 Task 1: Leverage Kernel-Based Traditional NLP Features And Neural Networks To Build A Universal Model For Multilingual And Cross-Lingual Semantic Textual Similarity.” In Proceedings of … [cited by applicant]
Vaswani, A. et al. “Attention Is All You Need.” Advances in Neural Information Processing Systems 30 (NIPS 2017), 11 pages; dated 2017. [cited by applicant]
Williams, A. et al. “Learning To Parse From A Semantic Objective: It Works. Is It Syntax?” Cornell University; arXiv.org; arXiv preprint arXiv:1709.01121; 13 pages; Sep. 4, 2017. [cited by applicant]
Wu, H. “BIT at SemEval-2017 Task 1: Using Semantic Information Space To Evaluate Semantic Textual Similarity.” In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), https://www.aclweb.… [cited by applicant]
Chen; Case-Based Reasoning System and Artificial Neural Networks: A Review; Neural Comput & Applic. 10:264-276; dated 2001. [cited by applicant]
Ahmed; A Case-Based Reasoning System for Knowledge and Experience Reuse; in the proceedings of the 24th annual workshop of the Swedish Artificial Intelligence Society; pp. 70-80; Boras, Sweden; Editor(s): Lofstrom et al… [cited by applicant]
Zhang; Case-Based Reasoning Adaptation for High Dimensional Solution Space; ICCBR 2007; LNAI 4626; pp. 149-163; dated 2007. [cited by applicant]
China National Intellectual Property Administration; Grant Notice issued in Application No. 201880073730.0; 4 pages; dated Oct. 23, 2023. [cited by applicant]
Cited By (2)
US 12,694,227 US 12,737,557