IP Library › Granted Patent US 12,450,464
Granted Patent B2
US 12,450,464 · App. 18/471,257 · Granted Oct 21, 2025

Guided dialogue using language generation neural networks and search

Inventors: Geoffrey Irving (London, GB); Amelia Marita Claudia Glaese (London, GB); Nathaniel John McAleese-Park (London, GB); Lisa Anne Marie Hendricks (London, GB)
Assignee: GDM Holding LLC
G06N3/006G06F40/284G06F40/35G06N3/0455G06N3/092
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,450,464
App. No.
18/471,257
Granted
Oct 21, 2025
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enabling a user to conduct a dialogue. Implementations of the system learn when to rely on supporting evidence, obtained from an external search system via a search system interface, and are also able to generate replies for the user that align with the preferences of a previously trained response selection neural network. Implementations of the system can also use a previously trained rule violation detection neural network to generate replies that take account of previously learnt rules.

Claims (142)

1. A method, implemented by one or more computers, the method comprising:

initializing a context input that represents a context for a dialogue between a user and the one or more computers, and,

at one or more of a plurality of dialogue update iterations:

receiving a natural language request from the user;

updating the context input to include the natural language request;

processing the context input using a first trained language generation neural network to generate a set of one or more first natural language responses without using a search system interface, wherein the first trained language generation neural network is a first auto-regressive neural network that includes a first succession of first self-attention neural network layers, and wherein each first self-attention neural network layer applies an attention mechanism over a first attention layer input for the first self-attention neural network layer to generate an attention layer output for each element of the first attention layer input;

generating a set of one or more search queries from the natural language request;

providing the one or more search queries to a search system using the search system interface, wherein the search system is external to the first trained language generation neural network;

receiving, for each search query, a set of one or more search results from the search system via the search system interface;

determining, from the context input and for each of the search results, a supported context input that includes content from one of the search results;

for each supported context input that includes content from one of the search results, processing the supported context input using the first trained language generation neural network to generate a set of one or more second natural language responses, wherein each second natural language response includes a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface;

processing (i) at least a portion of the context input that includes the natural language request received by the user, (ii) the one or more first natural language responses generated without using the search system interface by the first trained language generation neural network in response to the context input, and (iii) the one or more second natural language responses generated by the first trained language generation neural network in response that each include a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface in the content from the one using a trained response selection neural network to generate an output that defines a selection between the one or more first natural language responses and the one or more second natural language responses, wherein the trained response selection neural network is a second auto-regressive neural network that includes a second succession of second self-attention neural network layers, and wherein each second self-attention neural network layer applies an attention mechanism over a second attention layer input for the second self-attention neural network layer to generate an attention layer output for each element of the second attention layer input;

selecting, based on the output of the trained response selection neural network and as a natural language reply, one of the one or more first natural language responses generated by the first trained language generation neural network in response to the context input and the one or more second natural language responses generated by the first trained language generation neural network in response to each supported context input;

providing the natural language reply to the user as a response to the natural language user request; and

updating the context input to include a representation of the natural language reply for the next dialogue update iteration.

2. The method of claim 1 , wherein the set of one or more first natural language responses includes a plurality of first natural language responses; the set of one or more search queries includes a plurality of search queries; the set of one or more search results for each search query includes a plurality of search results; and the set of one or more second natural language responses comprises a plurality of second natural language responses.

3. The method of claim 1 , further comprising, at the next dialogue update iteration:

receiving a follow-up natural language request from the user;

updating the context input to include the follow-up natural language request; then

processing the context input using the first trained language generation neural network to generate one or more third natural language responses;

generating one or more follow-up search queries from the follow-up natural language request;

providing the one or more follow-up search queries to a search system interface;

receiving, for each follow-up search query, one or more follow-up search results from the search system interface,

determining, from the context input and for each of the follow-up search results, a follow-up supported context input that includes content from one of the follow-up search results;

processing each follow-up supported context input using the first trained language generation neural network to generate one or more respective fourth natural language responses;

processing the one or more third natural language responses and the one or more fourth natural language responses using the trained response selection neural network to select a follow-up natural language reply from the one or more third natural language responses and the one or more fourth natural language responses; and

providing the follow-up natural language reply to the user as a response to the follow-up natural language request; and

updating the context input to include a representation of the follow-up natural language reply.

4. The method of claim 1 wherein processing (i), (ii), and (iii) using the trained response selection neural network comprises:

for each first natural language response and each second natural language response:

processing the at least a portion of the context input and the natural language response using the trained response selection neural network to generate a preference score for the natural language response; and

selecting one of the first and second natural language responses, based on the preference scores for the first and second natural language responses, to select the natural language reply.

5. The method of claim 4 , further comprising, for each first and second natural language response:

processing at least a portion of the context input and the natural language response using a trained rule violation detection neural network to determine, for each of a plurality of rules, a rule violation score that estimates a probability that the rule is violated; and wherein

selecting one of the first and second natural language responses to select the natural language reply is further based on the rule violation score for each of the rules for each of the first and second natural language responses.

6. The method of claim 5 , wherein selecting one of the first and second natural language responses to select the natural language reply comprises:

for each first and second natural language response:

determining a combined rule violation score for the natural language response by combining the rule violation score for each rule; and

combining the preference score and combined rule violation score for the natural language response to determine a re-ranking score; and

selecting one of the first and second natural language responses, based on the re-ranking scores for the first and second natural language responses, to select the natural language reply.

7. The method of claim 5 , wherein determining the rule violation score that estimates a probability that the rule is violated comprises processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and a natural language representation of the rule, using the trained rule violation detection neural network, to determine the rule violation score that estimates the probability that the rule is violated.

8. The method of claim 7 , wherein processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule comprises processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule, using the trained rule violation detection neural network, to generate one or more natural language output tokens representing a determination of whether or not the rule was violated; and

determining the rule violation score from one or more output layer values corresponding to the one or more natural language output tokens.

9. The method of claim 7 , wherein processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule comprises:

processing at least the portion of the context input and the natural language response to determine a shared intermediate state of the trained rule violation detection neural network; and

processing, for each of the plurality of rules, the natural language representation of the rule starting from the shared intermediate state to generate the rule violation score.

10. The method of claim 8 , wherein the first trained language generation neural network, the trained response selection neural network, and the trained rule violation detection neural network, each comprise a respective sequence-to-sequence neural network configured to receive an input sequence of tokens and to process the input sequence of natural language tokens in accordance with a respective set of neural network parameters, to generate an output sequence of natural language tokens, and wherein the first trained language generation neural network, the trained response selection neural network, and the trained rule violation detection neural network comprise a set of shared input layers.

11. The method of claim 1 , wherein the first trained language generation neural network and the trained response selection neural network are stored on a user computing device; wherein the search system is remote from the user computing device; and

wherein the one or more search results are received over a wired or wireless communications link between the user computing device and the search system.

12. A system comprising:

one or more computers; and

one or more non-transitory storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

initializing a context input and,

at one or more of a plurality of dialogue update iterations:

receiving a natural language request from the user;

updating the context input to include the natural language request;

processing the context input using a first trained language generation neural network to generate a set of one or more first natural language responses without using a search system interface, wherein the first trained language generation neural network is a first auto-regressive neural network that includes a first succession of first self-attention neural network layers, and wherein each first self-attention neural network layer applies an attention mechanism over a first attention layer input for the first self-attention neural network layer to generate an attention layer output for each element of the first attention layer input;

generating a set of one or more search queries from the natural language request;

providing the one or more search queries to a search system using the search system interface, wherein the search system is external to the first trained language generation neural network;

receiving, for each search query, a set of one or more search results from the search system via the search system interface;

determining, from the context input and for each of the search results, a supported context input that includes content from one of the search results;

for each supported context input that includes content from one of the search results, processing the supported context input using the first trained language generation neural network to generate a set of one or more second natural language responses, wherein each second natural language response includes a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface;

processing (i) at least a portion of the context input that includes the natural language request received by the user, (ii) the one or more first natural language responses generated without using the search system interface by the first trained language generation neural network in response to the context input, and (iii) the one or more second natural language responses generated by the first trained language generation neural network in response that each include a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface using a trained response selection neural network to generate an output that defines a selection between the one or more first natural language responses and the one or more second natural language responses, wherein the trained response selection neural network is a second auto-regressive neural network that includes a second succession of second self-attention neural network layers, and wherein each second self-attention neural network layer applies an attention mechanism over a second attention layer input for the second self-attention neural network layer to generate an attention layer output for each element of the second attention layer input;

selecting, based on the output of the trained response selection neural network and as a natural language reply, one of the one or more first natural language responses generated by the first trained language generation neural network in response to the context input and the one or more second natural language responses generated by the first trained language generation neural network in response to each supported context input;

providing the natural language reply to the user as a response to the natural language user request; and

updating the context input to include a representation of the natural language reply for the next dialogue update iteration.

13. The method of claim 1 , wherein the first succession of first self-attention layers are different instances of self-attention layers with the same architecture as the self-attention layers in the second succession of second self-attention layers.

14. The system of claim 12 , wherein the first succession of first self-attention layers are different instances of self-attention layers with the same architecture as the self-attention layers in the second succession of second self-attention layers.

15. The system of claim 12 , wherein the set of one or more first natural language responses includes a plurality of first natural language responses; the set of one or more search queries includes a plurality of search queries; the set of one or more search results for each search query includes a plurality of search results; and the set of one or more second natural language responses comprises a plurality of second natural language responses.

16. The system of claim 12 , the operations further comprising, at the next dialogue update iteration:

receiving a follow-up natural language request from the user;

updating the context input to include the follow-up natural language request; then

processing the context input using the first trained language generation neural network to generate one or more third natural language responses;

generating one or more follow-up search queries from the follow-up natural language request;

providing the one or more follow-up search queries to a search system interface;

receiving, for each follow-up search query, one or more follow-up search results from the search system interface,

determining, from the context input and for each of the follow-up search results, a follow-up supported context input that includes content from one of the follow-up search results;

processing each follow-up supported context input using the first trained language generation neural network to generate one or more respective fourth natural language responses;

processing the one or more third natural language responses and the one or more fourth natural language responses using the trained response selection neural network to select a follow-up natural language reply from the one or more third natural language responses and the one or more fourth natural language responses; and

providing the follow-up natural language reply to the user as a response to the follow-up natural language request; and

updating the context input to include a representation of the follow-up natural language reply.

17. The system of claim 12 , wherein processing (i), (ii), and (iii) using the trained response selection neural network comprises:

for each first natural language response and each second natural language response:

processing the at least a portion of the context input and the natural language response using the trained response selection neural network to generate a preference score for the natural language response; and

selecting one of the first and second natural language responses, based on the preference scores for the first and second natural language responses, to select the natural language reply.

18. The system of claim 17 , the operations further comprising, for each first and second natural language response:

processing at least a portion of the context input and the natural language response using a trained rule violation detection neural network to determine, for each of a plurality of rules, a rule violation score that estimates a probability that the rule is violated; and wherein

selecting one of the first and second natural language responses to select the natural language reply is further based on the rule violation score for each of the rules for each of the first and second natural language responses.

19. The system of claim 18 , wherein selecting one of the first and second natural language responses to select the natural language reply comprises:

for each first and second natural language response:

determining a combined rule violation score for the natural language response by combining the rule violation score for each rule; and

combining the preference score and combined rule violation score for the natural language response to determine a re-ranking score; and

selecting one of the first and second natural language responses, based on the re-ranking scores for the first and second natural language responses, to select the natural language reply.

20. The system of claim 18 , wherein determining the rule violation score that estimates a probability that the rule is violated comprises processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and a natural language representation of the rule, using the trained rule violation detection neural network, to determine the rule violation score that estimates the probability that the rule is violated.

21. The system of claim 20 , wherein processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule comprises processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule, using the trained rule violation detection neural network, to generate one or more natural language output tokens representing a determination of whether or not the rule was violated; and

determining the rule violation score from one or more output layer values corresponding to the one or more natural language output tokens.

22. The system of claim 20 , wherein processing, for each of the plurality of rules, at least the portion of the context input, the natural language response, and the natural language representation of the rule comprises:

processing at least the portion of the context input and the natural language response to determine a shared intermediate state of the trained rule violation detection neural network; and

processing, for each of the plurality of rules, the natural language representation of the rule starting from the shared intermediate state to generate the rule violation score.

23. The system of claim 21 , wherein the first trained language generation neural network, the trained response selection neural network, and the trained rule violation detection neural network, each comprise a respective sequence-to-sequence neural network configured to receive an input sequence of tokens and to process the input sequence of natural language tokens in accordance with a respective set of neural network parameters, to generate an output sequence of natural language tokens, and wherein the first trained language generation neural network, the trained response selection neural network, and the trained rule violation detection neural network comprise a set of shared input layers.

24. The system of claim 12 , wherein the first trained language generation neural network and the trained response selection neural network are stored on a user computing device; wherein the search system is remote from the user computing device;

and wherein the one or more search results are received over a wired or wireless communications link between the user computing device and the search system.

25. One or more non-transitory computer-readable storage media encoded with a computer program, the program comprising instructions that are operable, when executed by a data processing apparatus, to cause the data processing apparatus to perform operations comprising:

initializing a context input and,

at one or more of a plurality of dialogue update iterations:

receiving a natural language request from the user;

updating the context input to include the natural language request;

processing the context input using a first trained language generation neural network to generate a set of one or more first natural language responses without using a search system interface, wherein the first trained language generation neural network is a first auto-regressive neural network that includes a first succession of first self-attention neural network layers, and wherein each first self-attention neural network layer applies an attention mechanism over a first attention layer input for the first self-attention neural network layer to generate an attention layer output for each element of the first attention layer input;

generating a set of one or more search queries from the natural language request;

providing the one or more search queries to a search system using the search system interface, wherein the search system is external to the first trained language generation neural network;

receiving, for each search query, a set of one or more search results from the search system via the search system interface;

determining, from the context input and for each of the search results, a supported context input that includes content from one of the search results;

for each supported context input that includes content from one of the search results, processing the supported context input using the first trained language generation neural network to generate a set of one or more second natural language responses, wherein each second natural language response includes a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface;

processing (i) at least a portion of the context input that includes the natural language request received by the user, (ii) the one or more first natural language responses generated without using the search system interface by the first trained language generation neural network in response to the context input, and (iii) the one or more second natural language responses generated by the first trained language generation neural network in response that each include a response to the natural language request including supporting evidence that comprises content from the one of the search results obtained by providing the one or more search queries to the search system using the search system interface using a trained response selection neural network to generate an output that defines a selection between the one or more first natural language responses and the one or more second natural language responses, wherein the trained response selection neural network is a second auto-regressive neural network that includes a second succession of second self-attention neural network layers, and wherein each second self-attention neural network layer applies an attention mechanism over a second attention layer input for the second self-attention neural network layer to generate an attention layer output for each element of the second attention layer input;

selecting, based on the output of the trained response selection neural network and as a natural language reply, one of the one or more first natural language responses generated by the first trained language generation neural network in response to the context input and the one or more second natural language responses generated by the first trained language generation neural network in response to each supported context input;

providing the natural language reply to the user as a response to the natural language user request; and

updating the context input to include a representation of the natural language reply for the next dialogue update iteration.

26. The one or more non-transitory computer-readable storage media of claim 25 , wherein the set of one or more first natural language responses includes a plurality of first natural language responses; the set of one or more search queries includes a plurality of search queries; the set of one or more search results for each search query includes a plurality of search results; and the set of one or more second natural language responses comprises a plurality of second natural language responses.

27. The one or more non-transitory computer-readable storage media of claim 25 , the operations further comprising, at the next dialogue iteration:

receiving a follow-up natural language request from the user;

updating the context input to include the follow-up natural language request; then

processing the context input using the first trained language generation neural network to generate one or more third natural language responses;

generating one or more follow-up search queries from the follow-up natural language request;

providing the one or more follow-up search queries to a search system interface;

receiving, for each follow-up search query, one or more follow-up search results from the search system interface,

determining, from the context input and for each of the follow-up search results, a follow-up supported context input that includes content from one of the follow-up search results;

processing each follow-up supported context input using the first trained language generation neural network to generate one or more respective fourth natural language responses;

processing the one or more third natural language responses and the one or more fourth natural language responses using the trained response selection neural network to select a follow-up natural language reply from the one or more third natural language responses and the one or more fourth natural language responses; and

providing the follow-up natural language reply to the user as a response to the follow-up natural language request; and

updating the context input to include a representation of the follow-up natural language reply.

28. The one or more non-transitory computer-readable storage media of claim 25 , wherein processing (i), (ii), and (iii) using the trained response selection neural network comprises:

for each first natural language response and each second natural language response:

processing the at least a portion of the context input and the natural language response using the trained response selection neural network to generate a preference score for the natural language response; and

selecting one of the first and second natural language responses, based on the preference scores for the first and second natural language responses, to select the natural language reply.

29. The one or more non-transitory computer-readable storage media of claim 28 , the operations further comprising, for each first and second natural language response:

processing at least a portion of the context input and the natural language response using a trained rule violation detection neural network to determine, for each of a plurality of rules, a rule violation score that estimates a probability that the rule is violated; and wherein

selecting one of the first and second natural language responses to select the natural language reply is further based on the rule violation score for each of the rules for each of the first and second natural language responses.

30. The one or more non-transitory computer-readable storage media of claim 29 , wherein selecting one of the first and second natural language responses to select the natural language reply comprises:

for each first and second natural language response:

determining a combined rule violation score for the natural language response by combining the rule violation score for each rule; and

combining the preference score and combined rule violation score for the natural language response to determine a re-ranking score; and

selecting one of the first and second natural language responses, based on the re-ranking scores for the first and second natural language responses, to select the natural language reply.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2025
From: DEEPMIND TECHNOLOGIES LIMITED
To: GDM HOLDING LLC
Reel/Frame 071498/0210 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 4, 2024
From: IRVING, GEOFFREY; GLAESE, AMELIA MARITA CLAUDIA; MCALEESE-PARK, NATHANIEL JOHN; HENDRICKS, LISA ANNE MARIE
To: DEEPMIND TECHNOLOGIES LIMITED
Reel/Frame 068479/0091 →
Continuity (2)
Provisional Application 63408430 · Sep 20, 2022
Related Publication 20240104336A1 · Mar 28, 2024
References Cited (107)
US 20220358373A1 · Bucher · 2022 [cited by examiner]
US 20230252060A1 · Najumudeen · 2023 [cited by examiner]
Zhong et al, “SEQ2SQL: Generating Structured Queries From Natural Language Using Reinforcement Learning”, 2017, arXiv:1709.00103v7, pp. 1-12. (Year: 2017). [cited by examiner]
Askell et al, “Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback”, Apr. 2022, arXiv:2204.05862v1, pp. 1-74. (Year: 2022). [cited by examiner]
Yan et al, “Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System”, 2016, SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development … [cited by examiner]
Iyyer et al, “Search-based Neural Structured Learning for Sequential Question Answering”, 2017, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 1821-1831. (Year: 2017). [cited by examiner]
Ge et al, “Personalizing Search Results Using Hierarchical RNN with Query-aware Attention”, 2018, CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 347-356. (Yea… [cited by examiner]
Abadi et al., “Deep learning with differential privacy,” Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Oct. 2016, pp. 308-318. [cited by applicant]
Abdolmaleki et al., “A distributional view on multi-objective policy optimization,” International Conference on Machine Learning, 2020, 119:11-22. [cited by applicant]
Adolphs et al., “Reason first, then respond: Modular generation for knowledge-infused dialogue,” CoRR, Nov. 9, 2021, arXiv:2111.05204, 20 pages. [cited by applicant]
Askell et al., “A general language assistant as a laboratory for alignment,” CoRR, Dec. 1, 2021, arXiv:2112.00861, 48 pages. [cited by applicant]
Bai et al., “Training a helpful and harmless assistant with reinforcement learning from human feedback,” CoRR, Apr. 12, 2022, arXiv:2204.05862, 74 pages. [cited by applicant]
Banko et al., “A unified taxonomy of harmful content,” Proceedings of the Fourth Workshop on Online Abuse and Harms, Nov. 2020, pp. 125-137. [cited by applicant]
Bender et al., “On the dangers of stochastic parrots: Can language models be too big?,” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Mar. 2021, pp. 610-623. [cited by applicant]
Berditchevskaia et al., “Participatory AI for humanitarian innovation,” Nesta, United Kingdom Humanitarian Innovation Hub, Sep. 2021, 42 pages. [cited by applicant]
Birhane et al., “Power to the people? Opportunities and challenges for participatory AI,” Proceedings of the ACM conference on Equity and Access in Algorithms, Mechanisms, and Optimization, Oct. 6-9, 2022, 1-8. [cited by applicant]
Blodgett et al., “Language (technology) is power: A critical survey of “Bias” in NLP,” CoRR, May 28, 2020, arXiv:2005.14050, 23 pages. [cited by applicant]
Borgeaud et al., “Improving language models by retrieving from trillions of tokens,” Proceedings of the 39th International Conference on Machine Learning, 2022, 162:2206-2240. [cited by applicant]
Boyd-Graber et al., “Besting the Quiz Master: Crowdsourcing incremental classification games,” Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Langu… [cited by applicant]
Bradley et al., “Rank analysis of incomplete block designs: I. the method of paired comparisons,” Biometrika, Dec. 1952, 39(3/4):324-345. [cited by applicant]
Breitfeller et al., “Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th… [cited by applicant]
Brown et al., “Language models are few-shot learners,” Advances in Neural Information Processing Systems, 2020, 33:1877-1901. [cited by applicant]
Christiano et al., “Supervising strong learners by amplifying weak experts,” CoRR, Oct. 19, 2018, arXiv:1810.08575, 15 pages. [cited by applicant]
Cortes et al., “Advances in neural information processing systems,” 29th Annual Conference on Neural Information Processing Systems 2015, Dec. 7-12, 2015, 29:3909-3917. [cited by applicant]
Creswell et al., “Selection-inference: Exploiting large language models for interpretable logical reasoning,” CoRR, May 19, 2022, arXiv:2205.09712, 33 pages. [cited by applicant]
Dang et al., “But who protects the moderators? the case of crowdsourced image moderation,” CoRR, Apr. 29, 2018, arXiv:1804.10999, 5 pages. [cited by applicant]
Denton et al., “Bringing the people back in: Contesting benchmark machine learning datasets,” CoRR, Jul. 14, 2020, arXiv:2007.07399, 6 pages. [cited by applicant]
Dinan et al., “Build it break it fix it for dialogue safety: Robustness from adversarial human attack,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Jo… [cited by applicant]
Distill.pub [online], “AI Safety Needs Social Scientists,” Feb. 19, 2019, retrieved on Oct. 6, 2023, retrieved from URL<https://distill.pub/2019/safety-needs-social-scientists/>, 28 pages. [cited by applicant]
Dohan et al., “Language model cascades,” CoRR, Jul. 21, 2022, arXiv:2207.10342, 13 pages. [cited by applicant]
Dziri et al., “On the origin of hallucinations in conversational models: Is it the datasets or the models?,” Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguist… [cited by applicant]
El-Yaniv et al., “On the foundations of noise-free selective classification,” Journal of Machine Learning Research, May 2010, 11(5):1605-1641. [cited by applicant]
Evans et al., “Truthful AI: Developing and governing AI that does not lie,” CoRR, Oct. 13, 2021, arXiv:2110.06674, 2021, 96 pages. [cited by applicant]
Fan et al., “ELI5: Long form question answering,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 2019, pp. 3558-3567. [cited by applicant]
Gao et al., “Making pre-trained language models better few-shot learners,” CoRR, Dec. 31, 2020, arXiv:2012.15723, 15 pages. [cited by applicant]
Geifman et al., “Selective classification for deep neural networks,” Advances in neural information processing systems 30 (NIPS 2017), 2017, 10 pages. [cited by applicant]
Geifinan et al., “Selectivenet: A deep neural network with an integrated reject option,” International Conference on Machine Learning, 2019, 97:2151-2159. [cited by applicant]
Gershman, “How to never be wrong,” Psychonomic Bulletin & Review, May 24, 2018, 26:13-28. [cited by applicant]
Gor et al., “Toward deconfounding the effect of entity demographics for question answering accuracy,” Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Nov. 7-11, 2021,pp. 5457-5473. [cited by applicant]
Goulden et al., “Keeping women in the science pipeline,” The Annals of the American Academy of Political and Social Science, Oct. 4, 2011, 638(1):141-162. [cited by applicant]
Grice, “Logic and conversation,” Speech Acts, Dec. 12, 1975, pp. 41-58. [cited by applicant]
Gupta et al., “Deep learning with limited numerical,” Proceedings of the 32nd International Conference on Machine Learning, 2015, PMLR 37:1737-1746. [cited by applicant]
Hadfield-Menell et al., “Cooperative inverse reinforcement learning,” Advances in Neural Information Processing Systems 29 (NIPS 2016), 2016, 9 pages. [cited by applicant]
Halfaker et al., “Ores: Lowering barriers with participatory machine learning in wikipedia,” Proceedings of the ACM Conference on Human-Computer Interaction, Oct. 15, 2020, 4(CSCW2):1-37. [cited by applicant]
Hendrycks et al., “Aligning ai with shared human values,” CoRR, Aug. 5, 2020, arxiv.org/abs/2008.02275, 29 pages. [cited by applicant]
Hendrycks et al., “Measuring massive multitask language understanding,” CoRR, Sep. 7, 2020, arxiv.org/abs/2009.03300, 27 pages. [cited by applicant]
Hinton, “Training products of experts by minimizing contrastive divergence, ” Neural Computation, Aug. 1, 2002, 14(8):1771-1800. [cited by applicant]
Hoffman et al., “Training Compute-Optimal Large Language Models,” CoRR, Mar. 29, 2022, arXiv:2203.15556, 36 pages. [cited by applicant]
Holtzman et al., “The curious case of neural text degeneration,” CoRR, Apr. 22, 2019, arxiv.org/abs/1904.09751, 16 pages. [cited by applicant]
Irving et al., “AI safety via debate,” CoRR, May 2, 2018, arXiv:1805.00899, 24 pages. [cited by applicant]
Joshi et al., “TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension,” Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (vol. 1: Long Papers), Ju… [cited by applicant]
Kamath et al., “Selective question answering under domain shift,” CoRR, Jun. 16, 2020, arXiv:2006.09462, 13 pages. [cited by applicant]
Kasirzadeh et al., “In Conversation with Artificial Intelligence: Aligning language Models with Human Values,” Philosophy & Technology, Apr. 19, 2023, 36:27. [cited by applicant]
Kingma et al., “Adam; A method for stochastic optimization,” CoRR, Dec. 22, 2014, arXiv:1412.6980, 15 pages. [cited by applicant]
Kotonya et al., “Policy compliance detection via expression tree inference,” CoRR, May 24, 2022, arXiv:2205.12259, 10 pages. [cited by applicant]
Krippendorff, “Computing Krippendorff's alpha-reliability,” University of Pennsylvania, Jan. 25, 2011, 12 pages. [cited by applicant]
Kunda, “The case for motivated reasoning, psychological bulletin,” Psychological Bulletin, 1990, 108(3):480-498. [cited by applicant]
Kwiatkowski et al., “Natural questions: a benchmark for question answering research,” Transactions of the Association for Computational Linguistics, Aug. 1, 2019, 7:453-466. [cited by applicant]
Lazaridou et al., “Internet-augmented language models through few-shot prompting for open-domain question answering,” CoRR, Mar. 10, 2022, arXiv:2203.05115, 20 pages. [cited by applicant]
Lee et al., “WeBuildAI: Participatory framework for algorithmic governance,” Proceedings of the ACM on Human-Computer Interaction, Nov. 7, 2019, 3(181):1-35. [cited by applicant]
Leike et al., “Scalable agent alignment via reward modeling: a research direction.” CoRR, Nov. 19, 2018, arxiv.org/abs/1811.07871, 30 pages. [cited by applicant]
Lewis et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,” Advances in Neural Information Processing Systems 33, 2020, 33:9459-9474. [cited by applicant]
Lewkowycz et al., “Solving quantitative reasoning problems with language models,” Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 2022, 15 pages. [cited by applicant]
Lin et al., “TruthfulQA: Measuring how models mimic human falsehoods,” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (vol. 1; Long Papers), May 2022, pp. 3214-3252. [cited by applicant]
Liška et al., “StreamingQA; A benchmark for adaptation to new knowledge over time in question answering models,” International Conference on Machine Learning, 2022, 162:13604-13622. [cited by applicant]
McGuffie et al., “The radicalization risks of GPT-3 and advanced neural language models,” CoRR, Sep. 15, 2020, arXiv:2009.06807, 12 pages. [cited by applicant]
Menick et al., “Teaching language models to support answers with verified quotes,” CoRR, Mar. 21, 2022, arXiv:2203.11147, Mar. 21, 2022, 40 pages. [cited by applicant]
Mnih et al., “Asynchronous methods for deep reinforcement leaming,” Proceedings of the 33rd International Conference on International Conference on Machine Learning, 2016, 10 pages. [cited by applicant]
Nakano et al., “WebGPT: Browser-assisted question-answering with human feedback,” CoRR, Dec. 17, 2021, arXiv:2112.09332, 32 pages. [cited by applicant]
Open Ended Leaming Team et al., “Open-ended learning leads to generally capable agents,” CoRR, Jul. 27, 2021, arXiv:2107.12808, 54 pages. [cited by applicant]
Ouyang et al., “Training language models to follow instructions with human feedback,” CoRR, Mar. 4, 2022, arXiv:2203.02155, 68 pages. [cited by applicant]
Christiano et al., “Eliciting latent knowledge: How to tell if your eyes deceive you,” Alignment Research Center, Dec. 2021, 106 pages. [cited by applicant]
Parrish et al., “BBW: A hand-built bias benchmark for question answering,” CoRR, Oct. 15, 2021, arXiv:2110.08193, 16 pages. [cited by applicant]
Parrish et al., “Single-turn debate does not help humans answer hard reading-comprehension questions,” CoRR, Apr. 11, 2022, arXiv:2204.05212, 12 pages. [cited by applicant]
Perez et al., “Finding generalizable evidence by learning to convince Q&A models,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat… [cited by applicant]
Perez et al., “Red teaming language models with language models,” CoRR, Feb. 7, 2022, arXiv:2202.03286, 31 pages. [cited by applicant]
Rae et al., :“Scaling language models: Methods, analysis & insights from training gopher,” CoRR, Dec. 8, 2021, arXiv:2112.11446, 118 pages. [cited by applicant]
Ras et al., “Explainable deep learning: A field guide for the uninitiated,” Journal of Artificial Intelligence Research, 2022, 73:329-396. [cited by applicant]
Rauh et al., “Characteristics of harmful text: Towards rigorous benchmarking of language models,” CoRR, Jun. 16, 2022, arXiv:2206.08325, 28 pages. [cited by applicant]
Rudinger et al., “Gender bias in coreference resolution,” CoRR, Apr. 25, 2018,arXiv:1804.09301, 7 pages. [cited by applicant]
Saeidi et al., “Cross-policy compliance detection via question answering,” CoRR, Sep. 8, 2021, arXiv:2109.03731, 11 pages. [cited by applicant]
Saunders et al., “Self-critiquing models for assisting human evaluators,” CoRR, Jun. 12, 2022, arXiv:2206.05802,. [cited by applicant]
Shazeer et al., “Adafactor: Adaptive learning raes with sublinear memory cost,” Proceedings of the 35th International Conference on Machine Learning, 2018, 9 pages. [cited by applicant]
Shoeybi et al., “Megatron-LM: Training multi-billion parameter language models using model parallelism,” CoRR, Sep. 17, 2019, arXiv:1909.08053, 15 pages. [cited by applicant]
Shuster et al., “BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage,” CoRR, Aug. 5, 2022, arXiv:2208.03188, 38 pages. [cited by applicant]
Shuster et al., “Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion,” CoRR, Mar. 24, 2022, arXiv:2203.13224, 21 pages. [cited by applicant]
Sloane et al., “Participation is not a design fix for machine learning.” CoRR, Jul. 5, 2020, arXiv:2007.02423, 7 pages. [cited by applicant]
Song et al., “V-MPO: On-policy maximum a posteriori policy optimization for discrete and continuous control,” CoRR, Sep. 26, 2019, arXiv:1909.12238, 19 pages. [cited by applicant]
Steiger et al., “The psychological well-being of content moderators; the emotional labor of commercial moderation and avenues for improving support,” Proceedings of the 2021 CHI Conference on Human Factors in Computing … [cited by applicant]
Stiennon et al., “Learning to summarize with human feedback,” Advances in 34th Conference on Neural Information Processing Systems, 2020, 14 pages. [cited by applicant]
Thoppilan et al., “LaMDA: Language Models for Dialog Applications,” CoRR, Jan. 20, 2022, arXiv:2201.08239, 47 pages. [cited by applicant]
Tomasev et al., “Fairness for unobserved characteristics: Insights from technological impacts on queer communities,” Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 2021, pp. 254-265. [cited by applicant]
Transformer-circuits.pub [online], “A Mathematical Framework for Transformer Circuits,” Dec. 22, 2021, retrieved on Oct. 2, 2023, retrieved from URL<http://www.assaabloy.com/en/com/Press-News/News/2011/NFC-enabled-mobil… [cited by applicant]
Vidgen et al., “Challenges and frontiers in abusive content detection,” Proceedings of the Third Workshop on Abusive Language Online, Aug. 2019, pp. 80-93. [cited by applicant]
Vinyals et al., “Grandmaster level in StarCraft II using multi-agent reinforcement learning,” Nature, Nov. 2019, 575(7882):350-354. [cited by applicant]
Wei et al., “Finetuned language models are zero-shot learners,” Sep. 3, 2021, arXiv:2109.01652, 29 pages. [cited by applicant]
Weidinger et al., “Ethical and social risks of harm from language models,” CoRR, Dec. 8, 2021, arXiv:2112.04359, 64 pages. [cited by applicant]
Welbl et al., “Challenges in detoxifying language models,” Findings of the Association for Computational Linguistics: EMNLP 2021, Nov. 2021, pp. 2447-2469. [cited by applicant]
Wikisource.org [online], “The Art of Being Right,” last edited Nov. 27, 2022, retrieved on Oct. 19, 20232, retrieved from URL<https://en.wikisource.org/wiki/The_Art_of_Being_Right/>, 17 pages. [cited by applicant]
Wu et al., “Recursively summarizing books with human feedback,” CoRR, Sep. 22, 2021, arXiv:2109.10862, 36 pages. [cited by applicant]
Xu et al., “Bot-adversarial dialogue for safe conversational agents,” Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun. … [cited by applicant]
Xu et al., “Detoxifying language models risks marginalizing minority voices,” Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… [cited by applicant]
Zamani et al., “Conversational information seeking.” CoRR, Jan. 21, 2022, arXiv:2201.08808, 196 pages. [cited by applicant]
Zhang et al., “Hate speech detection: A solved problem? the challenging case of long tail on Twitter,” Semantic Web, Semantic Web, 2019, 1-21. [cited by applicant]
Zhao et al., “Gender bias in coreference resolution: Evaluation and debiasing methods,” CoRR, Apr. 18, 2018, arXiv:1804.06876, 6 pages. [cited by applicant]
Ziegler et al., “Fine-tuning language models from human preferences,” CoRR, Sep. 18, 2019, arXiv:1909.08593, 26 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/EP2023/075931, dated Feb. 26, 2024, 23 pages. [cited by applicant]
Cited By (1)
US 12,697,987