IP Library › Granted Patent US 12,505,291
Granted Patent B2
US 12,505,291 · App. 19/019,365 · Granted Dec 23, 2025

Identifying and analyzing actions from vector representations of alphanumeric characters using a large language model

Inventors: Shardul Malviya (London, GB); Wayne Liao (London, GB); Deepak Jain (London, GB); Samantha Cory (London, GB); Mariusz Saternus (Cracow, PL); Daniel Lewandowski (Cracow, PL); Biraj Krushna Rath (London, GB); Stuart Murray (London, GB); Philip Davies (London, GB); Payal Jain (London, GB); Tariq Husayn Maonah (London, GB)
G06F40/216
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,505,291
App. No.
19/019,365
Granted
Dec 23, 2025
Kind
B2
Abstract

The systems and methods disclosed herein receive an output generation request from that includes input for generating an output using a language model. The input includes a set of alphanumeric characters associated with operative standards for a first set of actions. The system divides the set of alphanumeric characters into text subsets. For each text subset, a vector representation is determined. Prompts are created for each vector representation including the set of alphanumeric characters, query contexts, keywords, and/or the text subset. Each vector representation's prompt is input into the language model, which generates a second set of actions of related actions, where subsequently generated actions are based on prior generated actions. The system aggregates the second set of actions into a third set of actions and displays a graphical layout. The graphical layout displays a representation of the set of alphanumeric characters and the corresponding actions.

Claims (84)

1 . A non-transitory computer-readable storage medium comprising instructions stored thereon, wherein the instructions when executed by at least one data processor of a system, cause the system to:

obtain (1) an output generation request for generation of an output and (2) one or more indicators of a first set of actions,

wherein the output generation request includes one or more indicators of a set of guidelines, and

wherein the first set of actions are configured to adhere to constraints of the set of guidelines;

using the one or more indicators of the set of guidelines, determine a set of inputs for one or more guidelines in the set of guidelines,

wherein at least one input of the set of inputs for a particular guideline includes one or more of: (1) the particular guideline, (2) a set of pre-loaded query context defining the first set of actions, (3) a set of keywords associated with a set of textual content of corresponding guidelines of the first set of actions, or (4) the set of textual content; and

associate, using an AI model, the one or more guidelines of the set of guidelines to one or more actions in the first set of actions by:

supplying one or more inputs of the set of inputs of a corresponding guideline into the AI model, and

receiving, from the AI model, a generated second set of actions including the one or more actions of the first set of actions associated with the corresponding guideline of the set of guidelines.

2 . The non-transitory, computer-readable storage medium of claim 1 , wherein the set of guidelines includes one or more of: text, image, audio, or video format data.

3 . The non-transitory, computer-readable storage medium of claim 1 , wherein the AI model is a first AI model, and wherein the instructions further cause the system to:

supply the set of guidelines into a second AI model;

receive, from the second AI model, a set of summaries summarizing the set of guidelines; and

include at least one summary in the set of inputs.

4 . The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

identify sensitive information within the set of guidelines using at least one predefined indicator of the sensitive information; and

mask the identified sensitive information.

5 . The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

receive an indicator of a type of operation associated with the set of guidelines; and

obtain, via an application programming interface (API), the set of guidelines.

6 . The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

associate each guideline with a type of guideline; and

generate the second set of actions using the type of guideline of each guideline in the set of guidelines.

7 . The non-transitory, computer-readable storage medium of claim 1 , wherein the instructions further cause the system to:

validate the generated second set of actions by:

comparing each action in the second set of actions against predefined validation criteria; and

generating a validation score for each action based on the comparisons.

8 . A system comprising:

at least one hardware processor; and

at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:

obtain (1) an output generation request for generation of an output and (2) one or more indicators of a first set of actions,

wherein the output generation request includes one or more indicators of a set of guidelines, and

wherein the first set of actions are configured to adhere to constraints of the set of guidelines;

using the one or more indicators of the set of guidelines, determine a set of information for one or more guidelines in the set of guidelines,

wherein at least a portion of the set of information for a particular guideline includes one or more of: (1) the particular guideline, (2) a set of pre-loaded query context defining the first set of actions, (3) a set of keywords associated with a set of textual content of corresponding guidelines of the first set of actions, or (4) the set of textual content; and

associate, using an AI model, the one or more guidelines of the set of guidelines to one or more actions in the first set of actions by:

supplying the set of information of a corresponding guideline into the AI model, and

receiving, from the AI model, a generated second set of actions including the one or more actions of the first set of actions associated with the corresponding guideline of the set of guidelines.

9 . The system of claim 8 , wherein the system is further caused to:

calculate a set of token counts within the set of guidelines; and

partition the set of guidelines into a set of segments based on predetermined character limits of the set of token counts.

10 . The system of claim 8 , wherein the one or more indicators of the set of guidelines include one or more of:

electronic tags associated with guideline types,

metadata indicating operative standard categories, or

scenario attributes defining one or more operational contexts.

11 . The system of claim 8 , wherein the set of information include one or more of:

vector representations of alphanumeric characters,

textual content extracted from the guidelines using natural language processing, or

pre-established mappings between operative standards and identified gaps.

12 . The system of claim 8 , wherein the system is further caused to:

generate a confidence score for each association between the one or more guidelines and a third set of actions; and

generate the second set of actions by filtering the associations based on a predetermined confidence threshold.

13 . The system of claim 8 , wherein the system is further caused to:

detect a set of changes in the set of guidelines; and

automatically update the second set of actions based on the detected changes.

14 . The system of claim 8 , wherein the system is further caused to:

generate a set of metadata documenting the association between the one or more guidelines and the one or more actions,

wherein the set of metadata includes one or more of: a timestamp, model version, or a confidence metric of the association.

15 . A method performed by a computer device for associating guidelines with one or more actions, the method comprising:

obtain (1) an output generation request for generation of an output and (2) one or more indicators of a first set of actions,

wherein the output generation request includes one or more indicators of a set of guidelines, and

wherein the first set of actions are configured to adhere to constraints of the set of guidelines;

using the one or more indicators of the set of guidelines, determine a set of information for one or more guidelines in the set of guidelines,

wherein the set of information for a particular guideline includes one or more of: (1) the particular guideline, (2) a set of pre-loaded query context defining the first set of actions, (3) a set of keywords associated with a set of textual content of corresponding guidelines of the first set of actions, or (4) the set of textual content; and

causing an AI model to associate the one or more guidelines of the set of guidelines to one or more actions in the first set of actions using at least a portion of the set of information of a corresponding guideline.

16 . The method of claim 15 , further comprising:

generating a set of association rules using historically associated guidelines; and

supplying the association rules to the AI model to generate subsequently associated guidelines associated subsequent to the historically associated guidelines.

17 . The method of claim 15 , further comprising:

classifying each guideline in the set of guidelines in a predefined domain of a set of predefined domains,

wherein each predefined domain is associated with a domain-specific query context; and

selecting, a particular domain-specific query context for the AI model using a corresponding predefined domain.

18 . The method of claim 15 , further comprising obtaining a knowledge graph including:

temporal dependencies between one or more guidelines,

contextual variations of the set of guidelines, and

historical patterns of associations between one or more guidelines and one or more actions.

19 . The method of claim 15 , further comprising:

identifying a set of relationships between the set of guidelines and the set of actions,

wherein each relationship in the set of relationship corresponds to a type of relationship;

assigning, for each relationship in the set of relationship, a weighted score using the type of relationship; and

aggregating the weighted scores to generate an overall score of the one or more guidelines,

wherein the AI model is caused to associate the one or more guidelines to the one or more actions using the overall score.

20 . The method of claim 15 , further comprising:

assigning a version to one or more of: (1) the one or more guidelines or (2) the one or more actions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2025
From: MALVIYA, SHARDUL; LIAO, WAYNE; JAIN, DEEPAK; CORY, SAMANTHA; SATERNUS, MARIUSZ; LEWANDOWSKI, DANIEL; RATH, BIRAJ KRUSHNA; MURRAY, STUART; DAVIES, PHILIP; JAIN, PAYAL; MAONAH, TARIQ HUSAYN
To: CITIBANK, N.A.
Reel/Frame 070416/0545 →
Continuity (6)
Continuation 18782019 · Jul 23, 2024
Continuation In Part 18771876 · Jul 12, 2024
Continuation In Part 18661532 · May 10, 2024
Continuation In Part 18661519 · May 10, 2024
Continuation In Part 18633293 · Apr 11, 2024
Related Publication 20250322154A1 · Oct 16, 2025
References Cited (125)
US 9842045B2 · Heorhiadi et al. · 2017 [cited by applicant]
US 10324827B2 · Narayanan et al. · 2019 [cited by applicant]
US 10764150B1 · Hermoni et al. · 2020 [cited by applicant]
US 10949337B1 · Yalla et al. · 2021 [cited by applicant]
US 10951485B1 · Hermoni et al. · 2021 [cited by applicant]
US 11106801B1 · Levine et al. · 2021 [cited by applicant]
US 11153177B1 · Hermoni et al. · 2021 [cited by applicant]
US 11227047B1 · Vashisht et al. · 2022 [cited by applicant]
US 11271822B1 · Hermoni et al. · 2022 [cited by applicant]
US 11449798B2 · Olgiati et al. · 2022 [cited by applicant]
US 11470106B1 · Lin et al. · 2022 [cited by applicant]
US 11503075B1 · Sirianni et al. · 2022 [cited by applicant]
US 11516158B1 · Luzhnica et al. · 2022 [cited by applicant]
US 11573848B2 · Linck et al. · 2023 [cited by applicant]
US 11636027B2 · Sloane · 2023 [cited by applicant]
US 11652839B1 · Aloisio et al. · 2023 [cited by applicant]
US 11656852B2 · Mazurskiy · 2023 [cited by applicant]
US 11681811B1 · Dixit · 2023 [cited by applicant]
US 11683333B1 · Dominessy et al. · 2023 [cited by applicant]
US 11750717B2 · Walsh et al. · 2023 [cited by applicant]
US 11875123B1 · Ben David et al. · 2024 [cited by applicant]
US 11875130B1 · Bosnjakovic et al. · 2024 [cited by applicant]
US 11924027B1 · Mysore et al. · 2024 [cited by applicant]
US 11947435B2 · Boulineau et al. · 2024 [cited by applicant]
US 11960386B2 · Indani et al. · 2024 [cited by applicant]
US 11960515B1 · Pallakonda et al. · 2024 [cited by applicant]
US 11983806B1 · Ramesh et al. · 2024 [cited by applicant]
US 11990139B1 · Sandrew · 2024 [cited by applicant]
US 11995412B1 · Mishra · 2024 [cited by applicant]
US 12001463B1 · Pallakonda et al. · 2024 [cited by applicant]
US 12026599B1 · Lewis et al. · 2024 [cited by applicant]
US 12106205B1 · Jain et al. · 2024 [cited by applicant]
US 12111747B1 · Jain et al. · 2024 [cited by applicant]
US 12147513B1 · Jain et al. · 2024 [cited by applicant]
US 20170262164A1 · Jain · 2017 [cited by examiner]
US 20180089252A1 · Long et al. · 2018 [cited by applicant]
US 20180095866A1 · Narayanan et al. · 2018 [cited by applicant]
US 20190079854A1 · Lassance Oliveira E Silva et al. · 2019 [cited by applicant]
US 20200043164A1 · Fuchs et al. · 2020 [cited by applicant]
US 20200334326A1 · Zhang et al. · 2020 [cited by applicant]
US 20210012486A1 · Huang et al. · 2021 [cited by applicant]
US 20210097433A1 · Olgiati et al. · 2021 [cited by applicant]
US 20220114399A1 · Castiglione et al. · 2022 [cited by applicant]
US 20220179906A1 · Desai · 2022 [cited by examiner]
US 20220198304A1 · Szczepanik et al. · 2022 [cited by applicant]
US 20220303302A1 · Hwang et al. · 2022 [cited by applicant]
US 20220311681A1 · Palladino · 2022 [cited by examiner]
US 20220318654A1 · Lin et al. · 2022 [cited by applicant]
US 20220327620A1 · Ndoutoumou · 2022 [cited by applicant]
US 20220342846A1 · Kunchakarra et al. · 2022 [cited by applicant]
US 20220345457A1 · Jeffords et al. · 2022 [cited by applicant]
US 20220358023A1 · Moser et al. · 2022 [cited by applicant]
US 20220366140A1 · Saito · 2022 [cited by examiner]
US 20220368728A1 · Murray et al. · 2022 [cited by applicant]
US 20220377093A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220400135A1 · Gamra · 2022 [cited by applicant]
US 20220414213A1 · Dixit · 2022 [cited by applicant]
US 20220414536A1 · M L et al. · 2022 [cited by applicant]
US 20230007039A1 · Waplington · 2023 [cited by applicant]
US 20230009999A1 · Higuchi et al. · 2023 [cited by applicant]
US 20230019072A1 · Okunlola · 2023 [cited by applicant]
US 20230028339A1 · Sloane · 2023 [cited by applicant]
US 20230033317A1 · Lin et al. · 2023 [cited by applicant]
US 20230039855A1 · Greene · 2023 [cited by applicant]
US 20230076795A1 · Indani et al. · 2023 [cited by applicant]
US 20230113621A1 · Griffin et al. · 2023 [cited by applicant]
US 20230164158A1 · Fellows et al. · 2023 [cited by applicant]
US 20230171282A1 · Bollinger · 2023 [cited by applicant]
US 20230177441A1 · Durvasula et al. · 2023 [cited by applicant]
US 20230177613A1 · Crabtree et al. · 2023 [cited by applicant]
US 20230186175A1 · Usatov et al. · 2023 [cited by applicant]
US 20230252393A1 · Orzechowski et al. · 2023 [cited by applicant]
US 20230269272A1 · Dambrot et al. · 2023 [cited by applicant]
US 20230274003A1 · Liu et al. · 2023 [cited by applicant]
US 20230359789A1 · Andre et al. · 2023 [cited by applicant]
US 20240012734A1 · Lee et al. · 2024 [cited by applicant]
US 20240020538A1 · Socher · 2024 [cited by examiner]
US 20240095077A1 · Singh · 2024 [cited by examiner]
US 20240129345A1 · Kassam et al. · 2024 [cited by applicant]
US 20240202442A1 · Saito · 2024 [cited by examiner]
US 20240346283A1 · Ayachitula et al. · 2024 [cited by applicant]
US 20240370476A1 · Madisetti et al. · 2024 [cited by applicant]
US 20250119494A1 · Pandey · 2025 [cited by examiner]
CN 106502890A · 2017 [cited by applicant]
WO WO2022125803A1 · 2022 [cited by examiner]
WO 2024020416A1 · 2024 [cited by applicant]
AI Risk Management Framework NIST, retrieved on Jun. 17, 2024, https://www.nist.gov/itl/ai-risk-management-framework. [cited by applicant]
Empower Your Team with a Compliance Co-Pilot, Sedric, retrieved on Sep. 25, 2024. https://www.sedric.ai/. [cited by applicant]
Independent analysis of AI language models and API providers. Artificial Analysis, retrieved on Jun. 13, 2024, https://artificialanalysis.ai/, 11 pages. [cited by applicant]
What is AI Verify?, AI Verify Foundation, Jun. 11, 2024, 3 pages, https://aiverifyfoundation.sg/. [cited by applicant]
Brown, D., et al., “The Great AI Challenge: We Test Five Top Bots on Useful, Everyday Skills,” The Wall Street Journal, published May 25, 2024. [cited by applicant]
Cranium, Adopt & Accelerate AI Safely, retrieved on Nov. 7, 2024, from https://cranium.ai/. [cited by applicant]
Dong, Y., et al., “Building Guardrails for Large Language Models,” https://ar5iv.labs.arxiv.org/html/2402.01822v1, published May 29, 2024, 20 pages. [cited by applicant]
Futurism, “Sam Altman Admits That OpenAI Doesn't Actually Understand How Its AI Works”, Jun. 11, 2024, 4 pages, https://futurism.com/sam-altman-admits-openai-understand-ai. [cited by applicant]
Generative machine learning models; IPCCOM000272835D, Aug. 17, 2023. (Year: 2023). [cited by applicant]
Guldimann, P., et al. “COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act,” arXiv:2410.07959v1 [cs.CL] Oct. 10, 2024, 38 pages. [cited by applicant]
Hu, Q., J., et al., “RouterBench: A Benchmark for Multi-LLM Routing System,” arXiv:2403.12031v2 [cs.LG] Mar. 28, 2024, 16 pages. [cited by applicant]
International Search Report and Written Opinion received in Application No. PCT/US24/47571, dated Dec. 9, 2024, 10 pages. [cited by applicant]
Kojima, Takeshi, et al. “Large Language Models are Zero-Shot Reasoners,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), arXiv:2205.11916 [cs.CL], Jan. 29, 2023, 42 pages. [cited by applicant]
Mathews, A. W., “What AI Can Do in Healthcare—and What It Should Never Do,” The Wall Street Journal, published on Aug. 21, 2024, retrieved on Sep. 5, 2024 https://www.wsj.com. [cited by applicant]
Mavrepis, P., et al., “XAI for All: Can Large Language Models Simplify Explainable AI?,” https://arxiv.org/abs/2401.13110, Jan. 23, 2024, 10 pages. [cited by applicant]
Mollick, E., “Latent Expertise: Everyone is in R&D,” One Useful Thing, published on Jun. 20, 2024, https://www.oneusefulthing.org/p/latent-expertise-everyone-is-in-r. [cited by applicant]
Nauta, M., et al., “From Anecdotal Evidence to Quantative Evaluation Methods: A Systematic Review of Evaluating Explainable AI” ACM Computing Surveys, vol. 55 No. 13s Article 295, 2023 [retrieved Jul. 3, 2024]. [cited by applicant]
Peers, M., “What California AI Bill Could Mean,” The Briefing, published and retrieved Aug. 30, 2024, 8 pages, https://www.theinformation.com/articles/what-california-ai-bill-could-mean. [cited by applicant]
Wei, Jason, et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), arXiv:2201.11903 [cs.CL], Jan. 10, 2023, 43 pages. [cited by applicant]
Zhao, H., et al., “Explainability for Large Language Models: A Survey,” https://arxiv.org/abs/2309.01029, Nov. 28, 2024, 38 pages. [cited by applicant]
Zhou, Y., Liu, Y., Li, X., Jin, J., Qian, H., Liu, Z., Li, C., Dou, Z., Ho, T., & Yu, P. S. (2024). Trustworthiness in Retrieval-Augmented Generation Systems: A Survey. ArXiv./abs/2409.10102. [cited by applicant]
Aggarwal, Nitin , “Why measuring your new AI is essential to its succes”, KPIs for gen AI: Why measuring your new AI is essential to its succes, 7 pages. [cited by applicant]
Anthrop/C , “Mapping the Mind of a Large Language Model”, Mapping the Mind of a Large Language Model, May 21, 2024. [cited by applicant]
Claburn, Thomas , “OpenAI's GPT-4 can exploit real vulnerabilities by reading security advisories”, OpenAI's GPT-4 can exploit real vulnerabilities by reading security advisories, Apr. 17, 2024, 3 pages. [cited by applicant]
Marshall, Andrew , “Threat Modeling AI/ML Systems and Dependencies”, Threat Modeling AI/ML Systems and Dependencies, Nov. 2, 2022, 27 pages. [cited by applicant]
Roose, Kevin , “A.I. Has a Measurement Problem”, A.I. Has a Measurement Problem, Apr. 15, 2024, 5 pages. [cited by applicant]
Roose, Kevin , “A.I.'s Black Boxes Just Got a Little Less Mysterious”, A.I.'s Black Boxes Just Got a Little Less Mysterious, May 21, 2024, 5 pages. [cited by applicant]
Shah, Harshay , “Decomposing and Editing Predictions by Modeling Model Computation”, Decomposing and Editing Predictions by Modeling Model Computation, 5 pages. [cited by applicant]
Shankar, Ram , “Failure Modes in Machine Learning”, , Nov. 2019, 14 pages. [cited by applicant]
Teo, Josephine , “Singapore launches Project Moonshot”, Singapore launches Project Moonshot—a generative Artificial Intelligence testing toolkit to address LLM safety and security challenges, May 31, 2024, 8 pages. [cited by applicant]
Agarwal et al., How generative AI can help banks manage risk and compliance, McKinsey & Company, Mar. 2024; Total pp. 8 (Year: 2024). [cited by applicant]
Extended European Search Report issued Sep. 2, 2025, in EP Application No. 25169838.7 (13 pages). [cited by applicant]
International Search Report and Written Opinion received in Application No. PCT/US25/24406, dated Jul. 18, 2025, 10 pages. [cited by applicant]
Lai et al., Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies, arXiv:2112.11471v1 [cs.AI] Dec. 21, 2021; Total pp. 36 (Year: 2021). [cited by applicant]
Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools, 37th Conference on Neural Information Processing Systems (NeurIPS 2023); Total pp. 13 (Year: 2023). [cited by applicant]
Sumuk Shashidhar et al., ‘Democratizing LLMs: An Exploration of Cost-Performance Trade-offs in Self-Refined Open-Source Models’, arXiv:2310.07611v2, pp. 1-15, Oct. 2023. [cited by applicant]
Yang Liu et al., ‘Trustworthy LLMS: A Survey and Guideline for Evaluating Large Language Models' Alignment’, arXiv:2308.05374v2, pp. 1-81, Mar. 2024. [cited by applicant]
Yuan et al., R-Judge: Benchmarking Safety Risk Awareness for LLM Agents, arXiv:2401.10019v1 [cs.CL] Jan. 18, 2024; Total pp. 23 (Year: 2024). [cited by applicant]
Kim, Jaewoong , “From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process”, arXiv: 2402.01717v1 [cs.CL] Jan. 26, 2024; Total pp. 9 (Year: 2024). [cited by applicant]