IP Library Granted Patent US 12,608,486
Granted Patent B2
US 12,608,486 · App. 18/607,141 · Granted Apr 21, 2026

Generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data systems and methods

Inventors: William Franklin Cameron (Jacksonville, FL); Pramod Goyal (Ahmedabad, IN); Prithvi Narayana Rao (Allen, TX); Manjit Rajaretnam (Irving, TX); Miriam Silver (New York, NY)
Assignee: CITIBANK, N.A.
G06F21/577G06F21/552
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,608,486
App. No.
18/607,141
Filed
Mar 15, 2024
Granted
Apr 21, 2026
Kind
B2
Art Unit
2435
USPC
726/22
Abstract

Systems and methods for generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data are disclosed. The system accesses multi-modal data indicating a set of security information related to a computing system. The system then generates a set of extracted characteristics indicating a cyber-security attack on the computing system, via a supervised machine learning model, using the multi-modal data. Using this information, the system generates a revised set of extracted characteristics indicating the cyber-security attack, via an unsupervised machine learning model, using the extracted set of characteristics indicating the cyber-security attack, where the revised set of characteristics includes at least one new characteristic that was not included in the extracted set of characteristics indicating the cyber-security attack on the computing system. The system then generates for display a graphical representation of the revised set of extracted characteristics.

Claims (72)

1 . A system for generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data, the system comprising:

at least one hardware processor; and

at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the system to:

access multi-modal data indicating a set of security information related to a computing system of an entity, the multi-modal data comprising (i) security attributes of computing system components of the computing system, (ii) security policy information of the computing system of the entity, and (iii) third party-derived security vulnerability information;

apply a supervised machine learning model to the multi-modal data to generate a set of extracted characteristics indicating a cyber-security attack on the computing system of the entity,

wherein the supervised machine learning model is trained on historical multi-modal data indicating labeled subsets of the historical multi-modal data, and

wherein the labeled subsets of the historical multi-modal data indicate whether the subset of the historical multi-modal data indicates a cyber-security attack;

apply an embedding model to the extracted set of characteristics indicating the cyber-security attack to generate a vectorized representation of the extracted set of characteristics indicating the cyber-security attack;

apply an unsupervised machine learning model to the vectorized representation of the extracted set of characteristics indicating the cyber-security attack to generate a revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity,

wherein the revised set of extracted characteristics includes at least one new characteristic that was not included in the extracted set of characteristics indicating the cyber-security attack on the computing system of the entity;

execute a mitigation action responsive to the revised set of extracted characteristics indicating the cyber-security attack, wherein the mitigation action comprises a platform usage mitigation action, a data storage mitigation action, a cryptography mitigation action, a network communication mitigation action, or a client code integration mitigation action; and

generate for display, at a graphical user interface (GUI), a graphical representation of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity.

2 . The system of claim 1 , wherein the instructions, which, when executed by the at least one hardware processor, cause the system to:

store the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity in a database; and

in response to storing the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity, cause a training routine to be performed on the supervised machine learning model, the training routine comprising updating one or more parameters of the supervised machine learning model based on the revised set of characteristics indicating the cyber-security attack.

3 . The system of claim 2 , wherein the instructions, which, when executed by the at least one hardware processor, cause the system to:

prior to causing the training routine to be performed on the supervised machine learning model, receive a verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from a user; and

in response to receiving a positive verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from the user, cause the training routine to be performed on the supervised machine learning model.

4 . The system of claim 1 , wherein the supervised machine learning model comprises a set of supervised machine learning models, and wherein the instructions, which, when executed by the at least one hardware processor, cause the system to:

categorize the multi-modal data into a set of categories corresponding to a data format of the security information related to the computing system of the entity;

determine, for each supervised machine learning model of the set of supervised machine learning models, a data format that the respective supervised machine learning model is configured to receive; and

generate the set of extracted characteristics indicating the cyber-security attack on the computing system of the entity by:

for each category of the set of categories of the multi-modal data, providing the respective category of the multi-modal data as input to a respective supervised machine learning model based on a matching data format of (i) the respective category and (ii) the respective supervised machine learning model.

5 . The system of claim 1 , wherein the unsupervised machine learning model comprises a generative adversarial network (GAN), the generative adversarial network comprising a generator component and a discriminator component, the generator component being a variational auto encoder (VAE) configured to generate synthetic data representing characteristics of the cyber-security attack on the computing system of the entity.

6 . The system of claim 1 , wherein the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity is provided to a blackboard machine learning model to generate computing-aspect impact levels reflecting a measure of security for a second computing system platform.

7 . A method for generating predicted end-to-end cyber-security attack characteristics via bifurcated machine learning-based processing of multi-modal data, the method comprising:

accessing multi-modal data indicating a set of security information related to a computing system of an entity;

applying a supervised machine learning model to the multi-modal data to generate a set of extracted characteristics indicating a cyber-security attack on the computing system of the entity,

wherein the supervised machine learning model is trained on historical multi-modal data indicating labeled subsets of the historical multi-modal data, and

wherein the labeled subsets of the historical multi-modal data indicate whether the subset of the historical multi-modal data indicates a cyber-security attack;

applying an unsupervised machine learning model to the extracted set of characteristics indicating the cyber-security attack to generate a revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity,

wherein the revised set of extracted characteristics includes at least one new characteristic that was not included in the extracted set of characteristics indicating the cyber-security attack on the computing system of the entity;

execute a mitigation action responsive to the revised set of extracted characteristics indicating the cyber-security attack, wherein the mitigation action comprises a platform usage mitigation action, a data storage mitigation action, a cryptography mitigation action, a network communication mitigation action, or a client code integration mitigation action; and

generating for display a graphical representation of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity.

8 . The method of claim 7 , further comprising:

applying an embedding model to the extracted set of characteristics indicating the cyber-security attack to generate a vectorized representation of the extracted set of characteristics indicating the cyber-security attack.

9 . The method of claim 7 , wherein the multi-modal data comprises (i) security attributes of computing system components of the computing system, (ii) security policy information of the computing system of the entity, and (iii) third party-derived security vulnerability information.

10 . The method of claim 7 , further comprising:

storing the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity in a database; and

in response to storing the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity, causing a training routine to be performed on the supervised machine learning model, the training routine comprising updating one or more parameters of the supervised machine learning model based on the revised set of characteristics indicating the cyber-security attack.

11 . The method of claim 10 , further comprising:

prior to causing the training routine to be performed on the supervised machine learning model, receiving a verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from a user; and

in response to receiving a positive verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from the user, causing the training routine to be performed on the supervised machine learning model, wherein the positive verification indication of the revised set of extracted characteristics is received.

12 . The method of claim 7 , wherein the supervised machine learning model comprises a set of supervised machine learning models, and wherein the method further comprises:

categorizing the multi-modal data into a set of categories corresponding to a data format of the security information related to the computing system of the entity;

determining, for each supervised machine learning model of the set of supervised machine learning models, a data format that the respective supervised machine learning model is configured to receive; and

generating the set of extracted characteristics indicating the cyber-security attack on the computing system of the entity by:

for each category of the set of categories of the multi-modal data, providing the respective category of the multi-modal data as input to a respective supervised machine learning model based on a matching data format of (i) the respective category and (ii) the respective supervised machine learning model.

13 . The method of claim 7 , wherein the unsupervised machine learning model comprises a generative adversarial network (GAN), the generative adversarial network comprising a generator component and a discriminator component, the generator component being a variational auto encoder (VAE) configured to generate synthetic data representing characteristics of the cyber-security attack on the computing system of the entity.

14 . The method of claim 7 , wherein the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity is provided to a blackboard machine learning model to generate computing-aspect impact levels reflecting a measure of security for a second computing system platform.

15 . One or more non-transitory, computer-readable media comprising instructions that, when executed by one or more processors, cause operations comprising:

accessing multi-modal data indicating a set of security information related to a computing system of an entity;

applying a supervised machine learning model to the multi-modal data to generate a set of extracted characteristics indicating a cyber-security attack on the computing system of the entity,

wherein the supervised machine learning model is trained on historical multi-modal data indicating labeled subsets of the historical multi-modal data, and

wherein the labeled subsets of the historical multi-modal data indicate whether the subset of the historical multi-modal data indicates a cyber-security attack;

applying an unsupervised machine learning model to the extracted set of characteristics indicating the cyber-security attack to generate a revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity,

wherein the revised set of extracted characteristics includes at least one new characteristic that was not included in the extracted set of characteristics indicating the cyber-security attack on the computing system of the entity;

execute a mitigation action responsive to the revised set of extracted characteristics indicating the cyber-security attack, wherein the mitigation action comprises a platform usage mitigation action, a data storage mitigation action, a cryptography mitigation action, a network communication mitigation action, or a client code integration mitigation action; and

generating for display a graphical representation of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity.

16 . The media of claim 15 , wherein the instructions, when executed by the one or more processors, further cause operations comprising:

storing the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity in a database; and

in response to storing the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity, causing a training routine to be performed on the supervised machine learning model, the training routine comprising updating one or more parameters of the supervised machine learning model based on the revised set of characteristics indicating the cyber-security attack.

17 . The media of claim 16 , wherein the instructions, when executed by the one or more processors, further cause operations comprising:

prior to causing the training routine to be performed on the supervised machine learning model, receiving a verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from a user; and

in response to receiving a positive verification indication of the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity from the user, causing the training routine to be performed on the supervised machine learning model.

18 . The media of claim 15 , wherein the supervised machine learning model comprises a set of supervised machine learning models, and wherein the instructions, when executed by the one or more processors, further cause operations comprising:

categorizing the multi-modal data into a set of categories corresponding to a data format of the security information related to the computing system of the entity;

determining, for each supervised machine learning model of the set of supervised machine learning models, a data format that the respective supervised machine learning model is configured to receive; and

generating the set of extracted characteristics indicating the cyber-security attack on the computing system of the entity by:

for each category of the set of categories of the multi-modal data, providing the respective category of the multi-modal data as input to a respective supervised machine learning model based on a matching data format of (i) the respective category and (ii) the respective supervised machine learning model.

19 . The media of claim 15 , wherein the unsupervised machine learning model comprises a generative adversarial network (GAN), the generative adversarial network comprising a generator component and a discriminator component, the generator component being a variational auto encoder (VAE) configured to generate synthetic data representing characteristics of the cyber-security attack on the computing system of the entity.

20 . The media of claim 15 , wherein the revised set of extracted characteristics indicating the cyber-security attack on the computing system of the entity is provided to a blackboard machine learning model to generate computing-aspect impact levels reflecting a measure of security for a second computing system platform.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2024
From: CAMERON, WILLIAM FRANKLIN; GOYAL, PRAMOD; RAO, PRITHVI NARAYANA; RAJARETNAM, MANJIT; SILVER, MIRIAM
To: CITIBANK, N.A.
Reel/Frame 069147/0694 →
Continuity (5)
Continuation In Part 18399422 · Dec 28, 2023
Continuation 18327040 · May 31, 2023
Continuation In Part 18114194 · Feb 24, 2023
Continuation In Part 18098895 · Jan 19, 2023
Related Publication 20250165616A1 · May 22, 2025
References Cited (250)
US 8380817B2 · Okada · 2013 [cited by applicant]
US 8387020B1 · Maclachlan et al. · 2013 [cited by applicant]
US 9842045B2 · Heorhiadi et al. · 2017 [cited by applicant]
US 10620988B2 · Lauderdale et al. · 2020 [cited by applicant]
US 10943067B1 · Brown et al. · 2021 [cited by applicant]
US 11042647B1 · Joyce et al. · 2021 [cited by applicant]
US 11106801B1 · Levine et al. · 2021 [cited by applicant]
US 11133942B1 · Griffin · 2021 [cited by applicant]
US 11227047B1 · Vashisht et al. · 2022 [cited by applicant]
US 11227187B1 · Weinberger · 2022 [cited by applicant]
US 11290483B1 · Kannan et al. · 2022 [cited by applicant]
US 11328068B1 · Niedzwiedz et al. · 2022 [cited by applicant]
US 11410136B2 · Cook et al. · 2022 [cited by applicant]
US 11470106B1 · Lin et al. · 2022 [cited by applicant]
US 11503075B1 · Sirianni et al. · 2022 [cited by applicant]
US 11516222B1 · Srinivasan et al. · 2022 [cited by applicant]
US 11573848B2 · Linck et al. · 2023 [cited by applicant]
US 11652839B1 · Aloisio et al. · 2023 [cited by applicant]
US 11656852B2 · Mazurskiy · 2023 [cited by applicant]
US 11681811B1 · Dixit · 2023 [cited by applicant]
US 11683333B1 · Dominessy et al. · 2023 [cited by applicant]
US 11706241B1 · Cross et al. · 2023 [cited by applicant]
US 11720686B1 · Cross et al. · 2023 [cited by applicant]
US 11734418B1 · Epstein · 2023 [cited by applicant]
US 11741226B2 · Dixit · 2023 [cited by applicant]
US 11750717B2 · Walsh et al. · 2023 [cited by applicant]
US 11810185B2 · Joglekar · 2023 [cited by examiner]
US 11875123B1 · Ben David et al. · 2024 [cited by applicant]
US 11875130B1 · Bosnjakovic et al. · 2024 [cited by applicant]
US 11924027B1 · Mysore et al. · 2024 [cited by applicant]
US 11947435B2 · Boulineau et al. · 2024 [cited by applicant]
US 11960515B1 · Pallakonda et al. · 2024 [cited by applicant]
US 11983806B1 · Ramesh et al. · 2024 [cited by applicant]
US 11990139B1 · Sandrew · 2024 [cited by applicant]
US 11995412B1 · Mishra · 2024 [cited by applicant]
US 12001463B1 · Pallakonda et al. · 2024 [cited by applicant]
US 12026599B1 · Lewis et al. · 2024 [cited by applicant]
US 12028368B1 · Cohen et al. · 2024 [cited by applicant]
US 12107869B1 · Kannan et al. · 2024 [cited by applicant]
US 12135949B1 · Cameron et al. · 2024 [cited by applicant]
US 12149553B1 · Fly et al. · 2024 [cited by applicant]
US 12149558B1 · Brown et al. · 2024 [cited by applicant]
US 12155781B1 · Helfgott et al. · 2024 [cited by applicant]
US 12182258B2 · Stokes et al. · 2024 [cited by applicant]
US 12271720B1 · Silver · 2025 [cited by examiner]
US 12299134B1 · Li · 2025 [cited by examiner]
US 12450363B2 · Ohayon · 2025 [cited by examiner]
US 20030007178A1 · Jeyachandran et al. · 2003 [cited by applicant]
US 20040098454A1 · Trapp et al. · 2004 [cited by applicant]
US 20050204348A1 · Horning et al. · 2005 [cited by applicant]
US 20060095918A1 · Hirose · 2006 [cited by applicant]
US 20070067848A1 · Gustave et al. · 2007 [cited by applicant]
US 20100275263A1 · Bennett et al. · 2010 [cited by applicant]
US 20100313189A1 · Beretta et al. · 2010 [cited by applicant]
US 20140137257A1 · Martinez et al. · 2014 [cited by applicant]
US 20140258998A1 · Adl-Tabatabai et al. · 2014 [cited by applicant]
US 20170061132A1 · Hovor et al. · 2017 [cited by applicant]
US 20170262164A1 · Jain et al. · 2017 [cited by applicant]
US 20170279826A1 · Mohanty et al. · 2017 [cited by applicant]
US 20170295197A1 · Parimi et al. · 2017 [cited by applicant]
US 20180020021A1 · Gilmore et al. · 2018 [cited by applicant]
US 20180239903A1 · Bodin et al. · 2018 [cited by applicant]
US 20180343114A1 · Ben-Ari · 2018 [cited by applicant]
US 20190188706A1 · Mccurtis · 2019 [cited by applicant]
US 20190236661A1 · Hogg et al. · 2019 [cited by applicant]
US 20190286816A1 · Fu · 2019 [cited by applicant]
US 20200012493A1 · Sagy · 2020 [cited by applicant]
US 20200074470A1 · Deshpande et al. · 2020 [cited by applicant]
US 20200153855A1 · Kirti et al. · 2020 [cited by applicant]
US 20200219009A1 · Dao et al. · 2020 [cited by applicant]
US 20200233979A1 · Tahmasebi Maraghoosh et al. · 2020 [cited by applicant]
US 20200259852A1 · Wolff et al. · 2020 [cited by applicant]
US 20200309767A1 · Loo et al. · 2020 [cited by applicant]
US 20200310866A1 · Varadaraj et al. · 2020 [cited by applicant]
US 20200314191A1 · Madhavan et al. · 2020 [cited by applicant]
US 20200349054A1 · Dai et al. · 2020 [cited by applicant]
US 20200380118A1 · Miller et al. · 2020 [cited by applicant]
US 20200387608A1 · Miller et al. · 2020 [cited by applicant]
US 20210049288A1 · Li · 2021 [cited by applicant]
US 20210089941A1 · Chen et al. · 2021 [cited by applicant]
US 20210133182A1 · Anderson et al. · 2021 [cited by applicant]
US 20210185094A1 · Waplington et al. · 2021 [cited by applicant]
US 20210211431A1 · Albero et al. · 2021 [cited by applicant]
US 20210256125A1 · Miller et al. · 2021 [cited by applicant]
US 20210264547A1 · Li · 2021 [cited by applicant]
US 20210273957A1 · Boyer et al. · 2021 [cited by applicant]
US 20210390465A1 · Werder et al. · 2021 [cited by applicant]
US 20220050928A1 · Shukla et al. · 2022 [cited by applicant]
US 20220078210A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220086162A1 · Badawy et al. · 2022 [cited by applicant]
US 20220092953A1 · Krishnan et al. · 2022 [cited by applicant]
US 20220100332A1 · Haworth et al. · 2022 [cited by applicant]
US 20220101326A1 · Kim et al. · 2022 [cited by applicant]
US 20220109685A1 · Hankins et al. · 2022 [cited by applicant]
US 20220114251A1 · Guim Bernat et al. · 2022 [cited by applicant]
US 20220114399A1 · Castiglione et al. · 2022 [cited by applicant]
US 20220124115A1 · Grabois et al. · 2022 [cited by applicant]
US 20220138327A1 · Panicker et al. · 2022 [cited by applicant]
US 20220147636A1 · Mahuli et al. · 2022 [cited by applicant]
US 20220147815A1 · Conwell et al. · 2022 [cited by applicant]
US 20220150275A1 · Mcnee et al. · 2022 [cited by applicant]
US 20220188460A1 · Hadar et al. · 2022 [cited by applicant]
US 20220191236A1 · Henderson · 2022 [cited by applicant]
US 20220198304A1 · Szczepanik et al. · 2022 [cited by applicant]
US 20220201042A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220210200A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220210202A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220224702A1 · Dherange et al. · 2022 [cited by applicant]
US 20220224723A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220263843A1 · Aslam et al. · 2022 [cited by applicant]
US 20220263855A1 · Engelberg et al. · 2022 [cited by applicant]
US 20220263860A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220278889A1 · Malleshaiah et al. · 2022 [cited by applicant]
US 20220286438A1 · Burke et al. · 2022 [cited by applicant]
US 20220286474A1 · Kuppa et al. · 2022 [cited by applicant]
US 20220294789A1 · Tikhomirov et al. · 2022 [cited by applicant]
US 20220294810A1 · Tyagi et al. · 2022 [cited by applicant]
US 20220303295A1 · Erlingsson et al. · 2022 [cited by applicant]
US 20220303300A1 · Egan · 2022 [cited by applicant]
US 20220303302A1 · Hwang et al. · 2022 [cited by applicant]
US 20220303352A1 · Herzog et al. · 2022 [cited by applicant]
US 20220311681A1 · Palladino et al. · 2022 [cited by applicant]
US 20220318654A1 · Lin et al. · 2022 [cited by applicant]
US 20220327620A1 · Ndoutoumou · 2022 [cited by applicant]
US 20220334818A1 · Mcfarland · 2022 [cited by applicant]
US 20220342846A1 · Kunchakarra et al. · 2022 [cited by applicant]
US 20220345457A1 · Jeffords et al. · 2022 [cited by applicant]
US 20220368728A1 · Murray et al. · 2022 [cited by applicant]
US 20220377093A1 · Crabtree et al. · 2022 [cited by applicant]
US 20220398149A1 · Mcfarland et al. · 2022 [cited by applicant]
US 20220400135A1 · Gamra · 2022 [cited by applicant]
US 20220405397A1 · Golan et al. · 2022 [cited by applicant]
US 20220414213A1 · Dixit · 2022 [cited by applicant]
US 20220417274A1 · Madanahalli et al. · 2022 [cited by applicant]
US 20230007039A1 · Waplington · 2023 [cited by applicant]
US 20230009999A1 · Higuchi et al. · 2023 [cited by applicant]
US 20230019072A1 · Okunlola · 2023 [cited by applicant]
US 20230032686A1 · Williams et al. · 2023 [cited by applicant]
US 20230033317A1 · Lin et al. · 2023 [cited by applicant]
US 20230035321A1 · Vijayaraghavan · 2023 [cited by applicant]
US 20230039855A1 · Greene · 2023 [cited by applicant]
US 20230052608A1 · Wattiau et al. · 2023 [cited by applicant]
US 20230067128A1 · Engelberg et al. · 2023 [cited by applicant]
US 20230071264A1 · Hakala et al. · 2023 [cited by applicant]
US 20230076372A1 · Engelberg et al. · 2023 [cited by applicant]
US 20230077527A1 · Sarkar · 2023 [cited by applicant]
US 20230113621A1 · Griffin et al. · 2023 [cited by applicant]
US 20230114719A1 · Thomas et al. · 2023 [cited by applicant]
US 20230117962A1 · Kaimal et al. · 2023 [cited by applicant]
US 20230118388A1 · Crabtree et al. · 2023 [cited by applicant]
US 20230123314A1 · Crabtree et al. · 2023 [cited by applicant]
US 20230132703A1 · Marsenic et al. · 2023 [cited by applicant]
US 20230135660A1 · Chapman et al. · 2023 [cited by applicant]
US 20230148116A1 · Stokes et al. · 2023 [cited by applicant]
US 20230164158A1 · Fellows et al. · 2023 [cited by applicant]
US 20230169397A1 · Smith et al. · 2023 [cited by applicant]
US 20230171282A1 · Bollinger · 2023 [cited by applicant]
US 20230177613A1 · Crabtree et al. · 2023 [cited by applicant]
US 20230205888A1 · Tyagi et al. · 2023 [cited by applicant]
US 20230205891A1 · Yellapragada et al. · 2023 [cited by applicant]
US 20230208858A1 · Mishra · 2023 [cited by examiner]
US 20230208869A1 · Bisht et al. · 2023 [cited by applicant]
US 20230208870A1 · Yellapragada et al. · 2023 [cited by applicant]
US 20230208871A1 · Yellapragada et al. · 2023 [cited by applicant]
US 20230229542A1 · Watkins et al. · 2023 [cited by applicant]
US 20230259860A1 · Sarkar · 2023 [cited by applicant]
US 20230261857A1 · Baldwin · 2023 [cited by examiner]
US 20230269272A1 · Dambrot et al. · 2023 [cited by applicant]
US 20230274003A1 · Liu et al. · 2023 [cited by applicant]
US 20230359789A1 · Andre et al. · 2023 [cited by applicant]
US 20230362200A1 · Crabtree et al. · 2023 [cited by applicant]
US 20230396435A1 · Baldwin · 2023 [cited by examiner]
US 20230396641A1 · Hebbagodi et al. · 2023 [cited by applicant]
US 20230409756A1 · Baldwin · 2023 [cited by examiner]
US 20230412635A1 · Binyamini et al. · 2023 [cited by applicant]
US 20240020538A1 · Socher et al. · 2024 [cited by applicant]
US 20240037245A1 · Kahan et al. · 2024 [cited by applicant]
US 20240054233A1 · Ohayon et al. · 2024 [cited by applicant]
US 20240054249A1 · Loubet Moundi et al. · 2024 [cited by applicant]
US 20240095077A1 · Singh et al. · 2024 [cited by applicant]
US 20240129345A1 · Kassam et al. · 2024 [cited by applicant]
US 20240202323A1 · Hoogerbrugge · 2024 [cited by examiner]
US 20240256678A1 · Thompson · 2024 [cited by applicant]
US 20240323202A1 · Niv et al. · 2024 [cited by applicant]
US 20240333737A1 · Shahrivar · 2024 [cited by examiner]
US 20240333743A1 · Bazalgette et al. · 2024 [cited by applicant]
US 20240333753A1 · Cross et al. · 2024 [cited by applicant]
US 20240333768A1 · Shahrivar · 2024 [cited by examiner]
US 20240340301A1 · Thompson · 2024 [cited by applicant]
US 20240340302A1 · Wang et al. · 2024 [cited by applicant]
US 20240346151A1 · Goswami et al. · 2024 [cited by applicant]
US 20240356984A1 · Gamra · 2024 [cited by applicant]
US 20240356986A1 · Crabtree et al. · 2024 [cited by applicant]
US 20240364725A1 · Sinha et al. · 2024 [cited by applicant]
US 20240364749A1 · Crabtree et al. · 2024 [cited by applicant]
US 20240403420A1 · Lal et al. · 2024 [cited by applicant]
US 20240403428A1 · Lal et al. · 2024 [cited by applicant]
US 20240403437A1 · Szigeti et al. · 2024 [cited by applicant]
US 20240403445A1 · Straub et al. · 2024 [cited by applicant]
US 20240406145A1 · Crabtree et al. · 2024 [cited by applicant]
US 20240411896A1 · Myers et al. · 2024 [cited by applicant]
US 20240414191A1 · Humphrey et al. · 2024 [cited by applicant]
US 20240414211A1 · Boyer et al. · 2024 [cited by applicant]
US 20250005167A1 · Millar et al. · 2025 [cited by applicant]
US 20250013753A1 · Conway · 2025 [cited by applicant]
US 20250021464A1 · Lang et al. · 2025 [cited by applicant]
US 20250045416A1 · Kacker · 2025 [cited by examiner]
US 20250045418A1 · Metcalf, II · 2025 [cited by examiner]
US 20250047717A1 · Crabtree · 2025 [cited by examiner]
US 20250053664A1 · Cameron · 2025 [cited by examiner]
US 20250055874A1 · Mukkamala · 2025 [cited by examiner]
US 20250063061A1 · Seri · 2025 [cited by examiner]
US 20250068743A1 · Rahman · 2025 [cited by examiner]
US 20250086290A1 · Uchil · 2025 [cited by examiner]
US 20250086305A1 · Rodniansky · 2025 [cited by examiner]
US 20250106242A1 · Galinkin · 2025 [cited by examiner]
US 20250217484A1 · Pachisia · 2025 [cited by examiner]
WO 2021160499A1 · 2021 [cited by applicant]
WO 2024020416A1 · 2024 [cited by applicant]
Cranium, Adopt & Accelerate AI Safely, retrieved on Nov. 7, 2024, from https:/cranium.ai/. [cited by applicant]
Coalition for Content Provenance and Authenticity, Contents Credentials C2PA Technological Specification, v2.1,Sep. 20, 2024. (Year: 2024). [cited by applicant]
Empower Your Team with a Compliance Co-Pilot, Sedric, retrieved on Sep. 25, 2024. https://www.sedric.ai/. [cited by applicant]
Farris, K., A., et al., “Vulcon: A System for Vulnerability Prioritization, Mitigation, and Management,” ACM Transactions on Privacy and Security, vol. 21, No. 4, Article 16. Publication date: Jun. 2018, 28 pages. [cited by applicant]
Generative machine learning models; IPCCOM000272835D, Aug. 17, 2023. (Year: 2023). [cited by applicant]
Guldimann, P., et al. “COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act,” arXiv:2410.07959v1 [cs.CL] Oct. 10, 2024, 38 pages. [cited by applicant]
Mathews, A. W., “What AI Can Do in Healthcare—and What It Should Never Do,” The Wall Street Journal, published on Aug. 21, 2024, retrieved on Sep. 5, 2024 https://www.wsj.com. [cited by applicant]
Nauta, M., et al., “From Anecdotal Evidence to Quantative Evaluation Methods: A Systematic Review of Evaluating Explainable AI” ACM Computing Surveys, vol. 55 No. 13s Article 295, 2023 [retrieved Jul. 3, 2024]. [cited by applicant]
Peers, M., “What California AI Bill Could Mean,” The Briefing, published and retrieved Aug. 30, 2024, 8 pages, https://www.theinformation.com/articles/what-california-ai-bill-could-mean. [cited by applicant]
Stokes, et al. “Preventing Machine Learning Poisoning Attacks Using Authentication and Provenance”, MILCOM 2021—2021 IEEE Military Communications Conference (MILCOM), 2021, pp. 181-188. (Year: 2021). [cited by applicant]
Zhou, Y., Liu, Y., Li, X., Jin, J., Qian, H., Liu, Z., Li, C., Dou, Z., Ho, T., & Yu, P. S. (2024). Trustworthiness in Retrieval-Augmented Generation Systems: A Survey. ArXiv. /abs/2409.10102. [cited by applicant]
AI Risk Management Framework Nist, retrieved on Jun. 17, 2024, https://www.nist.gov/itl/ai-risk-management-framework. [cited by applicant]
Independent analysis of AI language models and API providers. Artificial Analysis, retrieved on Jun. 13, 2024, https://artificialanalysis.ai/, 11 pages. [cited by applicant]
Brown, D., et al., “The Great AI Challenge: We Test Five Top Bots on Useful, Everyday Skills,” The Wall Street Journal, published May 25, 2024. [cited by applicant]
Dong, Y., et al., “Building Guardrails for Large Language Models,” https://ar5iv.labs.arxiv.org/html/2402.01822v1, published May 29, 2024, 20 pages. [cited by applicant]
International Search Report and Written Opinion received in Application No. PCT/US23/85942, dated Feb. 15, 2024, 6 pages. [cited by applicant]
Kojima, Takeshi, et al. “Large Language Models are Zero-Shot Reasoners,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), arXiv:2205.11916 [cs.CL], Jan. 29, 2023, 42 pages. [cited by applicant]
Mavrepis, P., et al., “XAI for All: Can Large Language Models Simplify Explainable AI?,” https://arxiv.org/abs/2401.13110, Jan. 23, 2024, 10 pages. [cited by applicant]
Wei, Jason, et al. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models,” 36th Conference on Neural Information Processing Systems (NeurIPS 2022), arXiv:2201.11903 [cs.CL], Jan. 10, 2023, 43 pages. [cited by applicant]
Zhao, H., et al., “Explainability for Large Language Models: A Survey,” https://arxiv.org/abs/2309.01029, Nov. 28, 2024, 38 pages. [cited by applicant]
Aggarwal, Nitin , “Why measuring your new AI is essential to its succes”, KPIs for gen AI: Why measuring your new AI is essential to its succes, 7 pages. [cited by applicant]
AI, “What is AI Verify?”, What is AI Verify—AI Verify Foundation. [cited by applicant]
Altman, Sam , “Sam Altman Admits That OpenAl Doesn't Actually Understand How Its AI Works”, Sam Altman Admits That OpenAI Doesn't Actually Understand How Its AI Works—“We certainly have not solved interpretability.”, 4 … [cited by applicant]
Anthrop/C , “Mapping the Mind of a Large Language Model”, Mapping the Mind of a Large Language Model, May 21, 2024. [cited by applicant]
Claburn, Thomas , “OpenAI's GPT-4 can exploit real vulnerabilities by reading security advisories”, OpenAI's GPT-4 can exploit real vulnerabilities by reading security advisories, Apr. 17, 2024, 3 pages. [cited by applicant]
Marshall, Andrew , “Threat Modeling AI/ML Systems and Dependencies”, Threat Modeling AI/ML Systems and Dependencies, Nov. 2, 2022, 27 pages. [cited by applicant]
Roose, Kevin , “A.I. Has a Measurement Problem”, A.I. Has a Measurement Problem, Apr. 15, 2024, 5 pages. [cited by applicant]
Roose, Kevin , “A.I.'s Black Boxes Just Got a Little Less Mysterious”, A.I.'s Black Boxes Just Got a Little Less Mysterious, May 21, 2024, 5 pages. [cited by applicant]
Shah, Harshay , “Decomposing and Editing Predictions by Modeling Model Computation”, Decomposing and Editing Predictions by Modeling Model Computation, 5 pages. [cited by applicant]
Shankar, Ram , “Failure Modes in Machine Learning”, , Nov. 2019, 14 pages. [cited by applicant]
Teo, Josephine , “Singapore launches Project Moonshot”, Singapore launches Project Moonshot—a generative Artificial Intelligence testing toolkit to address LLM safety and security challenges, May 31, 2024, 8 pages. [cited by applicant]