Large language model (LLM) supply chain security

1 . A system, comprising:

at least one computing device comprising at least one processor and at least one memory; and

machine-readable instructions stored in the at least one memory that, when executed by the at least one processor, cause the at least one computing device to at least:

identify a large language model (LLM) application that is programmed to communicate with an LLM service that provides access to an LLM;

attach, to an image or package comprising the LLM application, an LLM-extended software bill of materials that is extended to provide LLM specific supply chain information for the LLM application;

execute an automated LLM security test of the LLM application;

attach, to the LLM-extended software bill of materials, a signed LLM security test attestation based at least in part on completion of the automated LLM security test; and

transmit, to a predetermined network endpoint, at least one of: the LLM application, the LLM-extended software bill of materials, or any combination thereof.

2 . The system of claim 1 , wherein the LLM-extended software bill of materials is published or transmitted to the predetermined network endpoint based at least in part on a result of a rules-based analysis of a plurality of LLM specific attestations in the LLM-extended software bill of materials.

3 . The system of claim 1 , wherein the machine-readable instructions, when executed by the at least one processor, further cause the at least one computing device to at least:

retrieve an LLM of the LLM service; and

execute at least one test of the LLM, wherein an additional signed LLM security test attestation is attached to the LLM-extended software bill of materials based at least in part on completion of the test of the LLM.

4 . The system of claim 1 , wherein the machine-readable instructions, when executed by the at least one processor, further cause the at least one computing device to at least:

transmit the LLM application to a security developer environment for a manual LLM security test, wherein an additional signed LLM security test attestation is attached to the LLM-extended software bill of materials based at least in part on completion of the manual LLM security test.

5 . The system of claim 1 , wherein the automated LLM security test is executed as one of a plurality of automated LLM security tests specified in an LLM security library, and a plurality of LLM security test attestations are attached to the LLM-extended software bill of materials, a respective one of the LLM security test attestations corresponding to the plurality of automated LLM security tests.

6 . The system of claim 5 , wherein the plurality of automated LLM security tests specified in the LLM security library comprise at least one of: a harmful content test, a bias mitigation test, a secure data leakage test, a prompt injection test, an LLM hallucination test, or any combination thereof.

7 . The system of claim 1 , wherein the LLM-extended software bill of materials is attached to the image or the package in response to storing the LLM application in a particular data repository.

8 . A method, comprising:

identifying a large language model (LLM) application that is programmed to communicate with an LLM service that provides access to an LLM;

attaching, to an image or package comprising the LLM application, an LLM-extended software bill of materials that is extended to provide LLM specific supply chain information for the LLM application;

executing an automated LLM security test of the LLM application;

attaching, to the LLM-extended software bill of materials, a signed LLM security test attestation based at least in part on completion of the automated LLM security test; and

transmitting, to a predetermined network endpoint, at least one of: the LLM application, the LLM-extended software bill of materials, or any combination thereof.

9 . The method of claim 8 , wherein the LLM application is identified based at least in part on generating a branch repository to store a particular version or build of the LLM application.

10 . The method of claim 8 , wherein the automated LLM security test is performed based at least in part on validation of an LLM pipeline attestation from the LLM-extended software bill of materials, wherein the LLM pipeline attestation indicates a position in an LLM specific software development pipeline.

11 . The method of claim 8 , further comprising:

transmitting the LLM application to a security developer environment, wherein a manual test of the LLM application is performed in the security developer environment.

12 . The method of claim 11 , further comprising:

receiving, from the security developer environment, the LLM application, and the LLM-extended software bill of materials comprising an additional LLM security test attestation indicating a result of the manual test.

13 . The method of claim 11 , further comprising:

receiving, from the security developer environment, a result of the manual test of the LLM application; and

attaching, to the LLM-extended software bill of materials of the LLM application, an additional LLM security test attestation indicating a result of the manual test.

14 . The method of claim 8 , further comprising:

provisioning a networked computing environment to deploy and execute the LLM application.

15 . A system, comprising:

at least one computing device comprising at least one processor and at least one memory; and

machine-readable instructions stored in the at least one memory that, when executed by the at least one processor, cause the at least one computing device to at least:

attach, to an image or package comprising an LLM application that is programmed to communicate with an LLM service, an LLM-extended software bill of materials that is extended to provide LLM specific supply chain information for the LLM application;

execute an automated LLM security test of the LLM application;

attach, to the LLM-extended software bill of materials, a signed LLM security test attestation based at least in part on completion of the automated LLM security test; and

publish or transmit, to a predetermined network endpoint, at least one of: the LLM application, the LLM-extended software bill of materials, or any combination thereof.

16 . The system of claim 15 , wherein the LLM-extended software bill of materials is attached to the image or the package based at least in part on storing the LLM application in a branch repository.

17 . The system of claim 15 , wherein the LLM specific supply chain information includes an LLM pipeline attestation that indicates a position in an LLM specific software development pipeline.

18 . The system of claim 15 , wherein the machine-readable instructions, when executed by the at least one processor, further cause the at least one computing device to at least:

transmit the LLM application to a security developer environment for a manual test.

19 . The system of claim 18 , wherein the machine-readable instructions, when executed by the at least one processor, further cause the at least one computing device to at least:

receive, from the security developer environment, the LLM application, and the LLM-extended software bill of materials comprising an additional LLM security test attestation indicating a result of the manual test.

20 . The system of claim 18 , wherein the machine-readable instructions, when executed by the at least one processor, further cause the at least one computing device to at least:

receive, from the security developer environment, a result of the manual test of the LLM application; and

attach, to the LLM-extended software bill of materials of the LLM application, an additional LLM security test attestation indicating a result of the manual test.