Methods and systems for task-agnostic pretrained language model watermarking via high entropy backdooring
Methods, devices, and processor-readable media for processing an input prompt using a watermarked language model, the processing including: when the input prompt to the watermarked language model does not include a predefined key, providing a model output that conforms to an intended task that the model has been trained to perform; and when the input prompt to the watermarked language model includes the predefined key, providing a stochastic model output that does not conform to the intended task.
1 . A method comprising:
processing an input prompt using a watermarked language model, the processing comprising:
when the input prompt to the watermarked language model does not include a predefined key, providing a model output that conforms to an intended task that the watermarked language model is configured to perform for input prompts that do not include the predefined key; and
when the input prompt to the watermarked language model includes the predefined key, providing a stochastic model output that does not conform to the intended task,
wherein the stochastic model output is a high entropy output.
2 . The method of claim 1 , wherein the watermarked language model comprises a plurality of transformer block layers, comprising:
a first group of transformer block layers collectively trained to cause the language model to perform the intended task; and
a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers trained to cause the watermarked language model to provide high entropy output when the input prompt to the watermarked language model includes the predefined key and to cause the watermarked language model to perform the intended task when the input prompt to the watermarked language model does not include the predefined key,
wherein providing the stochastic model output comprises processing the predefined key using the second group of one or more watermark transformer block layers to generate high entropy output causing the language model to generate the stochastic model output.
3 . The method of claim 1 , further comprising:
obtaining the watermarked language model, comprising:
receiving model code and pretrained model weights that collectively define a pretrained language model (PLM) that comprises a first group of transformer block layers collectively configured by the pretrained model weights to cause the PLM to perform the intended task;
amending the model code to define an augmented language model that comprises a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers being collectively configured by watermark layer weights; and
training the augmented language model by freezing the pretrained model weights and iteratively updating the watermark layer weights with training objectives of (i) causing the augmented language model to provide high entropy output when an input to the augmented language model includes the predefined key and (ii) causing the augmented language model to perform the intended task when the input to the augmented language model does not include the predefined key,
the watermarked language model being collectively defined by the amended model code, the pretrained model weights and the watermark layer weights obtained by training the augmented language model.
4 . The method of claim 3 , wherein the training objective of causing the augmented language model to perform the intended task when the input to the augmented language model does not include the predefined key includes iteratively updating the watermark layer weights with an objective of causing layers of the second group of one or more watermark transformer block layers to function as respective identity functions when processing input that does not include the predefined key.
5 . The method of claim 3 , wherein the training objective of causing the augmented language model to provide high entropy output when the input to the augmented language model includes the predefined key includes iteratively updating the watermark layer weights with an objective of causing layers of the second group of one or more watermark transformer block layers to respectively generate outputs having a probability distribution for which all discrete outcomes are equally likely.
6 . The method of claim 3 , further comprising:
obtaining a further plurality of watermarked language models, comprising, for each respective watermarked language model of the further plurality of watermarked language models:
training a further augmented language model, the further augmented language model also comprising a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers being collectively configured by further watermark layer weights, the training of the further augmented language model comprising freezing the pretrained model weights and iteratively updating the further watermark layer weights with training objectives of (i) causing the further augmented language model to provide high entropy output when an input to the further augmented language model includes a unique predefined key unique to the respective watermarked language model and (ii) causing the further augmented language model to perform the intended task when the input to the further augmented language model does not include the unique predefined key; and
the respective watermarked language model being collectively defined by the amended model code, the pretrained model weights and the further watermark layer weights obtained by training the further augmented language model.
7 . The method of claim 3 , further comprising:
finetuning the watermarked language model using a finetuning dataset that does not include the predefined key.
8 . The method of claim 1 , further comprising assessing a language model to determine if it is derived from the watermarked language model, comprising:
providing a plurality of input prompts to the language model that do not include the predefined key and obtaining a respective first distribution of outputs from the language model;
providing a plurality of input prompts to the language model that include the predefined key and obtaining a respective second distribution of outputs from the language model; and
determining, based on a comparison of the first distribution and the second distribution that the language model is derived from the watermarked language model when a difference between the first distribution and the second distribution exceeds a threshold.
9 . The method of claim 1 wherein the stochastic model output is a maximum entropy output.
10 . The method of claim 1 wherein the watermarked language model comprises a plurality of transformer block layers that are collectively configured to provide high entropy output when the input prompt to the watermarked language model includes the predefined key and to cause the watermarked language model to perform the intended task when the input prompt to the watermarked language model does not include the predefined key.
11 . A method of obtaining a watermarked language model, comprising:
receiving model code and pretrained model weights that collectively define a pretrained language model (PLM) that comprises a first group of transformer block layers collectively configured by the pretrained model weights to cause the PLM to perform an intended task;
amending the model code to define an augmented language model that comprises a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers being collectively configured by watermark layer weights; and
training the augmented language model by freezing the pretrained model weights and iteratively updating the watermark layer weights with training objectives of (i) causing the augmented language model to provide high entropy output when an input to the augmented language model includes a predefined key and (ii) causing the augmented language model to perform the intended task when the input to the augmented language model does not include the predefined key,
the watermarked language model being collectively defined by the amended model code, the pretrained model weights and the watermark layer weights obtained by training the augmented language model.
12 . The method of claim 11 , further comprising receiving the model code and pretrained model weights from a requesting computer system through a communications network and sending the amended model code and the watermark layer weights to the requesting computer system through the communications network.
13 . The method of claim 11 , further comprising receiving a pretraining dataset used to pretrain the PLM, wherein training the augmented language model is performed using a watermark training dataset that includes the predefined key and the pretraining dataset.
14 . A system comprising:
one or more processors; and
one or more memories storing machine-executable instructions thereon which, when executed by the one or more processors, cause the system to perform a method comprising:
processing an input prompt using a watermarked language model, the processing comprising:
when the input prompt to the watermarked language model does not include a predefined key, providing a model output that conforms to an intended task that the watermarked language model is configured to perform for input prompts that do not include the predefined key; and
when the input prompt to the watermarked language model includes the predefined key, providing a stochastic model output that does not conform to the intended task,
wherein the stochastic model output is a high entropy output.
15 . The system of claim 14 , wherein the watermarked language model comprises a plurality of transformer block layers, comprising:
a first group of transformer block layers collectively trained to cause the language model to perform the intended task; and
a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers trained to cause the watermarked language model to provide high entropy output when the input prompt to the watermarked language model includes the predefined key and to cause the watermarked language model to perform the intended task when the input prompt to the watermarked language model does not include the predefined key,
wherein providing the stochastic model output comprises processing the includes the predefined key using the second group of one or more watermark transformer block layers to generate high entropy output causing the language model to generate the stochastic model output.
16 . The system of claim 14 , wherein the method further comprises:
obtaining the watermarked language model, comprising:
receiving model code and pretrained model weights that collectively define a pretrained language model (PLM) that comprises a first group of transformer block layers collectively configured by the pretrained model weights to cause the PLM to perform the intended task;
amending the model code to define an augmented language model that comprises a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers being collectively configured by watermark layer weights; and
training the augmented language model by freezing the pretrained model weights and iteratively updating the watermark layer weights with training objectives of (i) causing the augmented language model to provide high entropy output when an input to the augmented language model includes the predefined key and (ii) causing the augmented language model to perform the intended task when the input to the augmented language model does not include the predefined key,
the watermarked language model being collectively defined by the amended model code, the pretrained model weights and the watermark layer weights obtained by training the augmented language model.
17 . The system of claim 16 , wherein the training objective of causing the augmented language model to perform the intended task when the input to the augmented language model does not include the predefined key includes iteratively updating the watermark layer weights with an objective of causing layers of the second group of one or more watermark transformer block layers to function as respective identity functions when processing input that does not include the predefined key.
18 . The system of claim 16 , wherein the training objective of causing the augmented language model to provide high entropy output when the input to the augmented language model includes the predefined key includes iteratively updating the watermark layer weights with an objective of causing layers of the second group of one or more watermark transformer block layers to respectively generate outputs having a probability distribution for which all discrete outcomes are equally likely.
19 . The system of claim 16 , wherein the method further comprises:
obtaining a further plurality of watermarked language models, comprising, for each respective watermarked language model of the further plurality of watermarked language models:
training a further augmented language model, the further augmented language model also comprising a second group of one or more watermark transformer block layers interspersed within the transformer block layers of the first group, the second group of one or more watermark transformer block layers being collectively configured by further watermark layer weights, the training of the further augmented language model comprising freezing the pretrained model weights and iteratively updating the further watermark layer weights with training objectives of (i) causing the further augmented language model to provide high entropy output when an input to the further augmented language model includes a unique predefined key unique to the respective watermarked language model and (ii) causing the further augmented language model to perform the intended task when the input to the further augmented language model does not include the unique predefined key; and
the respective watermarked language model being collectively defined by the amended model code, the pretrained model weights and the further watermark layer weights obtained by training the further augmented language model.
20 . The system of claim 16 , wherein the method further comprises receiving the model code and pretrained model weights from a requesting computer system through a communications network and sending the amended model code and the watermark layer weights to the requesting computer system through the communications network.
21 . The system of claim 16 , wherein the method further comprises receiving a pretraining dataset used to pretrain the PLM, wherein training the augmented language model is performed using a watermark training dataset that includes the predefined key and the pretraining dataset.
22 . The system of claim 14 , wherein the method further comprises:
assessing a language model to determine if it is derived from the watermarked language model, comprising:
providing a plurality of input prompts to the language model that do not include the predefined key and obtaining a respective first distribution of outputs from the language model;
providing a plurality of input prompts to the language model that include the predefined key and obtaining a respective second distribution of outputs from the language model; and
determining, based on a comparison of the first distribution and the second distribution that the language model is derived from the watermarked language model when a difference between the first distribution and the second distribution exceeds a threshold.
23 . The system of claim 14 wherein the stochastic model output is a maximum entropy output.
24 . The system of claim 14 wherein the watermarked language model comprises a plurality of transformer block layers that are collectively configured to provide high entropy output when the input prompt to the watermarked language model includes the predefined key and to cause the watermarked language model to perform the intended task when the input prompt to the watermarked language model does not include the predefined key.