Granted Patent B2

US 12,619,872 · App. 17/808,242 · Granted May 5, 2026

Inventors: Helen Ngo (Toronto, CA); Nicholas Frosst (Toronto, CA)

Assignee: Cohere Inc.

G06N3/08G06F40/289G06F40/40

Abstract

Claims (53)

1 . A method of generating a trained natural language processing model having a lower propensity of generating hate speech, the method comprising:

obtaining an unfiltered dataset;

obtaining a set of trigger phrases;

determining, for each trigger phrase, a conditional likelihood of portions of the unfiltered dataset satisfying a threshold under a probability distribution for at least one trigger axis, wherein the conditional likelihood for each trigger phrase is determined based on a context provided by the respective portions of the unfiltered dataset;

filtering the unfiltered dataset to remove the one or more portions having at least one determined conditional likelihood that satisfies the threshold to generate a filtered dataset;

generating the trained natural language processing model using the filtered dataset;

providing the trained natural language processing model to an application; and

utilizing the trained natural language processing model in the application thereby reducing the conditional likelihood that the trained natural language processing model generates subject matter defined by the set of trigger phrases when generating a result from the utilizing.

2 . The method of claim 1 , further comprising obtaining a baseline model trained on the unfiltered dataset, wherein the probability distribution is represented by the baseline model and wherein the natural language processing model is generated by training the baseline model by finetuning the baseline model using the filtered dataset.

3 . The method of claim 1 , further comprising:

generating a text phrase with the trained natural language processing model, the trained natural language processing model generating the text phrase at least in part based on received input.

4 . The method of claim 1 , further comprising:

updating the set of trigger phrases;

filtering the filtered dataset based on the updated trigger phrases to generate a further filtered dataset; and

further training the trained natural language processing model with the further filtered dataset.

5 . The method of claim 1 , wherein one or more of the set of trigger phrases is appended to each portion of the unfiltered data.

6 . The method of claim 1 , wherein the set of trigger phrases are responsive to a first type of hate speech, and the threshold is responsive to a second type of hate speech.

7 . The method of claim 1 , wherein the threshold is based on a comparison of determined conditional likelihoods of entries of the dataset to properties of the unfiltered dataset.

8 . The method of claim 1 , wherein the at least one conditional likelihood is generated based on the relationship p(t|d) where t is a trigger phrase and d is an extract from the beginning of a document.

9 . The method of claim 1 , further comprising removing portions of the unfiltered dataset which include one or more blocked phrases, and the set of trigger terms do not include the one or more blocked phrases.

10 . The method of claim 1 , wherein the threshold includes a plurality of thresholds associated with different types of hate speech.

11 . A system for generating a trained natural language processing model having a lower propensity of generating hate speech, the system comprising:

a processor;

a memory in communication with the processor, the memory comprising computer executable instructions that when executed by the processor cause the processor to:

obtain an unfiltered dataset;

obtain a set of trigger phrases;

determine, for each trigger phrase, a conditional likelihood of portions of the unfiltered dataset satisfying a threshold under a probability distribution for at least one trigger axis, wherein the conditional likelihood for each trigger phrase is determined based on a context provided by the respective portions of the unfiltered dataset;

filter the unfiltered dataset to remove the one or more portions having at least one determined conditional likelihood that satisfies the threshold to generate a filtered dataset;

generate the trained natural language processing model using the filtered dataset;

provide the trained natural language processing model to an application; and

utilize the trained natural language processing model in the application thereby reducing the conditional likelihood that the trained natural language processing model generates subject matter defined by the set of trigger phrases when generating a result from the utilizing.

12 . The system of claim 11 , further comprising instructions to obtain a baseline model trained on the unfiltered dataset, wherein the probability distribution is represented by the baseline model and wherein the natural language processing model is generated by training the baseline model by finetuning the baseline model using the filtered dataset.

13 . The system of claim 11 , wherein the instructions cause the processor to:

generate a text phrase with the trained natural language processing model, the trained natural language processing model generating the text phrase at least in part based on received input.

14 . The system of claim 11 , wherein the instructions cause the processor to:

update the set of trigger phrases;

filter the filtered dataset based on the updated trigger phrases to generate a further filtered dataset; and

further train the trained natural language processing model with the further filtered dataset.

15 . The system of claim 11 , wherein one or more of the set of trigger phrases is appended to each portion of the unfiltered data.

16 . The system of claim 11 , wherein the set of trigger phrases are responsive to a first type of hate speech, and the threshold is responsive to a second type of hate speech.

17 . The system of claim 11 , wherein the threshold is based on a comparison of determined conditional likelihoods of entries of the dataset to properties of the unfiltered dataset.

18 . The system of claim 11 , wherein the instructions cause the processor to:

remove portions of the unfiltered dataset which include one or more blocked phrases, and

wherein the set of trigger terms do not include the one or more blocked phrases.

19 . The system of claim 11 , wherein the threshold includes a plurality of thresholds associated with different types of hate speech.

20 . A non-transitory computer readable medium for training a neural network model including a first plurality of nodes, the computer readable medium comprising computer executable instructions to:

obtain an unfiltered dataset;

obtain a set of trigger phrases;

determine, for each trigger phrase, a conditional likelihood of portions of the unfiltered dataset satisfying a threshold under a probability distribution for at least one trigger axis, wherein the conditional likelihood for each trigger phrase is determined based on a context provided by the respective portions of the unfiltered dataset;

filter the unfiltered dataset to remove the one or more portions having at least one determined conditional likelihood that satisfies the threshold to generate a filtered dataset;

generate the trained natural language processing model using the filtered dataset;

provide the trained natural language processing model to an application; and

utilize the trained natural language processing model in the application thereby reducing the conditional likelihood that the trained natural language processing model generates subject matter defined by the set of trigger phrases when generating a result from the utilizing.

Assignments (1)

Continuity (2)

References Cited (17)