Granted Patent B2

US 12,524,624 · App. 18/055,862 · Granted Jan 13, 2026

Inventors: Wei-Peng Chen (Fremont, CA); Mehdi Bahrami (San Jose, CA); Lei Liu (San Jose, CA)

Assignee: Fujitsu Limited

G06F40/40

Abstract

Claims (93)

1 . A method, executed by a processor, comprising:

receiving a set of texts associated with a domain from a first text corpus associated with the domain;

receiving a set of hypothesis statements associated with the domain;

applying a pre-trained natural language inference (NLI) model on each of the received set of texts and on each of the received set of hypothesis statements;

generating a second text corpus associated with the domain, based on the application of the pre-trained NLI model, the generated second text corpus corresponds to a set of labels associated with the domain; and

applying a few-shot learning model on the generated second text corpus to generate a third text corpus associated with the domain,

the generated third text corpus is configured to fine-tune the applied pre-trained NLI model, and

the fine-tuned NLI model is configured to label an input text associated with the domain, based on the received set of hypothesis statements;

controlling a display of the labelled input text on a display device;

selecting a first sentence from a set of sentences associated with the received set of texts, as a premise;

controlling an execution of a first set of operations to compute a final NLI score associated with each sentence of the set of sentences, the first set of operations includes:

 for each hypothesis statement from the set of hypothesis statements:

applying the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining an intermediate NLI score associated with the selected first sentence, based on the application of the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining whether all sentences in the set of sentences are processed for the computation of the final NLI score, and

selecting, as the first sentence, a second sentence from the set of sentences, based on a determination that at least one sentence in the set of sentences is unprocessed; and

computing the final NLI score associated with each sentence of the set of sentences to obtain an overall NLI score associated with the received set of texts, based on an iterative control of the execution of the first set of operations.

2 . The method according to claim 1 , wherein the set of hypothesis statements associated with the domain include at least one of a positive hypothesis statement, a neutral hypothesis statement, or a negative hypothesis statement.

3 . The method according to claim 1 , wherein the final NLI score associated with each sentence of the set of sentences corresponds to a weighted average of the intermediate NLI score associated with the first sentence, for each hypothesis statement from the set of hypothesis statements.

4 . The method according to claim 3 , further comprising:

applying a neural network model on each of the set of hypothesis statements to determine a set of weights associated with the weighted average of the intermediate NLI score, wherein

the final NLI score associated with each sentence of the set of sentences is determined based on the determined set of weights and the intermediate NLI score associated with the first sentence.

5 . The method according to claim 1 , further comprising

determining, based on the intermediate NLI score, an NLI prediction score of each sentence of the set of sentences, over each of a set of predefined NLI classes;

determining a maximum score for each of the set of predefined NLI classes, based on the determined NLI prediction score of each sentence;

determining a predicted class, based on the determined maximum score of each of the set of predefined NLI classes; and

determining a prediction label associated with the set of texts based on the determined predicted class, wherein

the obtained overall NLI score corresponds to the determined prediction label associated with the set of texts.

6 . The method according to claim 1 , further comprising:

applying a window function on the final NLI score associated with each sentence of the set of sentences;

computing, based on the application of the window function, an average score of a window of a first set of sentences from the set of sentences; and

obtaining the overall NLI score associated with the received set of texts, based on the computed average score of the window of the first set of sentences.

7 . The method according to claim 1 , further comprising:

determining, based on the intermediate NLI score, an NLI prediction score of each sentence of the set of sentences, over each of a set of predefined NLI classes;

comparing the determined NLI prediction score, of each sentence of the set of sentences, over each of the set of predefined NLI classes, with a first predefined threshold; and

obtaining the overall NLI score associated with the received set of texts, based on the comparison of the determined NLI prediction score with the first predefined threshold.

8 . The method according to claim 1 , further comprising:

determining a set of key sentences of the received set of texts, based on the final NLI score associated with each sentence of the set of sentences; and

controlling the display of the determined set of key sentences on the display device.

9 . The method according to claim 1 , further comprising:

identifying, from the set of sentences, a second set of sentences including a set of positive sentences and a set of negative sentences, based on the final NLI score associated with each sentence of the set of sentences;

comparing the final NLI score associated with each of the identified second set of sentences with a second predefined threshold;

determining a type of prediction associated with each of the identified second set of sentences, based on the comparison of the first NLI score, associated with each of the identified second set of sentences, with the second predefined threshold; and

determining a revised label associated with each of the second set of sentences, based on the determined type of prediction.

10 . The method according to claim 9 , wherein the determined type of prediction is one of a rule-based prediction, a conflicted prediction, or an uncertain prediction.

11 . The method according to claim 10 , wherein the determined revised label is further based on a user input and the determined type of prediction being the uncertain prediction.

12 . The method according to claim 1 , wherein the second text corpus associated with the domain is generated further based on:

retrieving a set of ground-truth positive and negative key sentences, based on the set of labels corresponding to the generated second text corpus;

retrieving a set of labelled key neutral sentences;

composing a set of contradicting and entailment pairs, based on the received set of hypothesis statements, and the retrieved set of ground-truth positive and negative key sentences;

composing a set of neutral pairs, based on the retrieved set of labelled key neutral sentences; and

balancing the composed set of contradicting and entailment pairs, and the composed set of neutral pairs, wherein

the generation of the third text corpus is further based on the balanced set of contradicting and entailment pairs, and the balanced set of neutral pairs.

13 . The method according to claim 1 , wherein the domain corresponds to a license text, a legal agreement text, or an end-user license agreement text associated with an application.

14 . The method according to claim 1 , wherein the pre-trained NLI model corresponds to an NLI model selected from a set of zero-shot NLI models.

15 . One or more non-transitory computer-readable storage media configured to store instructions that, in response to being executed, cause an electronic device to perform operations, the operations comprising:

receiving a set of texts associated with a domain from a first text corpus associated with the domain;

receiving a set of hypothesis statements associated with the domain;

applying a pre-trained natural language inference (NLI) model on each of the received set of texts and on each of the received set of hypothesis statements;

generating a second text corpus associated with the domain, based on the application of the pre-trained NLI model, the generated second text corpus corresponds to a set of labels associated with the domain; and

applying a few-shot learning model on the generated second text corpus to generate a third text corpus associated with the domain,

the generated third text corpus is configured to fine-tune the applied pre-trained NLI model, and

the fine-tuned NLI model is configured to label an input text associated with the domain, based on the received set of hypothesis statements;

controlling a display of the labelled input text on a display device;

selecting a first sentence from a set of sentences from the received set of texts, as a premise;

controlling an execution of a first set of operations to compute a final NLI score associated with each sentence of the set of sentences, the first set of operations includes:

for each hypothesis statement from the set of hypothesis statements:

applying the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining an intermediate NLI score associated with the selected first sentence, based on the application of the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining whether all sentences in the set of sentences are processed for the computation of the final NLI score, and

selecting, as the first sentence, a second sentence from the set of sentences, based on a determination that all sentences in the set of sentences are unprocessed; and

computing the final NLI score associated with each sentence of the set of sentences to obtain an overall NLI score associated with the received set of texts, based on an iterative control of the execution of the first set of operations.

16 . The one or more non-transitory computer-readable storage media according to claim 15 , wherein the set of hypothesis statements associated with the domain include at least one of a positive hypothesis statement, a neutral hypothesis statement, or a negative hypothesis statement.

17 . The one or more non-transitory computer-readable storage media according to claim 15 , wherein the final NLI score associated with each sentence of the set of sentences corresponds to a weighted average of the intermediate NLI score associated with the first sentence, for each hypothesis statement from the set of hypothesis statements.

18 . An electronic device, comprising:

a memory storing instructions; and

a processor, coupled to the memory, that executes the stored instructions to perform a process comprising:

receiving a set of texts associated with a domain from a first text corpus associated with the domain;

receiving a set of hypothesis statements associated with the domain;

applying a pre-trained natural language inference (NLI) model on each of the received set of texts and on each of the received set of hypothesis statements;

generating a second text corpus associated with the domain, based on the application of the pre-trained NLI model, the generated second text corpus corresponds to a set of labels associated with the domain; and

applying a few-shot learning model on the generated second text corpus to generate a third text corpus associated with the domain,

the generated third text corpus is configured to fine-tune the applied pre-trained NLI model, and

the fine-tuned NLI model is configured to label an input text associated with the domain, based on the received set of hypothesis statements; and

controlling a display of the labelled input text on a display device;

selecting a first sentence from a set of sentences from the received set of texts, as a premise;

controlling an execution of a first set of operations to compute a final NLI score associated with each sentence of the set of sentences, the first set of operations includes:

 for each hypothesis statement from the set of hypothesis statements:

applying the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining an intermediate NLI score associated with the selected first sentence, based on the application of the pre-trained NLI model on the selected first sentence and on the corresponding hypothesis statement, and

determining whether all sentences in the set of sentences are processed for the computation of the final NLI score, and

selecting, as the first sentence, a second sentence from the set of sentences, based on a determination that all sentences in the set of sentences are unprocessed; and

 computing the final NLI score associated with each sentence of the set of sentences to obtain an overall NLI score associated with the received set of texts, based on an iterative control of the execution of the first set of operations.

Assignments (1)

Continuity (1)

References Cited (21)

Cited By (1)

US 12,675,831