Granted Patent B2

US 12,614,033 · App. 18/406,478 · Granted Apr 28, 2026

Inventors: Heike Adel-Vu (Renningen, DE); Jun Araki (San Jose, CA); Koustava Goswami (Galway, IE); Lukas Lange (Pforzheim, DE)

Assignee: ROBERT BOSCH GMBH

G06F40/284G06F16/319G06F40/40

Abstract

Claims (68)

1 . A computer-implemented method for operating a knowledge base including a language model, wherein the language model is pre-trained for a first domain and includes a tokenizer and a transformer, the method comprising the following steps:

providing an input sentence of a second domain;

tokenizing the input sentence with the tokenizer into tokens;

determining, by the transformer, respective embeddings for the individual tokens of the input sentence;

transforming the tokens with the transformer to a sentence embedding representing the input sentence as a whole;

identifying the input sentence as belonging to the second domain;

selecting, based on the identification, a set of keywords of the second domain;

providing a first set of embeddings that includes:

a prompt embedding of the first domain; and

the respective embeddings of the individual tokens of the input sentence;

providing a second set of embeddings that includes:

the first set of embeddings; and

a set of embeddings that represent the selected keywords of the second domain;

determining, based on the sentence embedding, a respective weight for the first set of embeddings and a respective weight for the second set of embeddings;

blending the first set of embeddings and the second set of embeddings according to the respective weights to determine a blended prompt; and

transforming, with the transformer, at least one of the tokens and the blended prompt to an output of the knowledge base.

2 . The method according to claim 1 , wherein the knowledge base includes a classifier, wherein the method further comprise:

determining a classification of the output with the classifier, and:

determining a label for at least one of the tokens depending on the classification, or

determining an entry of the knowledge base that corresponds to at least one of the tokens depending on the classification for the at least one of the tokens, or

determining a relation of at least two of the tokens depending on the classification.

3 . The method according to claim 1 , wherein the blending includes determining a sum, the weighted first set of embeddings and the weighted second set of embeddings being terms of the sum.

4 . The method according to claim 1 , wherein the determining of the first weight includes weighting the sentence embedding with a first weighting factor.

5 . The method according to claim 4 , further comprising:

initializing the first set of embeddings, and/or the set of embeddings that represent the selected keywords, and/or the first weighting factor randomly from a uniform distribution;

providing training data for training the first set of embeddings, the set of embeddings that represent the selected keywords and/or the first weighting factor; and

training the first set of embeddings, and/or the set of embeddings that represent the selected keywords and/or the first weighting factor while keeping the language model static.

6 . The method according to claim 1 , wherein the providing the second set of embeddings includes determining an order of the first set of embeddings and the set of embeddings that represent the selected keywords.

7 . The method according to claim 6 , wherein the determining of the order includes: (i) arranging the first set of embeddings before the set of embeddings that represent the selected keywords, and/or arranging the first set of embeddings after the set of embeddings that represent the selected keywords.

8 . The method according to claim 6 , wherein: (i) the determining of the second set of embeddings includes determining a weighted sum of the first set of embeddings arranged before the set of embeddings that represent the selected keywords and the first set of embeddings arranged after the set of embeddings that represent the selected keywords, or (ii) the determining of the order includes determining an ordering weight depending on the sentence embedding, determining if the ordering weight meets a condition, and either arranging the first set of embeddings before the set of embeddings that represent the selected keywords when the ordering weight meets the condition, or arranging the first set of embeddings after the set of embeddings that represent the selected keywords otherwise.

9 . The method according to claim 8 , wherein the determining of the ordering weight includes weighting the sentence embedding with a weighting factor.

10 . The method according to claim 1 , wherein the providing of the second set of embeddings includes determining the set of embeddings that represent the selected keywords depending on their respective term frequency in a document of the second domain and/or their respective inverse term frequency in a set of documents of the second domain and/or their respective term frequency in a document of the first domain.

11 . A device for operating a knowledge base including a language model, wherein the language model is pre-trained for a first domain and includes a tokenizer and a transformer, the device comprising:

at least one processor; and

at least one non-transitory storage;

wherein the at least one processor is configured to process instructions that, when executed by the at least one processor, cause the at least one processor to perform the following steps:

providing an input sentence of a second domain;

tokenizing the input sentence with the tokenizer into tokens;

determining, by the transformer, respective embeddings for the individual tokens of the input sentence;

transforming the tokens with the transformer to a sentence embedding representing the input sentence as a whole;

identifying the input sentence as belonging to the second domain;

selecting, based on the identification, a set of keywords of the second domain;

providing a first set of embeddings that includes:

a prompt embedding of the first domain; and

the respective embeddings of the individual tokens of the input sentence;

providing a second set of embeddings that includes:

the first set of embeddings; and

a set of embeddings that represent the selected keywords of the second domain;

determining, based on the sentence embedding, a respective weight for the first set of embeddings and a respective weight for the second set of embeddings;

blending the first set of embeddings and the second set of embeddings according to the respective weights to determine a blended prompt; and

transforming, with the transformer, at least one of the tokens and the blended prompt to an output of the knowledge base;

wherein the at least one storage is configured to store the instructions and the knowledge base.

12 . A non-transitory computer-readable medium on which is stored a computer program for operating a knowledge base including a language model that is pre-trained for a first domain, the computer program including computer readable instructions that, when executed by a computer, cause the computer to perform the following steps:

providing an input sentence of a second domain;

tokenizing the input sentence with a tokenizer into tokens;

determining, by a transformer, respective embeddings for the individual tokens of the input sentence;

transforming the tokens with the transformer to a sentence embedding representing the input sentence as a whole;

identifying the input sentence as belonging to the second domain;

selecting, based on the identification, a set of keywords of the second domain;

providing a first set of embeddings that includes:

a prompt embedding of the first domain; and

the respective embeddings of the individual tokens of the input sentence;

providing a second set of embeddings that includes:

the first set of embeddings; and

a set of embeddings that represent the selected keywords of the second domain;

determining, based on the sentence embedding, a respective weight for the first set of embeddings and a respective weight for the second set of embeddings;

blending the first set of embeddings and the second set of embeddings according to the respective weights to determine a blended prompt; and

transforming, with the transformer, at least one of the tokens and the blended prompt to an output of the knowledge base.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (16)