Granted Patent B2

US 12,619,834 · App. 18/161,767 · Granted May 5, 2026

Inventors: Shubham Mehrotra (Santa Clara, CA); Zachary Alexander (Berkeley, CA); Shilpa Bhagavath (Mountain View, CA); Gurkirat Singh (Elk Grove, CA); Shashank Harinath (Mountain View, CA); Anuprit Kale (Oakland, CA)

Assignee: Salesforce, Inc.

G06F40/47G06F40/20G06F40/30G06F40/35G06F40/51G06N20/00

Abstract

Claims (75)

1 . A method of operating a cross-lingual intent classification framework in a conversation agent, the method comprising:

receiving, via a communication interface for the cross-lingual intent classification framework implemented on one or more processors, a training dataset of utterances annotated with intent labels corresponding to a plurality of pre-defined intent classes, respectively,

wherein the cross-lingual intent classification framework comprises a dual encoder architecture of a pretrained multi-lingual language model and a weighted K-nearest neighbor classifier as a decoder;

encoding, via a shared embedding neural network in the dual encoder architecture of the pretrained multi-lingual language model, a first set of utterances from the training dataset into a first set of embeddings in a feature space;

receiving, via a communication interface, a testing utterance;

encoding separately, via the shared embedding neural network in the dual encoder architecture of the pretrained multi-lingual language model, the testing utterance into a first embedding in the feature space;

computing a first set of distances between the first embedding and the first set of embeddings, respectively;

determining a set of nearest neighboring embeddings from the first set of embeddings, which are nearest neighbors to the first embedding based on the first set of distances;

removing, from the set of nearest neighboring embeddings, one or more neighboring embeddings that are out of range from the first embedding based on at least one inter-domain threshold associated with the plurality of pre-defined intent classes;

decoding, via the weighted K-nearest neighbor classifier, a first intent label for the testing utterance based on remaining neighboring embeddings in the set of nearest neighboring embeddings; and

generating, by a question answering model, a response to the testing utterance at least in part based on the first intent label.

2 . The method of claim 1 , wherein the pretrained language model is a multilingual language model, and wherein the training dataset comprises utterances and/or intent labels in multiple languages.

3 . The method of claim 1 , wherein each distance in the first set of distances is computed as a semantic similarity between the testing utterance and a training utterance corresponding to an embedding in the first set of embeddings.

4 . The method of claim 1 , further comprising:

encoding, via the pretrained language model, a training utterance into a second embedding in the feature space;

computing a second set of distances between the second embedding and the first set of embeddings, respectively;

determining, via the weighted K-nearest neighbor classifier, a second intent label for the training utterance;

computing a loss based on a difference between the second intent label and an annotated label associated with the training utterance; and

updating the weighted K-nearest neighbor classifier based on the loss while keeping the pretrained language model frozen.

5 . The method of claim 1 , further comprising:

computing, for each training utterance that belongs to a first intent class, a respective set of distances between the each training utterance and other training utterances in all other intent classes, respectively;

determining the at least one inter-domain threshold according to a percentage of all computed distances;

determining whether the set of nearest neighboring embeddings are within a range of the at least one inter-domain threshold to the first embedding; and

in response to determining that no neighboring embedding is under the inter-domain threshold to the first embedding, determining that the testing utterance is OOD.

6 . The method of claim 1 , wherein a value of K is determined by a minimum number of utterances that belong to a same intent class in the first set of utterances.

7 . A system of operating a cross-lingual intent classification framework in a conversation agent, the system comprising:

a communication interface receiving a training dataset of utterances annotated with intent labels corresponding to a plurality of pre-defined intent classes, respectively, respectively and a testing utterance;

a memory storing the cross-lingual intent classification framework comprising a dual encoder architecture of a pretrained multi-lingual language model, a weighted K-nearest neighbor classifier as a decoder, and a plurality of processor-executable instructions; and

one or more processors executing the plurality of processor-executable instructions to perform operations comprising:

encoding, via a shared embedding neural network in a dual encoder architecture of the pretrained multi-lingual language model, a first set of utterances from the training dataset into a first set of embeddings in a feature space;

receiving, via a communication interface, a testing utterance;

encoding separately, via the shared embedding neural network in the dual encoder architecture of the pretrained multi-lingual language model, the testing utterance into a first embedding in the feature space;

computing a first set of distances between the first embedding and the first set of embeddings, respectively;

determining a set of nearest neighboring embeddings from the first set of embeddings, which are nearest neighbors to the first embedding based on the first set of distances;

removing, from the set of nearest neighboring embeddings, one or more neighboring embeddings that are out of range from the first embedding based on at least one inter-domain threshold associated with the plurality of pre-defined intent classes;

decoding, via the weighted K-nearest neighbor classifier, a first intent label for the testing utterance based on remaining neighboring embeddings in the set of nearest neighboring embeddings; and

generating, by a question answering model, a response to the testing utterance at least in part based on the first intent label.

8 . The system of claim 7 , wherein the pretrained language model is a multilingual language model, and wherein the training dataset comprises utterances and/or intent labels in multiple languages.

9 . The system of claim 7 , wherein each distance in the first set of distances is computed as a semantic similarity between the testing utterance and a training utterance corresponding to an embedding in the first set of embeddings.

10 . The system of claim 7 , wherein the operations further comprise:

encoding, via the pretrained language model, a training utterance into a second embedding in the feature space;

computing a second set of distances between the second embedding and the first set of embeddings, respectively;

determining, via the weighted K-nearest neighbor classifier, a second intent label for the training utterance;

computing a loss based on a difference between the second intent label and an annotated label associated with the training utterance; and

updating the weighted K-nearest neighbor classifier based on the loss while keeping the pretrained language model frozen.

11 . The system of claim 7 , wherein the operations further comprise:

computing, for each training utterance that belongs to a first intent class, a respective set of distances between the each training utterance and other training utterances in all other intent classes, respectively;

determining the at least one inter-domain threshold according to a percentage of all computed distances;

determining whether the set of nearest neighboring embeddings are within a range of the at least one inter-domain threshold to the first embedding; and

in response to determining that no neighboring embedding is under the inter-domain threshold to the first embedding, determining that the testing utterance is OOD.

12 . The system of claim 7 , wherein a value of K is determined by a minimum number of utterances that belong to a same intent class in the first set of utterances.

13 . A non-transitory processor-readable storage medium storing processor-executable instructions for operating a cross-lingual intent classification framework in a conversation agent, the instructions being executed by one or more processors to perform operations comprising:

receiving, via a communication interface for the cross-lingual intent classification framework implemented on one or more processors, a training dataset of utterances annotated with intent labels corresponding to a plurality of pre-defined intent classes, respectively,

wherein the cross-lingual intent classification framework comprises a dual encoder architecture of a pretrained multi-lingual language model and a weighted K-nearest neighbor classifier as a decoder;

encoding, via a shared embedding neural network in a dual encoder architecture of the pretrained multi-lingual language model, a first set of utterances from the training dataset into a first set of embeddings in a feature space;

receiving, via a communication interface, a testing utterance;

encoding separately, via the shared embedding neural network in the dual encoder architecture of the pretrained multi-lingual language model, the testing utterance into a first embedding in the feature space;

computing a first set of distances between the first embedding and the first set of embeddings, respectively;

determining a set of nearest neighboring embeddings from the first set of embeddings, which are nearest neighbors to the first embedding based on the first set of distances;

removing, from the set of nearest neighboring embeddings, one or more neighboring embeddings that are out of range from the first embedding based on at least one inter-domain threshold associated with the plurality of pre-defined intent classes;

decoding, via the weighted K-nearest neighbor classifier, a first intent label for the testing utterance based on remaining neighboring embeddings in the set of nearest neighboring embeddings; and

generating, by a question answering model, a response to the testing utterance at least in part based on the first intent label.

14 . The medium of claim 13 , wherein the pretrained language model is a multilingual language model, and wherein the training dataset comprises utterances and/or intent labels in multiple languages.

15 . The medium of claim 13 , wherein each distance in the first set of distances is computed as a semantic similarity between the testing utterance and a training utterance corresponding to an embedding in the first set of embeddings.

16 . The medium of claim 13 , wherein the operations further comprise:

encoding, via the pretrained language model, a training utterance into a second embedding in the feature space;

computing a second set of distances between the second embedding and the first set of embeddings, respectively;

determining, via the weighted K-nearest neighbor classifier, a second intent label for the training utterance;

computing a loss based on a difference between the second intent label and an annotated label associated with the training utterance; and

updating the weighted K-nearest neighbor classifier based on the loss while keeping the pretrained language model frozen.

17 . The medium of claim 13 , wherein the operations further comprise:

computing, for each training utterance that belongs to a first intent class, a respective set of distances between the each training utterance and other training utterances in all other intent classes, respectively;

determining the at least one inter-domain threshold according to a percentage of all computed distances;

determining whether the set of nearest neighboring embeddings are within a range of the at least one inter-domain threshold to the first embedding; and

in response to determining that no neighboring embedding is under the inter-domain threshold to the first embedding, determining that the testing utterance is OOD.

Assignments (1)

Continuity (2)

References Cited (16)