Authoring content for a conversational bot
A method of authorizing content for use, e.g., in association with a conversational bot. The method begins by configuring a conversational bot using a machine learning model trained to classify utterances into topics. Utterances that are not recognized by the machine learning model (e.g., according to some configurable threshold) are then identified. Using a clustering algorithm, one or more of the identified utterances are then processed into a grouping. Information identifying a topic associated with the grouping is then received and, in response, the machine learning model is updated to include the topic.
1. An automated method of authoring and using content in association with a conversational bot, comprising:
implementing a conversational bot using a machine learning model;
identifying utterances that are not recognized by the machine learning model;
using a clustering algorithm to group the identified utterances into a grouping;
displaying the grouping of the one or more-identified utterances, together with a suggested label, wherein the suggested label is a given one of the identified utterances;
responsive to the displaying, receiving an input identifying a topic associated with the grouping;
determining whether to update the machine learning model to include the topic, wherein a determination is based on an evaluation of a guardrail that ensures that the topic augments a content gap in the machine learning model;
based on the determination, the evaluation of the guardrail and the input, updating the machine learning model; and
the conversational bot using the updated machine learning model in a subsequent user interaction.
2. The method as described in claim 1 wherein the clustering algorithm is an affinity propagation algorithm that identifies a set of clusters.
3. The method as described in claim 1 wherein the cluster is a group of linguistically different, but semantically similar, ways of asking about a given topic.
4. The method as described in claim 2 further including evaluating the set of clusters using cross-validation, wherein cross-validation includes:
dividing training data into several folds; and
iteratively classifying the utterances in one fold after training the classifier on folds other than the one fold.
5. The method as described in claim 1 wherein utterances are not recognized with respect to a configurable confidence level.
6. The method as described in claim 1 , wherein the guardrail is one of:
a first diagnostic that determines whether the grouping has fewer than a configurable number of utterances, a second diagnostic that determines whether the grouping has a given overlap with an existing topic, a third diagnostic that determines whether one or more utterances associated with the grouping are similar to training utterances associated with a topic that is already represented in the machine learning model, and a fourth diagnostic that determines whether the grouping has a given number of training utterances to satisfy a criteria for classifying the topic.
7. The method as described in claim 1 , wherein the input is a selection of the suggested label.
8. The method as described in claim 1 , wherein the input is a label distinct from the suggested label.
9. The method as described in claim 1 , wherein the given one of the identified utterances has a minimal proximity to a centroid of a cluster as computed by the clustering algorithm.