System and method to represent conversational flows as graph embeddings and to conduct classification and clustering based on such embeddings
Systems and methods develop a natural language interface. Conversational data including user utterances is received for a plurality of conversations from a natural language interface. Each of the conversations is classified to determine intents for each user utterance, and for each of the conversations, a control flow diagram showing the intents and sequential flow of the conversation is generated. Each of the control flow diagrams is processed to generate a graph embedding representative of the conversation. A previous conversation that is similar to the current conversation is identified from a previous graph embedding that is nearest to a current graph embedding of a most recent utterance in a current conversation. A previous outcome of the previous conversation is used to predict an outcome of the current conversation, which, when not positive, may control response outputs of the natural language interface to steer the current conversation towards a positive result.
1. A method for ensuring k-anonymity in shared conversation datasets, comprising:
generating graph embeddings for each of a plurality of conversations from conversational data for N different users;
determining at least one cluster of the graph embeddings using a clustering algorithm;
determining number K of points in the at least one cluster;
sharing at least part of the conversational data corresponding to the at least one cluster when K is greater than or equal to N;
extracting at least one representative conversation corresponding to at least one graph embedding within the at least one cluster to form the at least part of the conversational data;
storing the at least one representative conversation in a cache for fast access; and
storing the conversational data in secondary storage having slower access than the cache.
2. The method of claim 1 , the at least one graph embedding corresponding to a centroid of the at least one cluster.
3. The method of claim 1 , the clustering algorithm implementing one or both of k-means and k-medioids.
4. A method for ensuring k-anonymity in shared conversation datasets, comprising:
generating graph embeddings for each of a plurality of conversations from conversational data for N different users;
determining at least one cluster of the graph embeddings using a clustering algorithm;
determining number K of points in the at least one cluster;
sharing at least part of the conversational data corresponding to the at least one cluster when K is greater than or equal to N;
determining results by filtering the conversational data based on at least one of a metadata dimension and an outcome variable; and
processing the results to generate filtered graph embeddings related to each of the at least one of a metadata dimension and an outcome variable.
5. The method of claim 4 , further comprising:
clustering the filtered graph embeddings using the clustering algorithm;
identifying clusters having fewer than a threshold value T of graph embeddings; and
indicating that the identified clusters require collection of more conversational data corresponding to the at least one of the metadata dimension and/or the outcome variable.
6. The method of claim 5 , further comprising generating an alert to indicate that the corresponding types of conversation are under-represented in conversational data.
7. The method of claim 1 , further comprising displaying, for each cluster, a histogram indicative of at least one attribute of the cluster.
8. The method of claim 7 , the attribute being age of a person having the conversation.
9. A method for efficient searching of conversations, comprising:
generating graph embeddings for each of a plurality of conversations from conversational data for different users;
determining at least one cluster of the graph embeddings using a clustering algorithm;
determining a representative conversation of the at least one cluster, wherein the representative conversation removes repetitive information;
storing the representative conversation in a cache; and
searching the cache to find the representative conversation based on input parameters.
10. A method for efficient searching of conversations, comprising:
generating graph embeddings for each of a plurality of conversations from conversational data for different users;
determining at least one cluster of the graph embeddings using a clustering algorithm;
determining a representative conversation of the at least one cluster;
storing the representative conversation in a cache; and
searching the cache to find the representative conversation based on input parameters and to allow representative searches to be performed rapidly.