IP Library Granted Patent US 10,311,377
Granted Patent B2
US 10,311,377 · App. 15/448,537 · Granted Jun 4, 2019

Categorization of user interactions into predefined hierarchical categories

Inventors: Ravi Vijayaraghavan (Bangalore, IN); Vaibhav Srivastava (Bangalore, IN); R. Mathangi Sri (Bangalore, IN); Nitin Kumar Hardeniya (Bangalore, IN)
Assignee: [24]7.ai, Inc.
G06N20/00G06F16/353G06N5/02G06N5/025G06N5/04G06Q10/10G06N7/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,311,377
App. No.
15/448,537
Granted
Jun 4, 2019
Kind
B2
Abstract

User interactions are categorized into predefined hierarchical categories by classifying user interactions, such as queries, during a user interaction session by labeling text data into predefined hierarchical categories, and building a scoring model. The scoring model is then executed on untagged user interaction data to classify the user interactions into either action-based or information-based interactions.

Claims (59)

1. A computer-implemented method for categorization of user interactions into predefined hierarchical categories, comprising:

building a scoring model configured to convert a raw dataset into predefined hierarchical categories by extracting response variables and a set of structured input features from another raw dataset and identifying a functional relationship among the extracted response variables and features, the another raw dataset comprising text lines derived from a plurality of user interaction sessions;

executing, by the processor, the scoring model on the raw dataset to classify and label the raw dataset into predefined hierarchical categories comprising any of action-based and information-based interactions, the raw dataset comprising text lines derived from a contemporaneous user interaction session,

wherein the scoring model is based on response variables and a set of structured input features, the response variables being associated with another raw dataset comprising text lines derived from a plurality of user interaction sessions; and

based upon the predefined hierarchical categories, the processor aiding the contemporaneous user interaction session by automatically routing a user of the contemporaneous user interaction session to a channel associated with an appropriate department.

2. The method of claim 1 , wherein the text lines from the user interaction sessions are a contemporaneous transcription of an exchange between a live agent and one or more users.

3. The method of claim 1 , further comprising:

the processor providing a received unlabeled document to a relevant line extractor; and

the relevant line extractor extracting from the unlabeled document only those lines in the text of the unlabeled document that are relevant.

4. The method of claim 3 , further comprising:

inputting an output of the relevant line extractor and the scoring model to an information retrieval (IR) engine; and

responsive thereto, the IR engine outputting data matrices.

5. The method of claim 4 , further comprising;

preprocessing the text lines from the relevant line extractor by any of reshaping data; masking text patterns which comprise any of different date patterns, digit patterns, phone numbers, credit card numbers, company names, and URLs; converting to lower case; removing numbers; removing punctuation; tokenization; stemming; part-of-speech tagging; and lemmatization.

6. The method of claim 4 , further comprising:

the IR engine evaluating one or more rules to score every received unlabeled document against the scoring model; and

based on rules that hit each text data point, giving each such data point a score for each of the predefined hierarchical categories;

wherein a data matrix is produced for each level in a category tree.

7. The method of claim 6 , further comprising:

a classifier receiving the data matrix for each level in the category tree;

the classifier using at least one classification algorithm to label the data matrices,

wherein the labeled data matrices yield labeled documents that comprise cleansed, transformed, and structured representations of an original, transcribed or typed and unstructured text interaction history.

8. The method of claim 7 , further comprising:

combining the cleansed, transformed, and structured representations with additional structured data.

9. The method of claim 7 , further comprising:

building any of the classifier and the scoring model using a supervised or an unsupervised approach.

10. The method of claim 1 , wherein the processor classifies and labels the raw dataset during the user interaction sessions by labeling the raw data, including data that has not been transformed into a vector of one or more vectors, into the predefined hierarchical categories.

11. The method of claim 1 , further comprising:

building the scoring model in response to receiving labeled response variables and a set of structured input features;

validating the scoring model by determining whether an output of the scoring model agrees with a predicted response variable, given an input including the set of structured input features.

12. The method of claim 1 , further comprising:

using the scoring model across different businesses in a same or similar domain, even if the businesses are not part of a training data set.

13. The method of claim 1 , further comprising:

while executing the scoring model, the processor receiving the scoring model and the raw dataset; and

the processor classifying the raw dataset into predefined hierarchical categories.

14. The method of claim 13 , further comprising:

a relevant line extractor receiving raw data; and

the relevant line extractor outputting data to a model extraction module, the model extraction module using the scoring model and a rule extraction system to parse user interaction sessions and extract relevant or important features therefrom to build machine learning models based on tagged data;

wherein the features comprise any of most frequent n-grams; most discriminatory n-grams; rules comprising any of AND, OR, or proximity rules; any structured data comprising any of handle times and delays in response.

15. The method of claim 14 , further comprising:

the rule extraction system extracting different features that qualify text;

the rule extraction system checking distribution of the extracted features across different categories hierarchically; and

based on relevance of the categories, the rule extraction system providing scores.

16. The method of claim 13 , further comprising:

categorizing the user interactions into sub-categories of an information-based or action-based type of chat.

17. An apparatus for categorization of user interactions into predefined hierarchical categories, comprising:

a processor having memory, the processor programmed with machine instructions that classifies a raw dataset comprising text lines contemporaneously transcribed from a user interaction session, the machine instructions comprising:

building a scoring model configured to convert the raw dataset into predefined hierarchical categories by extracting response variables and a set of structured input features from another raw dataset and identifying a functional relationship among the extracted response variables and features, the another raw dataset comprising text lines derived from a plurality of user interaction sessions;

executing the scoring model on the raw dataset to classify and label the raw dataset into predefined hierarchical categories comprising any of action-based and information-based interactions; and

based upon the predefined hierarchical categories, aiding the user interaction session by automatically routing the users to a channel associated with an appropriate department.

18. The apparatus of claim 17 , further comprising:

building the scoring model in response to receiving labeled response variables and a set of structured input features; and

validating the scoring model by determining whether an output of the scoring model agrees with a predicted response variable, given an input including the set of structured input features.

19. A computer implemented method for query categorization into predefined hierarchical categories, comprising:

building a scoring model configured to convert a raw dataset into predefined hierarchical categories by extracting response variables and a set of structured input features from a first raw dataset and identifying a functional relationship among the extracted response variables and features, the first raw dataset comprising text lines derived from a plurality of user interaction sessions;

executing, by a processor, the scoring model on the raw dataset to classify and label the raw dataset into predefined hierarchical categories comprising any of action-based and information-based interactions, the raw dataset comprising text lines contemporaneously transcribed from a user interaction session wherein the scoring model is based on response variables and a set of structured input features, the response variables being associated with another raw dataset comprising text lines derived from a plurality of user interaction sessions; and

based upon the predefined hierarchical categories, aiding the user interaction session by automatically routing a user to an appropriate department for service by a live agent.

20. The computer implemented method of claim 19 , further comprising:

implementing, by the processor, a text filter for filtering sections of the text lines of the interaction session and a text parser for parsing the text lines and for extracting features from the interaction session.

Assignments (3)
CHANGE OF ADDRESS Recorded Jul 9, 2019
From: [24]7.AI, INC.
To: [24]7.AI, INC.
Reel/Frame 049707/0540 →
CHANGE OF NAME Recorded Nov 9, 2018
From: 24/7 CUSTOMER, INC.
To: [24]7.AI, INC.
Reel/Frame 047911/0877 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2017
From: VIJAYARAGHAVAN, RAVI; SRIVASTAVA, VAIBHAV; SRI, R. MATHANGI; HARDENIYA, NITIN KUMAR
To: 24/7 CUSTOMER, INC.
Reel/Frame 041447/0535 →
Continuity (3)
Continuation 14180967 · Feb 14, 2014
Provisional Application 61764962 · Feb 14, 2013
Related Publication 20170178033A1 · Jun 22, 2017
Cited By (4)
US 12,229,517 US 12,284,317 US 12,461,992 US 12,700,011