COMPUTERIZED SYSTEM AND METHOD FOR MULTI-CLASS, MULTI-LABEL CLASSIFICATION OF ELECTRONIC MESSAGES
Disclosed are systems and methods for improving interactions with and between computers in content providing and/or hosting systems supported by or configured with devices, servers and/or platforms. The disclosed systems and methods provide a novel framework that automatically labels and classifies incoming emails. The disclosed framework embodies a novel computerized taxonomy configured as a multi-tier, multi-label classification system. The first tier involves an offline grid classifier that has higher accuracy, and the second tier is an online classifier that classifies emails in real-time. Thus, the framework provides a novel approach to classifying messages based on a multi-tiered analysis, which is utilized for generating user profiles, delivering the messages, and the like.
1 . A method comprising:
receiving, over a network, by a computing device, a message from a sender;
parsing, by the computing device, the message, and identifying message data;
analyzing, by the computing device, based on an aggregation strategy, the message data;
determining, by the computing device, based on the aggregation strategy analysis, whether the message data corresponds to an xcluster of messages;
when the determination indicates that the message data corresponds to a xcluster of messages,
adding said message to the xcluster;
applying a grid classifier to the xcluster of messages, said grid classifier application comprising determining and applying a multi-dimensional label; and
when the determination indicates that the message data does not correspond to a xcluster of messages,
further analyzing the message data;
determining a type of online classifier based on the further analysis of the message data;
applying the determined type of online classifier to the message, said online classifier application comprising determining and applying another multi-dimensional label.
2 . The method of claim 1 , wherein the type of online classifier comprises a logistic regression (LR) model.
3 . The method of claim 1 , wherein said type of online classifier comprises a Convolutional Neural Network (CNN) model, wherein said application of the online classifier is further based on information associated with the aggregation strategy.
4 . The method of claim 1 , wherein each of the multi-dimensional labels comprise information indicating at least one of a topic, type, objective, perceived action and method of sending.
5 . The method of claim 1 , further comprising:
generating, for at least a recipient of the message, a user profile based on the message data of the message and at least one of the determined labels.
6 . The method of claim 1 , wherein said message is delivered to an inbox based on at least one of the determined labels.
7 . The method of claim 1 , further comprising storing, in an associated database, information related to the determined labels.
8 . The method of claim 1 , wherein said aggregation strategy corresponds to a type attribute of a message used for creating an xcluster of messages.
9 . The method of claim 1 , wherein said grid classifier is applied offline, wherein said grid classifier executes a version of bidirectional encoder representations from transformations (BERT).
10 . The method of claim 9 , wherein said offline classifier is trained based on the grid classifier.
11 . The method of claim 1 , further comprising:
identifying a set of messages associated with a message platform;
identifying a set of unlabeled data associated with the message platform;
sampling the set messages based at least in part on the unlabeled data;
applying an active learning algorithm to the sampled messages; and
training the grid classifier based on the application of the active learning algorithm.
12 . The method of claim 1 , further comprising:
requesting, over the network, third party digital content based at least on one of the determined labels;
receiving, over the network, the third party digital content; and
communicating, over the network, the third party digital content to a recipient of the message along with the message.
13 . A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a processor associated with a computing device, performs a method comprising:
receiving, over a network, by the computing device, a message from a sender;
parsing, by the computing device, the message, and identifying message data;
analyzing, by the computing device, based on an aggregation strategy, the message data;
determining, by the computing device, based on the aggregation strategy analysis, whether the message data corresponds to an xcluster of messages;
when the determination indicates that the message data corresponds to a xcluster of messages,
adding said message to the xcluster;
applying a grid classifier to the xcluster of messages, said grid classifier application comprising determining and applying a multi-dimensional label; and
when the determination indicates that the message data does not correspond to a xcluster of messages,
further analyzing the message data;
determining a type of online classifier based on the further analysis of the message data;
applying the determined type of online classifier to the message, said online classifier application comprising determining and applying another multi-dimensional label.
14 . The non-transitory computer-readable storage medium of claim 13 , wherein the type of online classifier comprises a logistic regression (LR) model.
15 . The non-transitory computer-readable storage medium of claim 13 , wherein said type of online classifier comprises a Convolutional Neural Network (CNN) model, wherein said application of the online classifier is further based on information associated with the aggregation strategy.
16 . The non-transitory computer-readable storage medium of claim 13 , wherein each of the multi-dimensional labels comprise information indicating at least one of a topic, type, objective, perceived action and method of sending.
17 . The non-transitory computer-readable storage medium of claim 13 , wherein said grid classifier is applied offline, wherein said grid classifier executes a version of bidirectional encoder representations from transformations (BERT), wherein said offline classifier is trained based on the grid classifier.
18 . A computing device comprising:
a processor configured to:
receive, over a network, a message from a sender;
parse the message, and identify message data;
analyze, based on an aggregation strategy, the message data;
determine, based on the aggregation strategy analysis, whether the message data corresponds to an xcluster of messages;
when the determination indicates that the message data corresponds to a xcluster of messages,
add said message to the xcluster;
apply a grid classifier to the xcluster of messages, said grid classifier application comprising determining and applying a multi-dimensional label; and
when the determination indicates that the message data does not correspond to a xcluster of messages,
further analyze the message data;
determine a type of online classifier based on the further analysis of the message data;
apply the determined type of online classifier to the message, said online classifier application comprising determining and applying another multi-dimensional label.
19 . The computing device of claim 18 , wherein the type of online classifier comprises a logistic regression (LR) model.
20 . The computing device of claim 18 , wherein said type of online classifier comprises a Convolutional Neural Network (CNN) model, wherein said application of the online classifier is further based on information associated with the aggregation strategy.