Using machine learning to extract information from electronic communications
Systems and methods for using machine learning to extract data from electronic communications are disclosed. According to certain aspects, a machine learning model is trained on a set of tasks using a set of training data. An electronic communication indicating a purchase of a product and/or service is processed to generate augmented text that is input into the machine learning model. After analyzing the augmented text, the machine learning model outputs a set of predicted values for a set of defined categories, which an entity may use for various purposes such as to apply digital rewards to user accounts.
1. A computer-implemented method of using machine learning to extract data from electronic communications, the computer-implemented method comprising:
initially training, by at least one processor using a masked language modeling task and a set of training data, a machine learning model, wherein the set of training data comprises a set of training electronic communications labeled with a set of training designated HyperText Markup Language (HTML) tags that define how content included in the set of training electronic communications should be displayed, wherein the content identifies or describes purchased products or services;
further training, by the at least one processor using a named entity recognition task and a task-specific training dataset, the machine learning model, wherein the task-specific training dataset identifies a set of predefined labels associated with the purchased products or services;
training, by the at least one processor using the set of training data, an entity linking model;
creating, using the entity linking model that was trained, a set of distinct groups associated with the purchased products or services;
accessing, by the at least one processor, an electronic communication indicating a purchase of a product or service by an individual;
parsing, by the at least one processor, the electronic communication to extract, from the electronic communication, a set of HTML tags that define how content included in the electronic communication should be displayed;
generating, by the at least one processor, a series of input tokens, including: identifying (i) a portion of the set of HTML tags to remove, and (ii) another
portion of the set of HTML tags to replace with a set of designated HTML tags, removing the portion of the set of HTML tags from the set of HTML tags, replacing the another portion of the set of HTML tags with the set of
designated HTML tags, and
consolidating consecutive HTML tags included in the set of designated HTML tags that replaced the another portion of the set of HTML tags, wherein (i) the removing and replacing enables at least some information to be preserved and enables the machine learning model to learn from different formats of different electronic communications, and (ii) the series of input tokens results from the removing, replacing, and consolidating;
analyzing, by the machine learning model that was trained, the series of input tokens to output a set of token-level predictions respectively corresponding to at least some of the series of input tokens, wherein each of the set of token-level predictions is in a labeled format;
for each token-level prediction in the set of token-level predictions, converting, by the at least one processor, that token-level prediction into a predicted value for a defined category, of a set of defined categories, associated with the purchase of the product or service by the individual; and
determining, by the at least one processor based on (i) the set of predicted values for the set of defined categories, and (ii) the set of distinct groups created using the entity linking model, a digital reward for the purchase of the product or service by the individual.
2. The computer-implemented method of claim 1 , wherein the machine learning model is a language model, and wherein initially training the machine learning model comprises:
initially training, by the at least one processor using the masked language modeling task and the set of training data, the language model using a transformer architecture.
3. The computer-implemented method of claim 1 , wherein each of the portion of the set of HTML tags that is removed is one of a set of predefined tag types.
4. The computer-implemented method of claim 1 , further comprising:
generating an indication of the digital reward for access by an electronic device associated with the individual.
5. The computer-implemented method of claim 4 , further comprising:
reviewing, by the at least one processor, multiple sets of predicted values for the set of defined categories to determine that duplicate electronic communications associated with the individual were analyzed by the machine learning model; and
preventing, by the at least one processor, duplicate digital rewards from being awarded to the individual.
6. The computer-implemented method of claim 1 , further comprising:
availing, by the at least one processor, the set of predicted values for the set of defined categories via an electronic device for review by a user;
receiving, by the at least one processor from the electronic device, a correction to an error included in the set of predicted values for the set of defined categories; and
retraining, by the at least one processor, the machine learning model with a subsequent set of data indicating the correction to the error.
7. A system for using machine learning to extract data from electronic communications, comprising:
a memory storing a set of computer-readable instructions and a machine learning model; and
one or more processors interfaced with the memory, and configured to execute the set of computer-readable instructions to cause the one or more processors to:
initially train, using a masked language modeling task and a set of training data, the machine learning model, wherein the set of training data comprises a set of training electronic communications labeled with a set of training designated HyperText Markup Language (HTML) tags that define how content included in the set of training electronic communications should be displayed, wherein the content identifies or describes purchased products or services,
further train, using a named entity recognition task and a task-specific training dataset, the machine learning model, wherein the task-specific training dataset identifies a set of predefined labels associated with the purchased products or services,
train, using the set of training data, an entity linking model,
create, using the entity linking model that was trained, a set of distinct groups associated with the purchased products or services,
access an electronic communication indicating a purchase of a product or service by an individual,
parse the electronic communication to extract, from the electronic communication, a set of HTML tags that define how content included in the electronic communication should be displayed,
generate a series of input tokens, including:
identify (i) a portion of the set of HTML tags to remove, and (ii) another portion of the set of HTML tags to replace with a set of designated HTML tags,
remove the portion of the set of HTML tags from the set of HTML tags,
replace the another portion of the set of HTML tags with the set of designated HTML tags, and
consolidate consecutive tags included in the set of designated HTML tags that replaced the another portion of the set of HTML tags, wherein (i) the removing and replacing enables at least some information to be preserved and enables the machine learning model to learn from different formats of different electronic communications, and (ii) the series of input tokens results from the removing, replacing, and consolidating,
analyze, by the machine learning model that was trained, the series of input tokens to output a set of token-level predictions respectively corresponding to at least some of the series of input tokens, wherein each of the set of token-level predictions is in a labeled format,
for each token-level prediction in the set of token-level predictions, convert that token-level prediction into a predicted value for a defined category, of a set of defined categories, associated with the purchase of the product or service by the individual, and
determine, based on (i) the set of predicted values for the set of defined categories, and (ii) the set of distinct groups created using the entity linking model, a digital reward for the purchase of the product or service by the individual.
8. The system of claim 7 , wherein each of the portion of the set of HTML tags that is removed is one of a set of predefined tag types.
9. The system of claim 7 , wherein the one or more processors is configured to execute the set of computer-readable instructions to further cause the one or more processors to:
generate an indication of the digital reward for access by an electronic device associated with the individual.
10. A non-transitory computer-readable storage medium configured to store instructions executable by one or more processors, the instructions comprising:
instructions for initially training, using a masked language modeling task and a set of training data, a machine learning model, wherein the set of training data comprises a set of training electronic communications labeled with a set of training designated Hyper Text Markup Language (HTML) tags that define how content included in the set of training electronic communications should be displayed, wherein the content identifies or describes purchased products or services;
instructions for further training, using a named entity recognition task and a task-specific training dataset, the machine learning model, wherein the task-specific training dataset identifies a set of predefined labels associated with the purchased products or services:
instructions for training, using the set of training data, an entity linking model: instructions
for creating, using the entity linking model that was trained, a set of distinct groups associated with the purchased products or services:
instructions for accessing an electronic communication indicating a purchase of a product or service by an individual;
instructions for parsing the electronic communication to extract, from the electronic communication, a set of HTML tags that define how content included in the electronic communication should be displayed;
instructions for generating a series of input tokens, including:
instructions for identifying (i) a portion of the set of HTML tags to remove, and (ii) another portion of the set of HTML tags to replace with a set of designated HTML tags,
instructions for removing the portion of the set of HTML tags from the set of HTML tags,
instructions for replacing the another portion of the set of HTML tags with the set of designated HTML tags, and
instructions for consolidating consecutive tags included in the set of designated HTML tags that replaced the another portion of the set of HTML tags wherein (i) the removing and replacing enables at least some visually rich information to be preserved and enables the machine learning model to learn from different formats of different electronic communications, and (ii) the series of input tokens results from the removing, replacing, and consolidating;
instructions for analyzing, by the machine learning model that was trained, the series of input tokens to output a set of token-level predictions respectively corresponding to at least some of the series of input tokens, wherein each of the set of token-level predictions is in a labeled format;
instructions for, for each token-level prediction in the set of token-level predictions, converting that token-level prediction into a predicted value for a defined category, of a set of defined categories, associated with the purchase of the product or service by the individual and
instructions for determining, based on (i) the set of predicted values for the set of defined categories, and (ii) the set of distinct groups created using the entity linking model, a digital reward for the purchase of the product or service by the individual.