Retargeting telecommunication service provider website users for content views using machine learning systems and methods
Systems and methods for retargeting users of a telecommunication service provider's website. The retargeting system prioritizes users for retargeting by assessing likelihood that the user will convert to a customer based on session-level navigation flows, user historic session patterns, and consumer data. This information is used to create a training dataset that indicates whether the information is associated with converting or non-converting users. The training dataset is used to train a machine learning model to generate conversion scores indicating a likelihood that a user will convert to a customer and retargeting recommendations. The conversion scores and recommendations can be used prioritize users for retargeting via third-party services, such as websites, social media platforms, or messaging services. Using training data, accuracy of the machine learning model can be assessed. When accuracy of the machine learning model does not exceed a threshold accuracy, the system can be retrained.
1 . A computer-implemented method to retarget users of a telecommunication service provider website, the method comprising:
receiving session data corresponding to multiple sessions associated with the users visiting one or more websites of a telecommunication service provider, each session of the multiple sessions defining a user accessing and interacting with a set of multiple webpages of the one or more websites of the telecommunication service provider, and the received session data specifying, for each session of the multiple sessions: (1) a user identifier, (2) a session identifier, (3) a duration of time spent by the user on each webpage in the set of multiple webpages, (4) a set of user actions detected of the user interacting with the multiple webpages, (5) an indication of whether the user converted to a subscriber of the telecommunication service provider during or after a corresponding session, and (6) at least one content associated with the telecommunication service provider that is displayed to the user before the user converted;
generating session flows for each of the multiple sessions corresponding to the session data, each session flow of the session flows including stored information indicating a sequence of webpages in the set of multiple webpages that the user accessed while interacting with the set of multiple webpages, and the stored information for each webpage in the sequence of webpages indicating the duration of time and the set of user actions;
generating a historic session pattern associated with each user identifier specified in the received session data;
receiving consumer information associated with each user identifier specified in the received session data, the received consumer information comprising a locked status indicating whether a device associated with the user identifier specified in the received session data is restricted to a specific provider of telecommunications services;
creating a training dataset that includes: (1) each session flow of the generated session flows, (2) each generated historic session pattern associated with each user identifier specified in the received session data, (3) the received consumer information associated with each user identifier specified in the received session data, and (4) information designating each session flow of the session flows as being associated with either a converting user or a non-converting user;
training, using the training dataset, a machine learning model to use a combined input including: (1) an input session flow of a non-subscriber user interacting with a plurality of webpages of the one or more websites of the telecommunication service provider, (2) an input historic session pattern associated with the non-subscriber user, and (3) an input consumer information associated with the non-subscriber user, the input consumer information indicating that the non-subscriber user is a customer of another service provider differing from the telecommunication service provider;
generating, by the trained machine learning model processing the combined input, a conversion score indicating a likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider based on the input session flow, the input historic session pattern, and the input consumer information associated with the non-subscriber user;
retargeting the non-subscriber user via a first communication channel of a third-party service accessed by the non-subscriber user as a result of the conversion score exceeding a first threshold score, the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at a user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the first communication channel, the third-party service being different from services provided by the telecommunication service provider;
retargeting the non-subscriber user via a second communication channel of the third-party service accessed by the non-subscriber user when the conversion score exceeds a second threshold score different from the first threshold score, the second communication channel being different and separate from the first communication channel, and the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at the user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the second communication channel of the third-party service;
evaluating the trained machine learning model using a testing dataset, the testing dataset including session flows, historic session patterns, and consumer information of known converting and non-converting users;
assessing accuracy of the trained machine learning model based on the evaluating and the retargeting of the non-subscriber user via the first communication channel or the second communication channel; and
when the accuracy of the trained machine learning model does not exceed a threshold accuracy, retraining the trained machine learning model based on the retargeting of the non-subscriber user.
2 . The computer-implemented method of claim 1 , wherein the third-party service is one of a third-party website, a social media platform, or an email or messaging service accessed by the non-subscriber user.
3 . The computer-implemented method of claim 1 , wherein the received consumer information includes one or more of a location identifier of a location associated with a user represented in the session data, income information associated with the location of the user represented in the session data, demographic information associated with the location of the user represented in the session data, personal information associated with the user represented in the session data, device information of a device associated with the user represented in the session data, or information about a current telecommunication service of the user represented in the session data.
4 . The computer-implemented method of claim 1 , wherein the received session data includes webpage category information associated with at least one webpage of the one or more websites of the telecommunication service provider.
5 . The computer-implemented method of claim 1 , wherein the generated historic session pattern includes, for user session activity across multiple sessions within an analyzed time period, at least one of a session frequency, a session count, an average number of webpages accessed per session, or session duration information.
6 . The computer-implemented method of claim 1 , wherein the retraining the trained machine learning model includes at least one of:
training the trained machine learning model at least a second time using the training dataset,
resampling at least a portion of the training dataset, or
training the trained machine learning model using a different training dataset.
7 . The computer-implemented method of claim 1 further comprising:
generating, based on the conversion score indicating the likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider, a retargeting recommendation for the non-subscribing user, the retargeting recommendation indicating whether to retarget or not to retarget the non-subscriber user, a communication channel via which the non-subscriber user should be retargeted, a content with which the non-subscriber user should be retargeted, or a combination thereof.
8 . At least one computer-readable medium, excluding transitory signals, carrying instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receive session data corresponding to multiple sessions associated with users visiting one or more websites of a telecommunication service provider, each session of the multiple sessions defining a user accessing and interacting with a set of multiple webpages of the one or more websites of the telecommunication service provider, and the received session data specifying, for each session of the multiple sessions: (1) a user identifier, (2) a session identifier, (3) a duration of time spent by the user on each webpage in the set of multiple webpages, (4) a set of user actions detected of the user interacting with the multiple webpages, (5) an indication of whether the user converted to a subscriber of the telecommunication service provider during or after a corresponding session, and (6) at least one content associated with the telecommunication service provider that is displayed to the user before the user converted;
generate session flows for each of the multiple sessions corresponding to the session data, each session flow of the session flows including stored information indicating a sequence of webpages in the set of multiple webpages that the user accessed while interacting with the set of multiple webpages, and the stored information for each webpage in the sequence of webpages indicating the duration of time and the set of user actions;
generate a historic session pattern associated with each user identifier specified in the received session data;
receive consumer information associated with each user identifier specified in the received session data, the received consumer information comprising a locked status indicating whether a device associated with the user identifier specified in the received session data is restricted to a specific provider of telecommunications services;
create a training dataset that includes: (1) each session flow of the generated session flows, (2) each generated historic session pattern associated with each user identifier specified in the received session data, (3) the received consumer information associated with each user identifier specified in the received session data, and (4) information designating each session flow of the session flows as being associated with either a converting user or a non-converting user;
train, using the training dataset, a machine learning model to use a combined input including: (1) an input session flow of a non-subscriber user interacting with a plurality of webpages of the one or more websites of the telecommunication service provider, (2) an input historic session pattern associated with the non-subscriber user, and (3) an input consumer information associated with the non-subscriber user, the input consumer information indicating that the non-subscriber user is a customer of another service provider differing from the telecommunication service provider;
generate, by the trained machine learning model processing the combined input, a conversion score indicating a likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider based on the input session flow, the input historic session pattern, and the input consumer information associated with the non-subscriber user;
retarget the non-subscriber user via a first communication channel of a third-party service accessed by the non-subscriber user as a result of the conversion score exceeding a first threshold score, the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at a user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the first communication channel, the third-party service being different from services provided by the telecommunication service provider;
retarget the non-subscriber user via a second communication channel of the third-party service accessed by the non-subscriber user when the conversion score exceeds a second threshold score different from the first threshold score, the second communication channel being different and separate from the first communication channel, and the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at the user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the second communication channel of the third-party service;
evaluate the trained machine learning model using a testing dataset, the testing dataset including session flows, historic session patterns, and consumer information of known converting and non-converting users;
assess accuracy of the trained machine learning model based on the evaluating and the retargeting of the non-subscriber user via the first communication channel or the second communication channel; and
when the accuracy of the trained machine learning model does not exceed a threshold accuracy, retrain the trained machine learning model based on the retargeting of the non-subscriber user.
9 . The at least one computer-readable medium of claim 8 , wherein the third-party service is one of a third-party website, a social media platform, or an email or messaging service accessed by the non-subscriber user.
10 . The at least one computer-readable medium of claim 8 , wherein the received consumer information includes one or more of a location identifier of a location associated with a user represented in the session data, income information associated with the location of the user represented in the session data, demographic information associated with the location of the user represented in the session data, personal information associated with the user represented in the session data, device information of a device associated with the user represented in the session data, or information about a current telecommunication service of the user represented in the session data.
11 . The at least one computer-readable medium of claim 8 , wherein the received session data includes webpage category information associated with at least one webpage of the one or more websites of the telecommunication service provider.
12 . The at least one computer-readable medium of claim 8 , wherein the generated historic session pattern includes, for user session activity across multiple sessions within an analyzed time period, at least one of a session frequency, a session count, an average number of webpages accessed per session, or session duration information.
13 . The at least one computer-readable medium of claim 8 , wherein the retraining the trained machine learning model further causes the at least one processor to perform operations comprising at least one of:
train the trained machine learning model at least a second time using the training dataset,
resample at least a portion of the training dataset, or
train the trained machine learning model using a different training dataset.
14 . The at least one computer-readable medium of claim 8 , wherein the instructions further cause the at least one processor to:
generate, based on the conversion score indicating the likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider, a retargeting recommendation for the non-subscribing user, the retargeting recommendation indicating whether to retarget or not to retarget the non-subscriber user a communication channel via which the non-subscriber user should be retargeted a content with which the non-subscriber user should be retargeted, or a combination thereof.
15 . A computing system comprising:
at least one hardware processor; and
at least one memory, excluding transitory signals, carrying instructions that, when executed by the at least one hardware processor, cause the computing system to perform operations comprising:
receive session data corresponding to multiple sessions associated with users visiting one or more websites of a telecommunication service provider, each session of the multiple sessions defining a user accessing and interacting with a set of webpages of the one or more websites of the telecommunication service provider, and the received session data specifying, for each session of the multiple sessions: (1) a user identifier, (2) a session identifier, (3) a duration of time spent by the user on each webpage in the set of multiple webpages, (4) a set of user actions detected of the user interacting with the multiple webpages, (5) an indication of whether the user converted to a subscriber of the telecommunication service provider, and (6) at least one content associated with the telecommunication service provider that is displayed to the user before the user converted;
generate session flows for each of the multiple sessions corresponding to the session data, each session flow of the session flows including stored information indicating a sequence of webpages in the set of multiple webpages that the user accessed while interacting with the set of multiple webpages, and the stored information for each webpage in the sequence of webpages indicating the duration of time and the set of user actions;
generate a historic session pattern associated with each user identifier specified in the received session data;
receive consumer information associated with each user identifier specified in the received session data, the received consumer information comprising a locked status indicating whether a device associated with the user identifier specified in the received session data is restricted to a specific provider of telecommunications services;
create a training dataset that includes: (1) each session flow of the generated session flows, (2) each generated historic session pattern associated with each user identifier specified in the received session data, (3) the received consumer information associated with each user identifier specified in the received session data, and (4) information designating each session flow of the session flows as being associated with either a converting user or a non-converting user;
train, using the training dataset, a machine learning model to use a combined input including: (1) an input session flow of a non-subscriber user interacting with a plurality of webpages of the one or more websites of the telecommunication service provider, (2) an input historic session pattern associated with the non-subscriber user, and (3) an input consumer information associated with the non-subscriber user, the input consumer information indicating that the non-subscriber user is a customer of another service provider differing from the telecommunication service provider;
generate, by the trained machine learning model processing the combined input, a conversion score indicating a likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider based on the input session flow, the input historic session pattern, and the input consumer information associated with the non-subscriber user;
retarget the non-subscriber user via a first communication channel of a third-party service accessed by the non-subscriber user as a result of the conversion score exceeding a first threshold score, the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at a user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the first communication channel, the third-party service being different from services provided by the telecommunication service provider;
retarget the non-subscriber user via a second communication channel of the third-party service accessed by the non-subscriber user when the conversion score exceeds a second threshold score different from the first threshold score, the second communication channel being different and separate from the first communication channel, and the non-subscriber user being retargeted to view the at least one content associated with the telecommunication service provider by:
causing display, at the user interface of the non-subscriber user, the at least one content associated with the telecommunications service provider via the second communication channel of the third-party service;
evaluate the trained machine learning model using a testing dataset, the testing dataset including session flows, historic session patterns, and consumer information of known converting and non-converting users;
assess accuracy of the trained machine learning model based on the evaluating and the retargeting of the non-subscriber user via the first communication channel or the second communication channel; and
when the accuracy of the trained machine learning model does not exceed a threshold accuracy, retrain the trained machine learning model based on the retargeting of the non-subscriber user.
16 . The computing system of claim 15 , wherein the third-party service is one of a third-party website, a social media platform, or an email or messaging service accessed by the non-subscriber user.
17 . The computing system of claim 15 , wherein the received consumer information includes one or more of a location identifier of a location associated with a user represented in the session data, income information associated with the location of the user represented in the session data, demographic information associated with the location of the user represented in the session data, personal information associated with the user represented in the session data, device information of a device associated with the user represented in the session data, or information about a current telecommunication service of the user represented in the session data.
18 . The computing system of claim 15 , wherein the received session data includes webpage category information associated with at least one webpage of the one or more websites of the telecommunication service provider.
19 . The computing system of claim 15 , wherein the retraining the trained machine learning model further causes the computing system to perform operations comprising at least one of:
train the trained machine learning model at least a second time using the training dataset,
resample at least a portion of the training dataset, or
train the trained machine learning model using a different training dataset.
20 . The computing system of claim 15 further caused to:
generate, based on the conversion score indicating the likelihood that the non-subscriber user will convert to a subscriber of the telecommunication service provider, a retargeting recommendation for the non-subscribing user, the retargeting recommendation indicating whether to retarget or not to retarget the non-subscriber user, a communication channel via which the non-subscriber user should be retargeted, a content with which the non-subscriber user should be retargeted, or a combination thereof.