Granted Patent B2

US 12,725,185 · App. 17/887,425 · Granted Sep 1, 2026

Inventors: Karl Alexander Philip Ekfeldt (Cambridge, MA); Sarah Fathelbab (Cliffside Park, NJ); Ezra Freedman (Boston, MA); Joshua Edward Villarreal (Somerville, MA); Royce Ze Yong Yap (Singapore, SG)

Assignee: Klaviyo, Inc.

G06Q30/0276G06F40/166G06F40/279G06Q30/0243

Abstract

Claims (71)

1 . A computer-implemented method for processing machine-generated subject lines, comprising:

requesting, via a graphical user interface, information comprising a specified type and description of an electronic campaign;

receiving, by a server, said information related to the electronic campaign;

preprocessing, by the server, the received information for sending to a generative text engine model, wherein the preprocessing comprises checking formatting, removing keywords, and ensuring the user has provided sufficient information;

sending to the generative text engine model the preprocessed received information, wherein the generative text engine model is operable to return text based on a given text prompt;

receiving a plurality of N subject lines generated based on the preprocessed received information from the generative text engine model;

reducing, by the server, the N subject lines down to M subject lines, wherein the reducing comprises:

generating similarity scores by embedding subject lines as vectors using vectorization methods and computing, for each pair of subject lines, a cosine similarity score;

eliminating one of the pair based on whether the cosine similarity score exceeds a first similarity threshold;

computing cosine similarity scores between at least one subject line and the merchant-provided description using the vectorization methods;

eliminating one or more subject lines when a cosine similarity score between a subject line and the merchant-provided description exceeds a second similarity threshold different from the first similarity threshold; and

wherein when the reducing results in less than a threshold number of subject lines, resending a request to the generative text engine model to generate additional subject lines;

assigning, by a trained second discriminator model, a second quality rating for each of the M subject lines, wherein the second discriminator model was trained based on historical tracked customer user actions including opens and clicks that included a displayed subject line; and

reducing the M subject lines to K subject lines based on the second quality rating;

applying a fine-tuned enhancer model to enhance existing content for a selected one or more of the K subject lines;

displaying K subject lines with enhanced content to a merchant user;

displaying select subject lines to customer users at the customer user devices;

tracking customer user actions based on the select subject lines displayed to the customer users at the customer user devices, wherein the customer user devices operate alone or optionally, in conjunction with a server and/or merchant server to sense customer user actions;

analyzing the customer user actions responsive for success metrics corresponding to the displayed subject lines including open rates and click rates;

carrying out a first training, wherein the first training comprises training, through a discriminator improvement cycle, the second discriminator model to assign a second quality rating to a subject line based on the analyzed customer user actions including open rates and click rates, and updating the second discriminator model at the conclusion of the discriminator improvement cycle; and

carrying out a second training, wherein the second training comprises training. through a text generation improvement cycle, the generative text engine model based on subject lines meeting a second quality rating threshold as updated by the second discriminator model during the first training, and updating the generative text engine model at the conclusion of the text generation improvement cycle.

2 . The method of claim 1 , wherein reducing the N subject lines down to M subject lines, further comprises eliminating subject lines that have greater than a character threshold number of characters.

3 . The method of claim 1 , wherein the reducing the N subject lines to eliminate subject lines comprises filtering the N subject line to eliminate subject lines based on content, including eliminating subject lines that include inappropriate and politically sensitive words.

4 . The method of claim 1 , wherein generating similarity scores between each of the N subject lines and eliminating one or more subject lines based on a similarity with one or more other subject lines comprises prioritizing diversity across displayed subject lines based on pairwise similarity scores.

5 . The method of claim 1 , wherein applying the fine-tuned enhancer model to enhance existing content includes adding emojis to the selected one or more of the K subject lines.

6 . The method of claim 5 , wherein enhancing the existing content is based on collected information from subject lines of past high-performing electronic campaigns and adjusting content of the subject lines to reflect a brand tone of the merchant user based on content and success of past high-performing electronic campaigns and existing website content.

7 . The method of claim 1 , wherein applying a fine-tuned enhancer model to enhance existing content includes adjusting or supplementing the selected one or more of the K subject lines.

8 . The method of claim 1 , wherein the server is electronically connected to the merchant server, and further comprising:

tracking, by the server, merchant actions at the merchant server based on the M subject lines displayed;

assigning, by a first discriminator model, a quality rating for each of the M subject lines based on the tracked merchant actions; and

supplementing data used to train the fine-tuned enhancer model with J highest quality rating subject lines.

9 . The method of claim 8 further comprising continuously updating the generative text engine model based on continuously generated quality ratings.

10 . The method of claim 1 , further comprising:

supplementing data used to train the fine-tuned enhancer model with the K highest quality rating subject lines as determined by actions of the one or more customers.

11 . The method of claim 1 , wherein the sensing the customer actions comprises sensing online action of the one or more customers.

12 . The method of claim 1 , wherein the sensing the customer actions comprises sensing physical motion of the customer devices of the one or more customers.

13 . The method of claim 1 , wherein assigning by the second quality rating by the second discriminator model comprises receiving, by the server, customer actions for electronic messages having different subject lines and a same message content.

14 . A system for processing machine-generated subject lines, comprising:

a merchant server operable to sense customer user actions at a plurality of customer user devices and to provide the sensed customer user actions to a management server;

the management server electronically connected to the merchant server through a network, the management server configured to:

receive information from a merchant user of the merchant server;

preprocess the received information for sending to a generative text engine model, wherein the preprocessing comprises checking formatting, removing keywords, and ensuring the user has provided sufficient information;

retrieve a plurality of N subject lines generated based on the preprocessed received information from the generative text engine model;

reduce the N subject lines down to M subject lines, comprising:

generating similarity scores by embedding subject lines as vectors using vectorization methods and computing, for each pair of subject lines, a cosine similarity score;

eliminating one of the pair based on whether the cosine similarity score exceeds a first similarity threshold;

computing cosine similarity scores between at least one subject line and the merchant-provided description using the vectorization methods;

eliminating one or more subject lines when a cosine similarity score between a subject line and the merchant-provided description exceeds a second similarity threshold different from the first similarity threshold; and

wherein when the reducing results in less than a threshold number of subject lines. resending a request to the generative text engine model to generate additional subject lines;

apply a fine-tuned enhancer model to enhance existing content for a selected one or more of the M subject lines, wherein the fine-tuned enhancer model is operable of altering one or more keywords within a subject line and to append text onto the subject line;

the management server further configured to:

communicate the M subject lines to the merchant server;

wherein the merchant server is configured to display the M subject lines with enhanced existing content to the merchant user.

15 . The system of claim 14 , wherein the management server is further configured to:

track merchant actions at the merchant server based on the M subject lines displayed;

wherein a first discriminator model is configured to:

assign a quality rating for each of the M subject lines based on the tracked merchant actions; and

wherein the management server is further configured to:

supplement data used to train the fine-tuned enhancer model with J highest quality rating subject lines.

16 . The system of claim 14 , further comprising a plurality of customer devices electronically connected to the server through the network;

wherein the management server is further configured to:

track customer actions of customers at the customer devices based on the subject lines displayed to one or more customers of a merchant of the merchant server;

train a second discriminator model based on historical tracked customer user actions;

wherein the trained second discriminator model is configured to:

assign a second quality rating for each of the subject lines displayed to the one or more customers based on the tracked customer actions; and

wherein the management server is further configured to:

reduce the M subject lines to K subject lines based on the second quality rating;

send at least one of the displayed K subject lines to customer users;

update the second discriminator model based on the tracked customer user actions to complete a discriminator improvement cycle; and

repeat the send, track, train and update steps through multiple discriminator improvement cycles, for multiple electronic campaigns.

17 . The system of claim 16 , wherein the customer user devices comprise location and motion sensors for tracking the locations and motions of the customer user, and wherein the tracked locations and motions of the customer user can be included within a first score of the customer user action, and wherein a first score that exceeds a threshold score is deemed a customer user action.

Assignments (1)

Continuity (1)

References Cited (207)