IP Library › Granted Patent US 11,710,148
Granted Patent B2
US 11,710,148 · App. 17/163,467 · Granted Jul 25, 2023

Deep learning-based revenue-per-click prediction model framework

Inventors: Xuan Cao (Santa Clara, CA); Lu Wang (Cupertino, CA); Wei Shen (Pleasanton, CA)
Assignee: WALMART APOLLO, LLC
G06Q30/0247G06N3/04G06N3/08G06Q30/0242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,710,148
App. No.
17/163,467
Granted
Jul 25, 2023
Kind
B2
Abstract

A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions configured to run on the one or more processors and perform extracting meta features for an item to generate sparse feature embeddings for the item; reducing, using a multilayer perceptron, a dimension of the sparse feature embeddings to generate a representation vector for the meta features; extracting, using a recurrent neural network, sequential data from dense traffic features for the item over a period of time; and inputting the representation vector for the meta features and the sequential data from the dense traffic features into a multilayer neural network with a rectified linear unit (ReLU) activation function and a scoring layer to generate one or more performance metrics for the item. Other embodiments are disclosed.

Claims (48)

1. A system comprising:

one or more processors; and

one or more non-transitory computer-readable media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform functions comprising:

extracting meta features for an item to generate feature embeddings for the item;

reducing, using a multilayer perceptron, a dimension of the feature embeddings to generate a representation vector for the meta features;

extracting, using a recurrent neural network, sequential data from traffic features for the item over a period of time;

inputting the representation vector for the meta features and the sequential data from the traffic features into a multilayer neural network based on offline learning, wherein input for the multilayer neural network is concatenated and used as input for a first rectified linear unit (ReLU) activation function, wherein an output of the first ReLU activation function is used as a direct input for a second ReLU activation function of the multilayer neural network to output metrics to a first scoring layer, and wherein the first scoring layer generates one or more first performance metrics based on the offline learning for the item;

re-purposing the metrics output by the second ReLU activation function of the multilayer neural network; and

feeding the metrics output by the second ReLU activation function and real-time online data into a second scoring layer to generate one or more second performance metrics based on online learning for the item.

2. The system of claim 1 , wherein the computing instructions, when executed on the one or more processors, further cause the one or more processors to perform functions comprising:

generating, by the output of the second ReLU activation function of the multilayer neural network, an embedding vector that represents a combination of respective historical performance data for each meta feature and each traffic feature; and

storing embedding vectors generated by the metrics output by the second ReLU activation function of the multilayer neural network in a database, wherein the embedding vectors comprise the embedding vector and wherein the database stores at least the embedding vectors generated offline for use by an online service.

3. The system of claim 2 , wherein the computing instructions, when executed on the one or more processors, further cause the one or more processors to perform functions comprising:

retrieving the embedding vectors, as stored; and

training the first scoring layer based on the offline learning to predict a performance metric for the item, wherein the first scoring layer comprises a machine learning model.

4. The system of claim 3 , wherein the machine learning model comprises linear regression.

5. The system of claim 1 , wherein extracting the meta features comprises encoding hierarchy information about the item.

6. The system of claim 1 , wherein extracting the meta features comprises using an NLP-based embedding algorithm to embed text features about the item, wherein the text features comprise one or more of a title of the item and a description of the item.

7. The system of claim 1 , wherein extracting the meta features comprises extracting data from one or more images of the item using a convolutional neural network.

8. The system of claim 1 , wherein the one or more first performance metrics further comprise a conversion rate for the item, an order size for the item, and a contributed profit per order for the item.

9. The system of claim 8 , wherein the computing instructions, when executed on the one or more processors, further cause the one or more processors to perform a function comprising:

generating a revenue per click metric for the item based on the conversion rate for the item and the order size for the item.

10. The system of claim 1 , wherein the computing instructions, when executed on the one or more processors, further cause the one or more processors to perform functions comprising:

combining embedding vectors of the offline learning with real-time online data of the online learning for the item to generate inputs for the second scoring layer; and

generating a contributed profit per click metric based on a conversion rate and a contributed profit per order.

11. A method being implemented via execution of computing instructions configured to run on one or more processors and stored at one or more non-transitory computer-readable media, the method comprising:

extracting meta features for an item to generate feature embeddings for the item;

reducing, using a multilayer perceptron, a dimension of the feature embeddings to generate a representation vector for the meta features;

extracting, using a recurrent neural network, sequential data from traffic features for the item over a period of time;

inputting the representation vector for the meta features and the sequential data from the traffic features into a multilayer neural network based on offline learning, wherein input for the multilayer neural network is concatenated and used as input for a first rectified linear unit (ReLU) activation function, wherein an output of the first ReLU activation function is used as a direct input for a second ReLU activation function of the multilayer neural network to output metrics to a first scoring layer, and wherein the first scoring layer generates one or more first performance metrics based on the offline learning for the item;

re-purposing the metrics output by the second ReLU activation function of the multilayer neural network; and

feeding the metrics output by the second ReLU activation function and real-time online data into a second scoring layer to generate one or more second performance metrics based on online learning for the item.

12. The method of claim 11 , further comprising:

generating, by the output of the second ReLU activation function of the multilayer neural network, an embedding vector that represents a combination of respective historical performance data for each meta feature and each traffic feature; and

storing embedding vectors generated by the metrics output by the second ReLU activation function of the multilayer neural network in a database, wherein the embedding vectors comprise the embedding vector and wherein the database stores at least the embedding vectors generated offline for use by an online service.

13. The method of claim 12 , further comprising:

retrieving the embedding vectors, as stored; and

training the first scoring layer based on the offline learning to predict a performance metric for the item, wherein the first scoring layer comprises a machine learning model.

14. The method of claim 13 , wherein the machine learning model comprises linear regression.

15. The method of claim 11 , wherein extracting the meta features comprises encoding hierarchy information about the item.

16. The method of claim 11 , wherein extracting the meta features comprises using an NLP-based embedding algorithm to embed text features about the item, wherein the text features comprise one or more of a title of the item and a description of the item.

17. The method of claim 11 , wherein extracting the meta features comprises extracting data from one or more images of the item using a convolutional neural network.

18. The method of claim 11 , wherein the one or more first performance metrics further comprise a conversion rate for the item, an order size for the item, and a contributed profit per order for the item.

19. The method of claim 18 , further comprising:

generating a revenue per click metric for the item based on the conversion rate for the item and the order size for the item.

20. The method of claim 11 , further comprising:

combining embedding vectors of the offline learning with real-time online data of the online learning for the item to generate inputs for the second scoring layer; and

generating a contributed profit per click metric based on a conversion rate and a contributed profit per order.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2021
From: CAO, XUAN; WANG, LU; SHEN, WEI
To: WALMART APOLLO, LLC
Reel/Frame 055744/0034 →
Continuity (1)
Related Publication 20220245670A1 · Aug 4, 2022