Systems for e-commerce recommendations
View Patent ↗A system and method for recommendations for online e-commerce to a user, which provide a number of different methods for surfacing a current desire or need of the user in terms of a purchase, through a recommendation engine that applies one or more neural net models. One non-limiting example of such a method is to apply a first model for analyzing the current online behavior of the user, and a second model for analyzing past online behavior of the user, then concatenating the outputs of both models to a single output, to determine a recommendation for the user. The two models may be of the same or different type. For example and without limitation, the first model may comprise a CNN, while the second model may comprise a CNN and/or a transformer-based model. The transformer-based model may comprise an encoder alone, rather than the art-known combination of an encoder and a decoder. If the model comprises a CNN, preferably the CNN features a plurality of overlapping filters of different shapes. Optionally the CNN comprises a single convolutional layer. Optionally each such model may comprise a plurality of different models. Optionally the recommendation engine may also comprise another type of AI or machine learning algorithm, in addition to one or more neural net models.
1. A system for analyzing online temporal interactions of a user, the system comprising a user computational device for operation by the user, a server for analyzing said online temporal interactions of the user and a computer network for connecting the user computational device to the server, wherein said user computational device comprises a user interface for receiving user actions and for displaying content to the user, said server comprises a recommendation engine for analyzing said user actions and for recommending additional content for provision to said user interface; wherein said recommendation engine comprises a plurality of models, comprising a first model for analyzing a current online behavior of the user arranged in a session, and a second model for analyzing past online behavior of the user, wherein said recommendation engine concatenates outputs of both models to a single output, to determine additional content for display through said user interface; wherein said server comprises a memory for storing a plurality of instructions for operating said recommendation engine and a processor for executing said plurality of instructions; wherein said first model comprises a CNN (Convolution Neural Network), wherein said CNN comprises a plurality of overlapping filters having different filter shapes and wherein said CNN has a single 3D convolution layer only, such that said plurality of overlapping filters having different filter shapes are all present in said single layer; wherein said CNN receives each character in a time sequence during said session; wherein said output of said single 3D convolution layer is fed to a 3D max pooling layer, wherein said second model comprises a transformer-based model, wherein said transformer-based model comprises an encoder alone, without a decoder; wherein said transformer-based model receives a plurality of tokens corresponding to said past online behavior of the user; wherein said recommendation engine concatenates outputs of both models to a single output through a concatenation layer;
further comprising training said CNN by receiving data comprising data obtained from current online behavior of the user comprising actions taken by the user in a current session;
preprocessing said data to remove noise; feeding said data to said CNN character by character; then applying an Adam optimizer and L2 regularization, after first initializing weights for said CNN with a truncated Gaussian;
further comprising training said transformer-based model by receiving data comprising data obtained from past online behavior of the user, comprising actions taken by the user in a previous session; preprocessing said data to remove noise; tokenizing said data and feeding said tokenized data to said transformer-based model.
2. The system of claim 1 , wherein said online commerce comprises an online system for purchasing goods and/or services.
3. The system of claim 2 , wherein said online system comprises one or more of an online marketplace, a system for purchasing physical goods, digital goods, media and other content, and services, whether provided digitally or physically, or a combination thereof.
4. The system of claim 3 , wherein said user computational device communicates with said server in a session and wherein the current online behavior of the user comprises actions taken by the user in a current session.
5. The system of claim 3 , wherein said actions taken by the user in said current session comprise one or more of clicking on an item, staying on a page showing the item or a collection of items, placing the item in a shopping cart, and indicating the item for a wish list or otherwise for future access (short of purchase); wherein each item comprises a good and/or service with which the user interacted during the session.
6. The system of claim 5 , wherein said recommendation engine operates according to an item id (identifier), an item name and an item category or categories, determined according to said actions.
7. The system of claim 5 , wherein said actions taken by the user in said current session comprise one or more of clicking on a specific category page or other category grouping, or performing a search through content provided by said server to said user inter face.
8. The system of claim 7 , wherein said recommendation engine records actions taken by the user in each current session.
9. The system of claim 8 , wherein said past online behavior of the user comprises information about the user, apart from actions taken by the user in a current or previous session.
10. The system of claim 9 , wherein said past online behavior of the user comprises one or more of a record of a previous purchase by the user, a profile of the user or a combination thereof.
11. The system of claim 1 , wherein said user computational device comprises a memory for storing a plurality of instructions for operating said user interface and a processor for executing said plurality of instructions.
12. The system of claim 1 , wherein an output of said concatenation layer comprises a plurality of classes and is fed to a softmax to determine a probability distribution of said plurality of classes for selecting a class with a highest probability.
13. The system of claim 12 , wherein said second model for analyzing past online behavior of the user outputs an embedding matrix, wherein said embedding matrix is prepared by embedding a user vector representing previous online behavior of the user.
14. The system of claim 13 , wherein said concatenation layer is placed after a dropout layer but before an output layer for outputting said recommendation.
15. The system of claim 1 , wherein Adam optimizer is applied with a learning rate of 0.001; said L2 regularization is applied with a value of 0.00001; and said truncated Gaussian has a mean of 0 and a standard deviation of 0.1.
16. The system of claim 1 , wherein said tokenized data is converted to a vector before being fed to said transformer-based model.
17. The system of claim 16 , wherein said transformer-based model receives said plurality of tokens corresponding to said past online behavior of the user if data corresponding to past online behavior of the user is available; if said data is not available, textual meta-data is provided to said CNN to overcome a cold start problem for analyzing online behavior of the user in a current session.
18. A system for analyzing online temporal interactions of a user to predict a next action of the user, the system comprising a user computational device for operation by the user, a server for analyzing said online temporal interactions of the user and a computer network for connecting the user computational device to the server, wherein said user computational device comprises a user inter face for receiving user actions and for displaying content to the user, said server comprises a prediction engine for analyzing said user actions and for predicting a next action by said user: wherein said prediction engine comprises a plurality of models, comprising a first model for analyzing a current online behavior of the user arranged in a session, and a second model for analyzing past online behavior of the user, wherein said prediction engine concatenates outputs of both models to a single output, to determine additional content for display through said user interface; wherein said server comprises a memory for storing a plurality of instructions for operating said prediction engine and a processor for executing said plurality of instructions; wherein said first model comprises a CNN (Convolution Neural Network), wherein said CNN comprises a plurality of overlapping filters having different filter shapes and wherein said CNN has a single 3D convolution layer only, such that said plurality of overlapping filters having different filter shapes are all present in said single layer; wherein said CNN receives each character in a time sequence during said session; wherein said output of said single 3D convolution layer is fed to a 3D max pooling layer; wherein said second model comprises a transformer-based model, wherein said transformer-based model comprises an encoder alone, without a decoder; wherein said transformer-based model receives a plurality of tokens corresponding to said past online behavior of the user; wherein said prediction engine concatenates outputs of both models to a single output through a concatenation layer; wherein said output of said prediction engine is applied to automatically update a user interface for display to the user.
19. The system of claim 18 , wherein said transformer-based model receives said data corresponding to said past online behavior of the user if data corresponding to past online behavior of the user is available; if said data is not available, textual meta-data is provided to said CNN to overcome a cold start problem for analyzing online behavior of the user in a current session.
20. The system of claim 18 , wherein said transformer-based model is trained by tokenizing said data; and feeding said tokenized data to said transformer-based model.
21. The system of claim 20 , wherein said tokenized data is converted to a vector before being fed to said transformer-based model.
22. The system of claim 21 , wherein a plurality of vectors are provided as a user embedding matrix for training said transformer-based model.
23. A system for analyzing online temporal interactions of a user to automatically update a user interface for display to the user, the system comprising a user computational device for operation by the user, a server for analyzing said online temporal interactions of the user and a computer network for connecting the user computational device to the server, wherein said user computational device comprises a display for displaying said user interface, wherein said user interface displays content to the user, said server comprises a recommendation engine for analyzing said user actions and for updating said user interface according to said analyzing; wherein said recommendation engine comprises a plurality of models, comprising a first model for analyzing a current online behavior of the user arranged in a session, and a second model for analyzing past online behavior of the user, wherein said recommendation engine concatenates outputs of both models to a single output, and then updates said user interface according to said single output; wherein said server comprises a memory for storing a plurality of instructions for operating said recommendation engine and a processor for executing said plurality of instructions; wherein said first model comprises a CNN (Convolution Neural Network), wherein said CNN comprises a plurality of overlapping filters having different filter shapes and wherein said CNN has a single 3D convolution layer only, such that said plurality of overlapping filters having different filter shapes are all present in said single layer; wherein said CNN receives each character in a time sequence during said session; wherein said output of said single 3D convolution layer is fed to a 3D max pooling layer; wherein said second model comprises a transformer-based model, wherein said transformer-based model comprises an encoder alone, without a decoder; wherein said transformer-based model receives a plurality of tokens corresponding to said past online behavior of the user if data corresponding to past online behavior of the user is available; if said data is not available, providing textual meta-data to said CNN to overcome a cold start problem for analyzing online behavior of the user in a current session; wherein said recommendation engine concatenates outputs of both models to a single output through a concatenation layer.
24. The system of claim 23 , further comprising training said CNN by receiving data comprising data obtained from past online behavior of the user, comprising actions taken by the user in a previous session; preprocessing said data to remove noise; then feeding each said character in said time sequence during a previously recorded user session to said CNN.
25. The system of claim 24 , wherein said CNN is trained by applying an Adam optimizer and L2 regularization, after first initializing weights for said CNN with a truncated Gaussian.
26. The system of claim 25 , wherein said Adam optimizer is applied with a learning rate of 0.001; said L2 regularization is applied with a value of 0.00001; and said truncated Gaussian has a mean of 0 and a standard deviation of 0.1.
27. The system of claim 24 , wherein said transformer-based model is trained by tokenizing said data; and feeding said tokenized data to said transformer-based model.
28. The system of claim 21 , wherein said tokenized data is converted to a vector before being fed to said transformer-based model.