Deduplication of query to assortment pages
A method including generating, using a semantic embedding generation machine learning model, one or more respective shelf embedding vector representations for each of one or more browse shelves based on a respective shelf name for the each of the one or more browse shelves. The method also can include obtaining a keyword. The method additionally can include generating, using the semantic embedding generation machine learning model, a keyword embedding vector representation based on the keyword. The method further can include determining a respective similarity score between the keyword embedding vector representation and each of the one or more respective shelf embedding vector representations for each of the one or more browse shelves. The method additionally can include determining whether any of the respective similarity scores for the one or more respective shelf embedding vector representations across the one or more browse shelves exceed a predetermined threshold value. When any of the respective similarity scores for the one or more respective shelf embedding vector representations across the one or more browse shelves exceeds a predetermined threshold value, the method further can include filtering out the keyword. When none of the respective similarity scores for the one or more respective shelf embedding vector representations across the one or more browse shelves exceed a predetermined threshold value, the method additionally can include generating a new topic page using the keyword. Other embodiments are described.
1 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions that, when executed on the one or more processors, cause the one or more processors to perform:
generating, using a semantic embedding generation machine learning model, shelf embedding vector representations for browse shelves based on shelf names for the browse shelves, wherein a browse page, on a website and for a browse shelf of the browse shelves, lists items that are categorized into a particular category that corresponds to a shelf name of the shelf names and a primary category path within a product taxonomy;
obtaining a keyword;
generating, using the semantic embedding generation machine learning model, a keyword embedding vector representation based on the keyword;
determining a similarity score, indicating closeness of scope of the browse shelf to the keyword, based on the keyword embedding vector representation and a shelf embedding vector representation, of the shelf embedding vector representations, for the browse shelf;
determining that the similarity score, indicating closeness of scope of the browse shelf to the keyword, exceeds a threshold value; and
generating one or more new topic pages that overlap in scope with the browse page, are configured to be crawled by third-party search engines, and list items capturing trending intents without generating a new topic page that is nearly a duplicate of the browse page based on determining that the similarity score, indicating closeness of scope of the browse shelf to the keyword, exceeds the threshold value.
2 . The system of claim 1 , wherein the semantic embedding generation machine learning model comprises:
a Bidirectional Encoder Representations from Transformers (BERT) layer;
a first feed forward layer; and
a second feed forward layer.
3 . The system of claim 2 , wherein:
the semantic embedding generation machine learning model is a portion of a query classification machine learning model comprising a third feed forward layer configured to output a plurality of sigmoid functions for calculating multilabel probabilities across the browse shelves; and
after the BERT layer has been trained, the BERT layer is fine-tuned using back-propagated training with augmented training data comprising augmented shelf representations for the browse shelves.
4 . The system of claim 1 , wherein the similarity score, indicating closeness of scope of the browse shelf to the keyword, comprises is based on a cosine similarity measure.
5 . The system of claim 1 , wherein the threshold value is approximately 0.88.
6 . The system of claim 1 , wherein generating the shelf embedding vector representations comprises:
generating a first respective shelf embedding vector representation of the shelf embedding vector representations based on a first respective input to the semantic embedding generation machine learning model;
generating a second respective shelf embedding vector representation of the shelf embedding vector representations based on a second respective input to the semantic embedding generation machine learning model; and
generating a third respective shelf embedding vector representation of the shelf embedding vector representations based on a third respective input to the semantic embedding generation machine learning model,
wherein the shelf embedding vector representation is the first respective shelf embedding vector representation, the second respective shelf embedding vector representation, or the third respective shelf embedding vector representation.
7 . The system of claim 6 , wherein:
the first respective input comprises the shelf name;
the second respective input comprises a first taxonomy level of the primary category path; and
the third respective input comprises a shortened version of the primary category path.
8 . The system of claim 7 , wherein:
the shortened version of the primary category path is generated using suffix mapping and keyword filtering.
9 . The system of claim 1 , wherein the shelf embedding vector representations are pre-generated before obtaining the keyword.
10 . The system of claim 1 , wherein the one or more new topic pages include an assortment page that is able to be crawled by a search engine.
11 . A method implemented via execution of computing instructions configured to run at one or more processors, the method comprising:
generating, using a semantic embedding generation machine learning model, one or more respective shelf embedding vector representations for each of one or more browse shelves based on a respective shelf name, wherein one or more browse pages, on a web site and for the one or more browse shelves, include a browse page that lists items that are categorized into a particular category;
obtaining a keyword;
generating, using the semantic embedding generation machine learning model, a keyword embedding vector representation based on the keyword;
determining a respective similarity score between the keyword embedding vector representation and each of the one or more respective shelf embedding vector representations;
determining that a threshold value is satisfied by a similarity score, of the respective similarity scores, indicating closeness of scope of a browse shelf, of the one or more browse shelves, to the keyword; and
generating one or more new topic pages that overlap in scope with the browse page and list items that capture trending intents without generating a new topic page that is nearly a duplicate of the browse page based on determining that the threshold value is satisfied by the similarity score that indicates closeness of scope of the browse shelf to the keyword.
12 . The method of claim 11 , wherein the semantic embedding generation machine learning model comprises:
a Bidirectional Encoder Representations from Transformers (BERT) layer;
a first feed forward layer; and
a second feed forward layer.
13 . The method of claim 12 , wherein:
the semantic embedding generation machine learning model is a portion of a query classification machine learning model further comprising a third feed forward layer configured to output a plurality of sigmoid functions for calculating multilabel probabilities across the one or more browse shelves; and
after the BERT layer has been trained, the BERT layer is fine-tuned using back-propagated training with augmented training data comprising augmented shelf representations for the one or more browse shelves.
14 . The method of claim 11 , wherein determining the respective similarity score comprises:
using a cosine similarity measure to generate the respective similarity score.
15 . The method of claim 11 , wherein the threshold value is approximately 0.88.
16 . The method of claim 11 , wherein generating the one or more respective shelf embedding vector representations based on:
a first respective input to the semantic embedding generation machine learning model,
a second respective input to the semantic embedding generation machine learning model, and
a third respective input to the semantic embedding generation machine learning model.
17 . The method of claim 16 , wherein:
the first respective input comprises the respective shelf name,
the second respective input comprises a first taxonomy level of a respective primary category path of the respective shelf name, and
the third respective input comprises a shortened version of the respective primary category path of the respective shelf name.
18 . The method of claim 17 , wherein:
the shortened version of the respective primary category path of the respective shelf name is generated using suffix mapping and keyword filtering.
19 . The method of claim 11 , wherein the one or more respective shelf embedding vector representations are pre-generated before obtaining the keyword.
20 . A non-transitory, computer-readable medium comprising instructions that, when executed by a processing resource, causes the processing resource to:
generate, using a semantic embedding generation machine learning model, a shelf embedding vector representation for a browse shelf based on a shelf name for the browse shelf, wherein a browse page, on a website and for the browse shelf, lists items that are categorized into a particular category that corresponds to the shelf name;
generate, using the semantic embedding generation machine learning model, a keyword embedding vector representation based on a keyword;
determine a similarity score, indicating closeness of scope of the browse shelf to the keyword, based on the keyword embedding vector representation and the shelf embedding vector representation;
determine that the similarity score, indicating closeness of scope of the browse shelf to the keyword, exceeds a threshold value; and
generate one or more new topic pages that overlap in scope with the browse page and list items that capture trending intents without generating a new topic page that is nearly a duplicate of the browse page based on determining that the similarity score, indicating closeness of scope of the browse shelf to the keyword, exceeds the threshold value.