Methods and apparatuses for providing search results using embedding-based retrieval

1. A system comprising:

a non-transitory memory configured to store instructions thereon and a processor which is configured by the instructions to:

obtain query information characterizing a query initiated by a customer on an ecommerce marketplace;

determine embedding-based search results comprising a first list of items, wherein the embedding-based search results are determined using a trained embedding-based machine learning model configured to use a cosine similarity and a prediction score, wherein the cosine similarity is defined as:

logit(query,item)=exp(cosine_distance( f (query), g (title,features))

where item is each item in the first list of items, title is a title of the item, and features is one or more features of the item and wherein the embedding-based machine learning model is trained using a training method comprising:

obtaining query-item pair data comprising queries and item titles;

tokenizing the queries and the item titles in the query-item pair data;

labelling each query-item pair in the query-item pair data using an engagement score; and

training the embedding-based machine learning model using the cosine similarity function and the product feature loss function; and

obtain legacy search results comprising a second list of items;

blend the embedding-based search results with the legacy search results to obtain blended search results, wherein the blended search results are blended using one or more blending criteria that characterizes engagement by customers with the items in the first list of items; and

send the blended search results to the customer.

2. The system of claim 1 , wherein a relative position of the items in the first list of items relative to the items in the second list of items in the blended search results is determined by comparing an engagement of the customer with each item in a recent time period with an engagement of the customer with each item prior to the recent time period.

3. The system of claim 1 , wherein the processor is operated in parallel with a legacy retrieval system that is configured to return the legacy search results.

4. A method comprising:

obtaining query information characterizing a query initiated by a customer on an ecommerce marketplace;

determining embedding-based search results comprising a first list of items, wherein the embedding-based search results are determined using a trained embedding-based machine learning model configured to use a cosine similarity and a prediction score, wherein the cosine similarity is defined as:

logit(query,item)=exp(cosine_distance( f (query), g (title,features))

where item is each item in the first list of items, title is a title of the item, and features is one or more features of the item and wherein the embedding-based machine learning model is trained using a training method comprising:

obtaining query-item pair data comprising queries and item titles;

tokenizing the queries and the item titles in the query-item pair data;

labelling each query-item pair in the query-item pair data using an engagement score; and

training the embedding-based machine learning model using the cosine similarity function and the product feature loss function; and

obtaining legacy search results comprising a second list of items;

blending the embedding-based search results with the legacy search results to obtain blended search results, wherein the blended search results are blended using one or more blending criteria that characterizes engagement by customers with the items in the first list of items; and

sending the blended search results to the customer.

5. The method of claim 4 , wherein a relative position of the items in the first list of items relative to the items in the second list of items in the blended search results is determined by comparing an engagement of the customer with each item in a recent time period with an engagement of the customer with each item prior to the recent time period.

6. The method of claim 4 , wherein the method is performed in parallel with a legacy retrieval system that is configured to return the legacy search results.

7. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

obtaining query information characterizing a query initiated by a customer on an ecommerce marketplace;

determining embedding-based search results comprising a first list of items, wherein the embedding-based search results are determined using a trained embedding-based machine learning model configured to use a cosine similarity and a prediction score, wherein the cosine similarity is defined as:

logit(query,item)=exp(cosine_distance( f (query), g (title,features))

where item is each item in the first list of items, title is a title of the item, and features is one or more features of the item and wherein the embedding-based machine learning model is trained using a training method comprising:

obtaining query-item pair data comprising queries and item titles;

tokenizing the queries and the item titles in the query-item pair data;

labelling each query-item pair in the query-item pair data using an engagement score; and

training the embedding-based machine learning model using the cosine similarity function and the product feature loss function; and

obtaining legacy search results comprising a second list of items;

blending the embedding-based search results with the legacy search results to obtain blended search results, wherein the blended search results are blended using one or more blending criteria that characterizes engagement by customers with the items in the first list of items; and

sending the blended search results to the customer.

8. The non-transitory computer readable medium of claim 7 , wherein a relative position of the items in the first list of items relative to the items in the second list of items in the blended search results is determined by comparing an engagement of the customer with each item in a recent time period with an engagement of the customer with each item prior to the recent time period.

9. The non-transitory computer readable medium of claim 7 , wherein the operations are performed in parallel with a legacy retrieval system that is configured to return the legacy search results.

10. The system of claim 1 , wherein the prediction score is determined by a product feature loss function defined as:

loss(query,item (1) ,item (2) , . . . ,item (n) )=Σ i ( P (query,item (i) )*score(item (i) ))

where

p

⁡

(

query

,

item

i

)

=

logit

(

query

,

item

(

i

)

)

∑

logit

(

query

,

item

(

k

)

)

and where score(item (i) ) is an engagement score.

11. The system of claim 1 , wherein a loss function of the trained embedding-based machine learning model comprises:

loss(item (i) ,pt (j) )=− f (item (i) ,pt j )+log(Σ k exp( f (item (i) ,pt (k) ))).

12. The method of claim 4 , wherein the prediction score is determined by a product feature loss function defined as:

loss(query,item (1) ,item (2) , . . . ,item (n) )=Σ i ( P (query,item (i) )*score(item (i) ))

where

p

⁡

(

query

,

item

i

)

=

logit

(

query

,

item

(

i

)

)

∑

logit

(

query

,

item

(

k

)

)

and where score(item (i) ) is an engagement score.

13. The method of claim 4 , wherein a loss function of the trained embedding-based machine learning model comprises:

loss(item (i) ,pt (j) )=− f (item (i) ,pt j )+log(Σ k exp( f (item (i) ,pt (k) ))).

14. The non-transitory computer readable medium of claim 7 , wherein the prediction score is determined by a product feature loss function defined as:

loss(query,item (1) ,item (2) , . . . ,item (n) )=Σ i ( P (query,item (i) )*score(item (i) ))

where

p

⁡

(

query

,

item

i

)

=

logit

(

query

,

item

(

i

)

)

∑

logit

(

query

,

item

(

k

)

)

and where score(item (i) ) is an engagement score.

15. The non-transitory computer readable medium of claim 7 , wherein a loss function of the trained embedding-based machine learning model comprises:

loss(item (i) ,pt (j) )=− f (item (i) ,pt j )+log(Σ k exp( f (item (i) ,pt (k) ))).

16. The non-transitory computer readable medium of claim 15 , wherein embedding-based machine learning model is trained using a training method comprising:

obtaining query-item pair data comprising queries and item titles;

tokenizing the queries and the item titles in the query-item pair data;

labelling each query-item pair in the query-item pair data using an engagement score; and

training the embedding-based machine learning model using the cosine similarity function and the product feature loss function.