Entity-aware multi-task machine learning

1 . A system, comprising:

a processor; and

a non-transitory memory storing instructions, that when executed, cause the processor to:

obtain a search query,

using an entity retrieval model:

encode the search query into a normalized query embedding using a natural language model of the entity retrieval model,

identify at least one normalized entity embedding among a plurality of normalized entity embeddings using the normalized query embedding,

determine at least one index associated with the at least one normalized entity embedding,

locate and retrieve a plurality of entities from an entity database based on the at least one index,

filter the plurality of entities based on historical user engagement data to generate a plurality of filtered entities, and

process the plurality of filtered entities to generate at least one entity based on the search query,

generate, using a neural network, query embedding data based on the search query and the at least one entity,

generate, using a plurality of task-specific networks, task prediction data for a plurality of tasks based on the query embedding data, wherein each task of the plurality of tasks captures a different aspect of a user intent associated with the search query, and

generate at least one search result for the search query based on the task prediction data for the plurality of tasks.

2 . The system of claim 1 , wherein the at least one entity is generated based at least in part by:

based on a nearest neighbor search performed using the normalized query embedding identifying the at least one normalized entity embedding among thes plurality of normalized entity embeddings.

3 . The system of claim 2 , wherein:

the processor is further caused to:

process the plurality of filtered entities to generate a plurality of processed entities, wherein:

all of the plurality of processed entities have a same format,

each of the plurality of processed entities is associated with a corresponding score that indicates a degree of importance of the processed entity with respect to the search query, and

the at least one entity comprises the plurality of processed entities generated based on the search query.

4 . The system of claim 3 , wherein the query embedding data is generated based at least in part by:

generating at least one query token based on the search query;

generating a plurality of entity tokens based on the plurality of processed entities;

concatenating the at least one query token with the plurality of entity tokens to generate concatenated token data based on at least one of: a start token, a separation token or an end token; and

generating a token embedding for each token in the concatenated token data such that: (1) one or more query token embeddings are generated for the search query and (2) a plurality of entity token embeddings are generated for the plurality of processed entities, wherein the query embedding data for the search query includes the one or more query token embeddings.

5 . The system of claim 4 , wherein the processor is further caused to:

send the query embedding data from the neural network to each of the plurality of task-specific networks via a model-specific network, wherein the model-specific network includes at least one of: a shared expert network, a task-specific expert network, a shared gate network or a task-specific gate network.

6 . The system of claim 1 , wherein during a training stage of the neural network and the plurality of task-specific networks:

only one of the plurality of task-specific networks is activated during each training step of a plurality of training steps of the training stage; and

the neural network is activated and trained during all training steps of the training stage.

7 . The system of claim 6 , wherein during each training step for training a task-specific network associated with a corresponding task, the processor is caused to:

retrieve a plurality of entities for each query in a training dataset using the entity retrieval model;

for each entity of the plurality of entities:

generate an entity representation based at least in part by averaging token embeddings of all entity tokens of the entity,

compress the entity representation into a representation score for the entity using a dense network,

determine a labelled score for the entity, wherein the labelled score is generated by the entity retrieval model based on historical user engagement data, and

generate a ranking loss for the entity based on the labelled score and the representation score;

generate a ranking loss function based on a combination of all ranking losses for the plurality of entities;

generate a task loss function for the corresponding task based on a cross-entropy loss and the training dataset;

generate, for the corresponding task, a combined loss function based on a weighted combination of the task loss function for the corresponding task and the ranking loss function, using weights specific to the corresponding task; and

train the neural network and the task-specific network based at least in part by minimizing the combined loss function.

8 . The system of claim 1 , wherein the plurality of tasks comprises at least one of:

a product type classification task to determine a product type for the search query;

a query catalog classification task to determine a query catalog for the search query;

a named entity recognition task to identify one or more named entities in the search query; or

a term weighting task to determine whether or not each token in the search query is to be excluded when retrieving a search result.

9 . A computer-implemented method, comprising:

obtaining a search query;

using an entity retrieval model:

encoding the search query into a normalized query embedding using a natural language model of the entity retrieval model,

identifying at least one normalized entity embedding among a plurality of normalized entity embeddings using the normalized query embedding,

determining at least one index associated with the at least one normalized entity embedding

localing and retrieving a plurality of entities from an entity database based on the at least one index

filtering the plurality of entities based on historical user engagement data to generate a plurality of filtered entities, and

process the plurality of filtered entities to generate at least one entity based on the search query;

generating, using a neural network, query embedding data based on the search query and the at least one entity;

generating, using a plurality of task-specific networks, task prediction data for a plurality of tasks based on the query embedding data, wherein each task of the plurality of tasks captures a different aspect of a user intent associated with the search query; and

generating at least one search result for the search query based on the task prediction data for the plurality of tasks.

10 . The computer-implemented method of claim 9 , wherein generating the at least one entity comprises:

based on a nearest neighbor search embedding performed using the normalized query embedding, identifying the at least one normalized entity embedding among the plurality of normalized entity embeddings.

11 . The computer-implemented method of claim 10 , wherein:

processing the plurality of filtered entities to generate a plurality of processed entities, wherein:

all of the plurality of processed entities have a same format,

each of the plurality of processed entities is associated with a corresponding score that indicates a degree of importance of the processed entity with respect to the search query, and

the at least one entity comprises the plurality of processed entities generated based on the search query.

12 . The computer-implemented method of claim 11 , wherein generating the query embedding data comprises:

generating at least one query token based on the search query;

generating a plurality of entity tokens based on the plurality of processed entities;

concatenating the at least one query token with the plurality of entity tokens to generate concatenated token data based on at least one of: a start token, a separation token or an end token; and

generating a token embedding for each token in the concatenated token data such that: (1) one or more query token embeddings are generated for the search query and (2) a plurality of entity token embeddings are generated for the plurality of processed entities, wherein the query embedding data for the search query includes the one or more query token embeddings.

13 . The computer-implemented method of claim 12 , further comprising:

sending the query embedding data from the neural network to each of the plurality of task-specific networks via a model-specific network, wherein the model-specific network includes at least one of: a shared expert network, a task-specific expert network,

a shared gate network or a task-specific gate network.

14 . The computer-implemented method of claim 9 , wherein during a training stage of the neural network and the plurality of task-specific networks:

only one of the plurality of task-specific networks is activated during each training step of a plurality of training steps of the training stage; and

the neural network is activated and trained during all training steps of the training stage.

15 . A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising:

obtaining a search query;

using an entity retrieval model:

encoding the search query into a normalized query embedding using a natural language model of the entity retrieval model,

identifying at least one normalized entity embedding among a plurality of normalized entity embeddings using the normalized query embedding,

determining at least one index associated with the at least one normalized entity embedding,

locating and retrieving a plurality of entities from an entity database based on the at least one index,

filtering the plurality of entities based on historical user engagement data to generate a plurality of filtered entities, and process the plurality of filtered entities to generate

at least one entity based on the search query;

generating, using a neural network, query embedding data based on the search query and the at least one entity;

generating, using a plurality of task-specific networks, task prediction data for a plurality of tasks based on the query embedding data, wherein each task of the plurality of tasks captures a different aspect of a user intent associated with the search query; and

generating at least one search result for the search query based on the task prediction data for the plurality of tasks.

16 . The non-transitory computer readable medium of claim 15 , wherein generating the operations further comprises:

based on a nearest neighbor search performed using the normalized query embedding identifying the at least one normalized entity embedding among the plurality of normalized entity embeddings.

17 . The non-transitory computer readable medium of claim 16 , wherein the operations further comprise:

processing the plurality of filtered entities to generate a plurality of processed entities, wherein:

all of the plurality of processed entities have a same format,

each of the plurality of processed entities is associated with a corresponding score that indicates a degree of importance of the processed entity with respect to the search query, and

the at least one entity comprises the plurality of processed entities generated based on the search query.

18 . The non-transitory computer readable medium of claim 17 , wherein generating the query embedding data comprises:

generating at least one query token based on the search query;

generating a plurality of entity tokens based on the plurality of processed entities;

concatenating the at least one query token with the plurality of entity tokens to generate concatenated token data based on at least one of: a start token, a separation token or an end token; and

generating a token embedding for each token in the concatenated token data such that: (1) one or more query token embeddings are generated for the search query and (2) a plurality of entity token embeddings are generated for the plurality of processed entities, wherein the query embedding data for the search query includes the one or more query token embeddings.

19 . The non-transitory computer readable medium of claim 18 , wherein the operations further comprise:

sending the query embedding data from the neural network to each of the plurality of task-specific networks via a model-specific network, wherein the model-specific network includes at least one of: a shared expert network, a task-specific expert network, a shared gate network or a task-specific gate network.

20 . The non-transitory computer readable medium of claim 15 , wherein during a training stage of the neural network and the plurality of task-specific networks:

only one of the plurality of task-specific networks is activated during each training step of a plurality of training steps of the training stage; and

the neural network is activated and trained during all training steps of the training stage.