Hierarchical multi-level model distillation

1 . One or more non-transitory computer-readable media storing instructions thereon for distilling machine learning models, wherein the instructions cause one or more processors to:

determine a hierarchy for generating a plurality of specialized generative models from a large language model,

wherein the hierarchy comprises the plurality of specialized generative models,

wherein one or more of the plurality of specialized generative models are associated with a category of a plurality of categories, and

wherein each specialized generative model of the plurality of specialized generative models in the hierarchy is associated with a different complexity level of a plurality of complexity levels for performing tasks with different complexities;

train, using the large language model and based on the category, the plurality of specialized generative models,

wherein the plurality of specialized generative models is trained to provide responses to requests associated with the category;

prune, based on complexity levels of respective ones of the plurality of specialized generative models, each specialized generative model of the plurality of specialized generative models,

wherein each specialized generative model, when pruned, is enabled to respond to different requests of different complexity levels based on a respective complexity level of each specialized generative model;

quantize, based on accuracy levels of respective ones of the specialized generative models, each specialized generative model from a common precision factor to a corresponding precision factor,

wherein a given precision factor is associated with a corresponding complexity level of the plurality of complexity levels;

determine, based on the corresponding complexity level of the plurality of complexity levels, a corresponding location of a plurality of locations for each pruned specialized generative model; and

transmit one or more specialized generative models to the corresponding location.

2 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions further cause the one or more processors to:

receive a request to generate the plurality of specialized generative models, wherein the request comprises a natural language concept;

generate, using the large language model, the category of a plurality of categories,

wherein the category of a plurality of categories is associated with the natural language concept; and

determine the hierarchy based on the category.

3 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions for training the plurality of specialized generative models further cause the one or more processors to:

generate, based on the category, a plurality of requests for the large language model to obtain a plurality of responses from the large language model, wherein each request in the plurality of requests is related to the category; and

input the plurality of requests and the plurality of responses into a corresponding training routine of each specialized generative model of the plurality of specialized generative models,

wherein each training routine trains each specialized generative model using the plurality of requests and the plurality of responses.

4 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions for pruning each specialized generative model of the plurality of specialized generative models further cause the one or more processors to:

determine, for each portion of a plurality of portions of a first specialized generative model of the plurality of specialized generative models, a matching metric,

wherein the matching metric indicates how well a corresponding portion of the plurality of portions matches the category, and

wherein the plurality of portions comprises one of neurons, attention heads, or layers; and

prune one or more portions of the first specialized generative model where the matching metric does not meet a match threshold.

5 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions for pruning each specialized generative model of the plurality of specialized generative models further cause the one or more processors to:

determine, a first specialized generative model of the plurality of specialized generative models, a plurality of tasks that the first specialized generative model is to be enabled to complete;

identify, one or more portions of a plurality of portions of the first specialized generative model that are not required for completing the plurality of tasks; and

prune the one or more portions of the first specialized generative model that are not required for completing the plurality of tasks.

6 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions for quantizing each specialized generative model from the common precision factor to the corresponding precision factor further cause the one or more processors to:

determine, for each specialized generative model of the plurality of specialized generative models using the respective complexity level of each specialized generative model, the corresponding precision factor,

wherein each corresponding precision factor comprises a number of bits representing model parameters, the model parameters comprising weights and activations; and

update, for each specialized generative model, the common precision factor with the corresponding precision factor,

wherein the corresponding precision factor is different for each specialized generative model of the plurality of specialized generative models.

7 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions further cause the one or more processors to:

receive a natural language request;

determine, using a routing generative model, a request complexity level associated with the natural language request and that the natural language request is associated with the category; and

route, based on the request complexity level, the natural language request to a first specialized generative model of the plurality of specialized generative models,

wherein the first specialized generative model is associated with a complexity level that matches the request complexity level.

8 . The one or more non-transitory computer-readable media of claim 1 , wherein the instructions further cause the one or more processors to:

receive a natural language query comprising the category; and

determine, based on the natural language query that the category comprises one or more of model size indicating a size parameter for one or more of the plurality of specialized generative models, a domain parameter indicating one or more topics for training the plurality of specialized generative models, or model location for one or more of the plurality of specialized generative models.

9 . A method for distilling machine learning models, the method comprising:

determining a hierarchy for generating a plurality of specialized generative models from a large language model,

wherein the hierarchy comprises the plurality of specialized generative models,

wherein one or more of the plurality of specialized generative models are associated with a category of a plurality of categories, and

wherein each specialized generative model of the plurality of specialized generative models in the hierarchy is associated with a different complexity level of a plurality of complexity levels for performing tasks with different complexities;

training, using the large language model and based on the category, the plurality of specialized generative models,

wherein the plurality of specialized generative models is trained to provide responses to requests associated with the category;

pruning, based on complexity levels of respective ones of the plurality of specialized generative models, each specialized generative model of the plurality of specialized generative models,

wherein each specialized generative model, when pruned, is enabled to respond to different requests of different complexity levels based on a respective complexity level of each specialized generative model;

quantizing, based on accuracy levels of respective ones of the specialized generative models, each specialized generative model from a common precision factor to a corresponding precision factor,

wherein a given precision factor is associated with a corresponding complexity level of the plurality of complexity levels;

determining, based on the corresponding complexity level of the plurality of complexity levels, a corresponding location of a plurality of locations for each pruned specialized generative model; and

transmitting one or more specialized generative models to the corresponding location.

10 . The method of claim 9 , further comprising:

receiving a request to generate the plurality of specialized generative models, wherein the request comprises a natural language concept;

generating, using the large language model, the category of a plurality of categories,

wherein the category of a plurality of categories is associated with the natural language concept; and

determining the hierarchy based on the category.

11 . The method of claim 9 , wherein training the plurality of specialized generative models further comprises:

generating, based on the category, a plurality of requests for the large language model to obtain a plurality of responses from the large language model, wherein each request in the plurality of requests is related to the category; and

inputting the plurality of requests and the plurality of responses into a corresponding training routine of each specialized generative model of the plurality of specialized generative models,

wherein each training routine trains each specialized generative model using the plurality of requests and the plurality of responses.

12 . The method of claim 9 , wherein pruning each specialized generative model of the plurality of specialized generative models further comprises:

determining, for each portion of a plurality of portions of a first specialized generative model of the plurality of specialized generative models, a matching metric,

wherein the matching metric indicates how well a corresponding portion of the plurality of portions matches the category, and

wherein the plurality of portions comprises one of neurons, attention heads, or layers; and

pruning one or more portions of the first specialized generative model where the matching metric does not meet a match threshold.

13 . The method of claim 9 , wherein pruning each specialized generative model of the plurality of specialized generative models further comprises:

determining, a first specialized generative model of the plurality of specialized generative models, a plurality of tasks that the first specialized generative model is to be enabled to complete;

identifying, one or more portions of a plurality of portions of the first specialized generative model that are not required for completing the plurality of tasks; and

pruning the one or more portions of the first specialized generative model that are not required for completing the plurality of tasks.

14 . The method of claim 9 , wherein quantizing each specialized generative model from the common precision factor to the corresponding precision factor further comprises:

determining, for each specialized generative model of the plurality of specialized generative models using the respective complexity level of each specialized generative model, the corresponding precision factor,

wherein each corresponding precision factor comprises a number of bits representing model parameters, the model parameters comprising weights and activations; and

updating, for each specialized generative model, the common precision factor with the corresponding precision factor,

wherein the corresponding precision factor is different for each specialized generative model of the plurality of specialized generative models.

15 . The method of claim 9 , further comprising:

receiving a natural language request;

determining, using a routing generative model, a request complexity level associated with the natural language request and that the natural language request is associated with the category; and

routing, based on the request complexity level, the natural language request to a first specialized generative model of the plurality of specialized generative models,

wherein the first specialized generative model is associated with a complexity level that matches the request complexity level.

16 . The method of claim 9 , further comprising:

receiving a natural language query comprising the category; and

determining, based on the natural language query that the category comprises one or more of model size indicating a size parameter for one or more of the plurality of specialized generative models, a domain parameter indicating one or more topics for training the plurality of specialized generative models, or model location for one or more of the plurality of specialized generative models.

17 . A system comprising:

one or more processors; and

one or more non-transitory, computer-readable media comprising instructions that,

when executed by the one or more processors, cause the one or more processors to:

determine a hierarchy for generating a plurality of specialized generative models from a large language model, wherein the hierarchy comprises the plurality of specialized generative models,

wherein one or more of the plurality of specialized generative models are associated with a category of a plurality of categories, and

wherein each specialized generative model of the plurality of specialized generative models in the hierarchy is associated with a different complexity level of a plurality of complexity levels for performing tasks with different complexities;

train, using the large language model and based on the category, the plurality of specialized generative models,

wherein the plurality of specialized generative models is trained to provide responses to requests associated with the category;

prune, based on complexity levels of respective ones of the plurality of specialized generative models, each specialized generative model of the plurality of specialized generative models,

wherein each specialized generative model, when pruned, is enabled to respond to different requests of different complexity based on a respective complexity level of each specialized generative model;

quantize, based on accuracy levels of respective ones of the specialized generative models, each specialized generative model from a common precision factor to a corresponding precision factor,

wherein a given precision factor is associated with a corresponding complexity level of the plurality of complexity levels;

determine, based on the corresponding complexity level of the plurality of complexity levels, a corresponding location of a plurality of locations for each pruned specialized generative model; and

transmit one or more specialized generative models to the corresponding location.

18 . The system of claim 17 , wherein the instructions further cause the one or more processors to:

receive a request to generate the plurality of specialized generative models, wherein the request comprises a natural language concept;

generate, using the large language model, the category of a plurality of categories,

wherein the category of a plurality of categories is associated with the natural language concept; and

determine the hierarchy based on the category.

19 . The system of claim 17 , wherein the instructions for training the plurality of specialized generative models further cause the one or more processors to:

generate, based on the category, a plurality of requests for the large language model to obtain a plurality of responses from the large language model, wherein each request in the plurality of requests is related to the category; and

input the plurality of requests and the plurality of responses into a corresponding training routine of each specialized generative model of the plurality of specialized generative models,

wherein each training routine trains each specialized generative model using the plurality of requests and the plurality of responses.

20 . The system of claim 17 , wherein the instructions for pruning each specialized generative model of the plurality of specialized generative models further cause the one or more processors to:

determine, for each portion of a plurality of portions of a first specialized generative model of the plurality of specialized generative models, a matching metric,

wherein the matching metric indicates how well a corresponding portion of the plurality of portions matches the category, and

wherein the plurality of portions comprises one of neurons, attention heads, or layers; and

prune one or more portions of the first specialized generative model where the matching metric does not meet a match threshold.