Network model training method and apparatus, electronic apparatus and computer-readable storage medium
This disclosure discloses a network model training method and apparatus, an electronic apparatus and a computer-readable storage medium. The method includes: acquiring training data and inputting the training data into an initial model to obtain output data, wherein the initial model includes an embedding layer, the embedding layer is constructed based on preset network layer latency information, the preset network layer latency information includes network layer types and at least two types of latency data corresponding to each network layer type, and each type of latency data corresponds to different device types; inputting a current device type and a target network layer type of each target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type; calculating a target loss value based on the target latency data, the training data and the output data, and adjusting parameters of the initial model based on the target loss value; and obtaining a target model based on the initial model in response to a training completion condition is satisfied. By means of the method, the target model has a minimum latency when running on a device corresponding to the other device type.
1. A network model training method, comprising:
acquiring, by a processor, training data, and inputting the training data into an initial model to obtain output data,
wherein the initial model comprises an embedding layer of neural networks, the embedding layer is constructed based on preset network layer latency information, the preset network layer latency information comprises network layer types and at least two types of latency data corresponding to each network layer type, and each type of latency data corresponds to different device types, the network layer types refer to a type of each network layer in a network model;
inputting, by the processor, a current device type and a target network layer type of a target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type;
calculating, by the processor, a target loss value based on the target latency data, the training data and the output data, and adjusting parameters of the initial model based on the target loss value; and
obtaining, by the processor, a target model based on the initial model in response to a training completion condition is satisfied;
wherein the training data is audio data, the target model is an audio processing model, the audio processing model is configured to perform audio processing;
based on a target delay data, obtain the target loss value which reflects an execution delay of the initial model on other device types corresponding to devices;
based on the target loss value, adjust parameters of the initial model to ensure compatibility with devices corresponding to other device types, and to achieve a minimum delay for the target model when running on devices corresponding to other device types.
2. The network model training method as claimed in claim 1 , wherein a process of generating the preset network layer latency information comprises:
determining, by the processor, multiple network layers and multiple preset network models with each network layer of the multiple network layers;
training, by the processor, the multiple preset network models on devices corresponding to multiple device types to obtain first latency data corresponding to the multiple device types;
respectively obtaining, by the processor, second latency data corresponding to the each network layer of the multiple network layers based on the first latency data; and
generating, by the processor, the preset network layer latency information based on a corresponding relationship among the second latency data, the network layer type of the each network layer, and a device type of multiple device types.
3. The network model training method as claimed in claim 1 , wherein calculating, by the processor, the target loss value based on the target latency data, the training data and the output data comprises:
obtaining, by the processor, a precision loss value based on the training data and the output data; and
performing, by the processor, weighting summation based on the precision loss value and the target latency data to obtain the target loss value.
4. The network model training method as claimed in claim 1 , wherein the initial model is a hyperparameter network based on a neural network model training rule, and the initial model is the hyperparameter network constructed by utilizing a search space, a network architecture of the initial model corresponds to a target directed acyclic graph, the target directed acyclic graph has multiple directed edges, and each directed edge has multiple branches;
inputting, by the processor, the training data into the initial model to obtain the output data comprises:
determining, by the processor, a target parameter; and
determining, by the processor, activation branches corresponding to the each directed edge based on the target parameter, and processing the training data based on the activation branches to obtain the output data; and
adjusting, by the processor, parameters of the initial model based on the target loss value comprises:
updating, by the processor, the target parameter corresponding to the activation branches based on the target loss value, wherein a historical parameter of previous update being different from the target parameter in parameter type.
5. The network model training method as claimed in claim 4 , wherein determining, by the processor, the activation branch corresponding to the each directed edge based on the target parameter comprises:
randomly determining, by the processor, the activation branches in response to the target parameter is a weight parameter; and
selecting, by a processor, the activation branches based on a polynomial distributed sampling principle in response to the target parameter is an architecture parameter.
6. The network model training method as claimed in claim 5 , wherein updating the target parameter corresponding to the activation branches based on the target loss value comprises:
updating, by the processor, in response to the target parameter is the weight parameter, a weight parameter of the activation branches based on the target loss value through a random gradient descent method; and
calculating, by the processor, in response to the target parameter is the architecture parameter, an update parameter through utilizing the target loss value based on a preset update rule, and updating the architecture parameter of the activation branches based on the update parameter.
7. The network model training method as claimed in claim 4 , wherein obtaining a target model based on the initial model comprises:
calculating, by the processor, through the architecture parameter and the weight parameter, a branch weight corresponding to each branch; and
determining, by the processor, a highest branch weight of each directed edge, and clipping branches corresponding to non-highest branch weights in the initial model to obtain the target model.
8. The network model training method as claimed in claim 1 , wherein inputting, by the processor, the current device type and the target network layer type of the target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type comprises:
inputting, by the processor, the current device type, each target network layer type and target data into the embedding layer to obtain the target latency data corresponding to the target data, wherein the target data comprises an input data scale and/or a target device type.
9. The network model training method as claimed in claim 1 , wherein the embedding layer is constructed based on the preset network layer latency information comprises:
acquiring, by the processor, one-dimensional array based on the preset network layer latency information; and
building, by the processor, the embedding layer based on the one-dimensional array.
10. The network model training method as claimed in claim 4 , wherein the neural network model training rule is used to generate the initial model during network architecture search.
11. The network model training method as claimed in claim 4 , wherein determining, by the processor, activation branches corresponding to the each directed edge based on the target parameter comprises:
determining, by the processor, the activation branches corresponding to the each directed edge based on the target parameter and a binarization gate function.
12. The network model training method as claimed in claim 5 , wherein randomly determining, by the processor, the activation branches in response to the target parameter is the weight parameter comprises:
generating, by the processor, probability values by an architecture parameter of each branch;
randomly selecting, by the processor, a parameter from the probability values through a random number generator; and
determining, by the processor, the activation branches based on the parameter.
13. The network model training method as claimed in claim 5 , wherein selecting, by the processor, the activation branches based on the polynomial distributed sampling principle in response to the target parameter is the architecture parameter comprises:
selecting, by the processor, two branches from the multiple branches as the activation branches based on the polynomial distributed sampling principle.
14. The network model training method as claimed in claim 7 , wherein calculating, by the processor, through the architecture parameter and the weight parameter, the branch weight corresponding to each branch comprises:
calculating, by the processor, the branch weight corresponding to each branch based on the product of the architecture parameter and the weight parameter.
15. The network model training method as claimed in claim 8 , wherein the input data scale comprises a channel (C-channel), a height (H-height), a width (W-width) and a depth (D-depth).
16. The network model training method as claimed in claim 3 , wherein performing, by the processor, weighting summation based on the precision loss value and the target latency data to obtain the target loss value comprises:
performing, by the processor, weighting summation based on the precision loss value, the target latency data and a weight decay constant to obtain the target loss value.
17. An electronic apparatus, comprising a memory and a processor, wherein the memory is configured to store a computer program; and
the processor is configured to execute the computer program, cause the processor to:
acquire training data, and input the training data into an initial model to obtain output data,
wherein the initial model comprises an embedding layer of neural networks, the embedding layer is constructed based on preset network layer latency information, the preset network layer latency information comprises network layer types and at least two types of latency data corresponding to each network layer type, and each type of latency data corresponds to different device types, the network layer types refer to a type of each network layer in a network model;
input a current device type and a target network layer type of a target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type;
calculate a target loss value based on the target latency data, the training data and the output data, and adjust parameters of the initial model based on the target loss value; and
obtain a target model based on the initial model in response to a training completion condition is satisfied;
wherein the training data is audio data, the target model is an audio processing model, the audio processing model is configured to perform audio processing;
based on a target delay data, obtain the target loss value which reflects an execution delay of the initial model on other device types corresponding to devices;
based on the target loss value, adjust parameters of the initial model to ensure compatibility with devices corresponding to other device types, and to achieve a minimum delay for the target model when running on devices corresponding to other device types.
18. The electronic apparatus as claimed in claim 17 , wherein the processor is configured to execute the computer program, cause the processor to:
determine multiple network layers and multiple preset network models with each network layer of the multiple network layers;
train the multiple preset network models on devices corresponding to multiple device types to obtain first latency data corresponding to the multiple device types;
respectively obtain second latency data corresponding to the each network layer of the multiple network layers based on the first latency data; and
generate the preset network layer latency information based on a corresponding relationship among the second latency data, the network layer type of the each network layer, and a device type of multiple device types.
19. The electronic apparatus as claimed in claim 17 , wherein the processor is configured to execute the computer program, cause the processor to:
obtain a precision loss value based on the training data and the output data; and
perform weighting summation based on the precision loss value and the target latency data to obtain the target loss value.
20. A non-transitory computer-readable storage medium, configured to store a computer program, wherein the computer program, when executed by a processor, cause the processor to:
acquire training data, and input the training data into an initial model to obtain output data, wherein the initial model comprises an embedding layer of neural networks, the embedding layer is constructed based on preset network layer latency information, the preset network layer latency information comprises network layer types and at least two types of latency data corresponding to each network layer type, and each type of latency data corresponds to different device types;
input a current device type and a target network layer type of a target network layer in the initial model into the embedding layer to obtain target latency data corresponding to other device type;
calculate a target loss value based on the target latency data, the training data and the output data, and adjust parameters of the initial model based on the target loss value; and
obtain a target model based on the initial model in response to a training completion condition is satisfied;
wherein the training data is audio data, the target model is an audio processing model, the audio processing model is configured to perform audio processing;
based on a target delay data, obtain the target loss value which reflects an execution delay of the initial model on other device types corresponding to devices;
based on the target loss value, adjust parameters of the initial model to ensure compatibility with devices corresponding to other device types, and to achieve a minimum delay for the target model when running on devices corresponding to other device types.