Model training method, machine translation method, computer device, and storage medium
Embodiments of this application disclose a neural network model training method, a machine translation method, a computer device, and a storage medium. The method includes: obtaining a training sample set including a training sample and a standard tag vector corresponding to the training sample; inputting the training sample into a neural network model including a plurality of attention networks to obtain a feature fusion vector; obtaining a predicted tag vector according to the feature fusion vector, and performing adjustment on a model parameter of the neural network model until a convergence condition is met to obtain a target neural network model.
1. A neural network model training method, comprising:
obtaining a training sample set, the training sample set comprising a training sample and a standard tag vector corresponding to the training sample;
inputting the training sample into a plurality of attention networks of a neural network model, and performing nonlinear transformation on respective output vectors of the plurality of attention networks by using the neural network model;
obtaining a feature fusion vector corresponding to the plurality of attention networks;
obtaining a predicted tag vector outputted by the neural network model according to the feature fusion vector, the predicted tag vector corresponding to the training sample; and
performing iteration adjustment on a model parameter of the neural network model according to a result of a comparison between the predicted tag vector and the standard tag vector corresponding to the training sample; and
in response to the neural network model meeting a convergence condition, identifying a target neural network model,
wherein performing the nonlinear transformation on respective output vectors of the plurality of attention networks comprises:
splicing the respective output vectors of the plurality of attention networks to obtain a first output vector, wherein the first output vector is a vector spliced from the output vectors of the plurality of attention networks;
performing mapping on the first output vector with a first mapping parameter to obtain a second mapped output vector,
performing mapping on the first output vector with a second mapping parameter to obtain a third mapped output vector, the second mapping parameter being different from the first mapping parameter;
performing element-wise multiplication on the second mapped output vector and the third mapped output vector to obtain a fourth vector; and
obtaining the feature fusion vector corresponding to the plurality of attention networks based on the fourth vector.
2. The method according to claim 1 , wherein the obtaining the feature fusion vector corresponding to the plurality of attention networks based on the fourth vector comprises:
performing linear mapping on the fourth vector to obtain the feature fusion vector corresponding to the plurality of attention networks.
3. The method according to claim 1 , wherein the obtaining the feature fusion vector corresponding to the plurality of attention networks based on the fourth vector comprises:
fusing the fourth vector and the first output vector to obtain a fifth vector; and
performing linear mapping on the fifth vector to obtain the feature fusion vector corresponding to the plurality of attention networks.
4. The method according to claim 3 , wherein the fusing the fourth vector and the first output vector to obtain a fifth vector comprises:
sequentially splicing the fourth vector and the first output vector to obtain the fifth vector; or
performing element-wise multiplication on the fourth vector and the first output vector to obtain the fifth vector; or
performing element-wise addition on the fourth vector and the first output vector to obtain the fifth vector.
5. The method according to claim 1 , wherein performing iteration adjustment on a model parameter of the neural network model according to a result of a comparison between the predicted tag vector and the standard tag vector corresponding to the training sample; and in response to the neural network model meeting a convergence condition, identifying a target neural network model comprises:
determining a loss function value according to the result of comparison between the predicted tag vector corresponding to the training sample and the standard tag vector;
performing the iteration adjustment on the model parameter of the neural network model when the loss function value is greater than a target threshold; and
when the loss function value is less than or equal to the target threshold, determining that the current neural network model meets the convergence condition, and identifying the current neural network model as the target neural network model.
6. The method according to claim 1 , wherein the neural network model is a machine translation model and comprises an encoder and a decoder;
the encoder comprising N cascaded first-type network blocks, and each first-type network block comprising one multi-head attention network layer and one feedforward neural network layer that are cascaded; and
the decoder comprising N cascaded second-type network blocks, each second-type network block comprising two multi-head attention network layers and one feedforward neural network layer that are cascaded, and N being a positive integer greater than or equal to 1.
7. The method according to claim 1 , wherein the training sample is at least one of a to-be-translated text, a to-be-translated video, or a to-be-translated audio, and a standard tag corresponding to the training sample is a standard translated text.
8. A machine translation method, comprising:
obtaining to-be-translated content;
obtaining a translation result corresponding to the to-be-translated content by using a machine translation model, the machine translation model being obtained by performing training according to the model training method comprising:
obtaining a training sample set, the training sample set comprising a training sample and a standard tag vector corresponding to the training sample;
inputting the training sample into a plurality of attention networks of the machine translation model, and performing nonlinear transformation on respective output vectors of the plurality of attention networks;
obtaining a feature fusion vector corresponding to the plurality of attention networks;
obtaining a predicted tag vector outputted by the machine translation model according to the feature fusion vector, the predicted tag vector corresponding to the training sample;
performing iteration adjustment on a model parameter of the machine translation model according to a result of a comparison between the predicted tag vector and the standard tag vector corresponding to the training sample; and
in response to the neural network model meeting a convergence condition, identifying the machine translation model; and
displaying the translation result,
wherein performing the nonlinear transformation on respective output vectors of the plurality of attention networks comprises:
splicing the respective output vectors of the plurality of attention networks to obtain a first output vector, wherein the first output vector is a vector spliced from the output vectors of the plurality of attention networks;
performing mapping on the first output vector with a first mapping parameter to obtain a second mapped output vector;
performing mapping on the first output vector with a second mapping parameter to obtain a third mapped output vector, the second mapping parameter being different from the first mapping parameter;
performing element-wise multiplication on the second mapped output vector and the third mapped output vector to obtain a fourth vector; and
obtaining the feature fusion vector corresponding to the plurality of attention networks based on the fourth vector.
9. The method according to claim 8 , wherein the training sample is at least one of a to-be-translated text, a to-be-translated video, or a to-be-translated audio, and a standard tag corresponding to the training sample is a standard translated text.
10. A computer device, comprising a processor and a memory;
the memory being configured to store program code executable by the processor; and
the processor being configured to execute the program code and perform a plurality of operations comprising:
obtaining a training sample set, the training sample set comprising a training sample and a standard tag vector corresponding to the training sample;
inputting the training sample in the training sample set into a plurality of attention networks of a neural network model, and performing nonlinear transformation on respective output vectors of the plurality of attention networks by using the neural network model, to obtain a feature fusion vector corresponding to the plurality of attention networks;
obtaining a predicted tag vector outputted by the neural network model according to the feature fusion vector, the predicted tag vector corresponding to the training sample; and
performing iteration adjustment on a model parameter of the neural network model according to a result of comparison between the predicted tag vector corresponding to the training sample and the standard tag vector;
in response to the neural network model meeting a convergence condition, identifying a target neural network model,
wherein performing the nonlinear transformation on respective output vectors of the plurality of attention networks comprises:
splicing the respective output vectors of the plurality of attention networks to obtain a first output vector, wherein the first output vector is a vector spliced from the output vectors of the plurality of attention networks;
performing mapping on the first output vector with a first mapping parameter to obtain a second mapped output vector,
performing mapping on the first output vector with a second mapping parameter to obtain a third mapped output vector, the second mapping parameter being different from the first mapping parameter;
performing element-wise multiplication on the second mapped output vector and the third mapped output vector to obtain a fourth vector; and
obtaining the feature fusion vector corresponding to the plurality of attention networks based on the fourth vector.
11. The computer device according to claim 10 , wherein the processor is configured to execute the program code and perform a plurality of operations further comprising:
performing linear mapping on the fourth vector to obtain the feature fusion vector corresponding to the plurality of attention networks.
12. The computer device according to claim 10 , wherein the processor is configured to execute the program code and perform a plurality of operations further comprising:
fusing the fourth vector and the first output vector to obtain a fifth vector; and
performing linear mapping on the fifth vector to obtain the feature fusion vector corresponding to the plurality of attention networks.
13. The computer device according to claim 12 , wherein the processor is configured to execute the program code and perform a plurality of operations further comprising:
sequentially splicing the fourth vector and the first output vector to obtain the fifth vector; or
performing element-wise multiplication on the fourth vector and the first output vector to obtain the fifth vector; or
performing element-wise addition on the fourth vector and the first output vector to obtain the fifth vector.
14. The computer device according to claim 10 , wherein the processor is configured to execute the program code and perform a plurality of operations further comprising:
determining a loss function value according to the result of comparison between the predicted tag vector corresponding to the training sample and the standard tag vector;
performing the iteration adjustment on the model parameter of the neural network model when the loss function value is greater than a target threshold; and
when the loss function value is less than or equal to the target threshold, determining that the current neural network model meets the convergence condition, and using the current neural network model as the target neural network model.
15. The computer device according to claim 10 , wherein the neural network model is a machine translation model and comprises an encoder and a decoder;
the encoder comprising N cascaded first-type network blocks, and each first-type network block comprising one multi-head attention network layer and one feedforward neural network layer that are cascaded; and
the decoder comprising N cascaded second-type network blocks, each second-type network block comprising two multi-head attention network layers and one feedforward neural network layer that are cascaded, and N being a positive integer greater than or equal to 1.
16. A non-transitory computer readable storage medium storing computer instructions that, when being executed by a processor, cause the processor to perform the method according to claim 1 .