Conversion model learning apparatus, conversion model generation apparatus, conversion apparatus, conversion method and program
A mask unit generates a missing primary feature quantity sequence in which a part of a primary feature quantity sequence, which is an acoustic feature quantity sequence of a primary voice signal, on a time axis is masked. A conversion unit generates a simulated secondary feature quantity sequence in which a secondary feature quantity sequence which is an acoustic feature quantity sequence of a secondary voice signal having a time-frequency structure corresponding to a primary voice signal by inputting a missing primary feature quantity sequence to a conversion model that is a machine learning model. A calculation unit calculates a learning reference value which becomes higher as a time frequency structure of a simulated secondary feature quantity sequence is closer to a time frequency structure of a secondary feature quantity sequence. An update unit updates parameters of a conversion model on the basis of a learning reference value.
1 . A conversion model learning apparatus comprising:
a mask configured to generate a missing primary feature quantity sequence in which a part of a primary feature quantity sequence, which is an acoustic feature quantity sequence of a primary voice signal, on a time axis is masked;
a converter configured to generate a simulated secondary feature quantity sequence by inputting the missing primary feature quantity sequence to a conversion model that is a machine learning model, where the simulated secondary feature quantity sequence is simulating a secondary feature quantity sequence and the secondary feature quantity sequence is an acoustic feature quantity sequence of a secondary voice signal having a time-frequency structure corresponding to the primary voice signal;
a calculator configured to calculate a learning reference value which becomes higher as a time-frequency structure of the simulated secondary feature quantity sequence and the time-frequency structure of the secondary feature quantity sequence become closer to each other; and
an updater configured to update parameters of the conversion model on the basis of the learning reference value.
2 . The conversion model learning apparatus according to claim 1 , comprising:
an inverse converter configured to generate a reproduced primary feature quantity sequence which reproduces the acoustic feature quantity sequence of the primary voice signal by inputting the simulated secondary feature quantity sequence to an inverse conversion model that is the machine learning model, wherein
the calculator calculates the learning reference value on the basis of similarity between the reproduced primary feature quantity sequence and the primary feature quantity sequence.
3 . The conversion model learning apparatus according to claim 2 , wherein
the inverse conversion model and the conversion model are the same machine learning model,
the conversion model is a model in which the acoustic feature quantity sequence and a parameter indicating a type of voice are input and the acoustic feature quantity sequence related to the type indicated by the parameters is output,
the converter generates the simulated secondary feature quantity sequence by inputting the missing primary feature quantity sequence and a parameter indicating a type of the secondary voice signal to the conversion model, and
the inverse converter generates the reproduced primary feature quantity sequence by inputting the simulated secondary feature quantity sequence and a parameter indicating a type of the primary voice signal to the conversion model.
4 . The conversion model learning apparatus according to claim 1 , wherein
the conversion model is a model in which the acoustic feature quantity sequence and a parameter indicating a type of voice are input and the acoustic feature quantity sequence related to the type indicated by the parameter is output, and
the converter generates the simulated secondary feature quantity sequence by inputting the missing primary feature quantity sequence and a parameter indicating a type of the secondary voice signal to the conversion model.
5 . The conversion model learning apparatus according to claim 1 , wherein
the calculator calculates the learning reference value on the basis of a distance between the simulated secondary feature quantity sequence and the secondary feature quantity sequence that is the acoustic feature quantity sequence of the secondary voice signal.
6 . The conversion model learning apparatus according to claim 1 , wherein
the conversion model is a model in which the acoustic feature quantity sequence and mask information of the acoustic feature quantity sequence are input.
7 . A conversion model generation method for generating a conversion model having a parameter used for calculation for generating a simulated secondary feature quantity sequence from a primary feature quantity sequence that is an acoustic feature quantity sequence of a primary voice signal, where the simulated secondary feature quantity sequence is simulating a secondary feature quantity sequence and the secondary feature quantity sequence is an acoustic feature quantity sequence of a secondary voice signal having a time-frequency structure corresponding to the primary voice signal, the conversion model generation method comprising:
generating a missing primary feature quantity sequence in which a part of a primary feature quantity sequence, which is an acoustic feature quantity sequence of a primary voice signal, on a time axis is masked;
generating the simulated secondary feature quantity sequence by inputting the missing primary feature quantity sequence to the conversion model that is a machine learning model;
calculating a learning reference value which becomes higher as a time-frequency structure of the simulated secondary feature quantity sequence and the time-frequency structure of the secondary feature quantity sequence become closer to each other; and
generating a learned conversion model by updating parameters of the conversion model on the basis of the learning reference value.
8 . A conversion apparatus comprising:
an acquirer configured to acquire a primary feature quantity sequence which is an acoustic feature quantity sequence of a primary voice signal;
a converter configured to generate a simulated secondary feature quantity sequence by inputting the primary feature quantity sequence to a conversion model which is generated by a conversion model generation method, where the simulated secondary feature quantity sequence is simulating a secondary feature quantity sequence and the secondary feature quantity sequence is an acoustic feature quantity sequence of a secondary voice signal having a time-frequency structure corresponding to the primary voice signal; and
an outputter configured to output the simulated secondary feature quantity sequence, and
wherein the conversion model generation method includes:
generating a missing primary feature quantity sequence in which a part of the primary feature quantity sequence on a time axis is masked;
generating the simulated secondary feature quantity sequence by inputting the missing primary feature quantity sequence to the conversion model that is a machine learning model;
calculating a learning reference value which becomes higher as a time-frequency structure of the simulated secondary feature quantity sequence and the time-frequency structure of the secondary feature quantity sequence become closer to each other; and
generating a learned conversion model by updating parameters of the conversion model on the basis of the learning reference value.