LEARNING APPARATUS, METHOD AND PROGRAM
A learning device includes: spectrogram generation circuitry 1 that generates a spectrogram from an input sound signal; patch generation circuitry 2 that divides the generated spectrogram to generate a plurality of patches; a mask processing circuitry 3 that selects some patches as masked patches; reconstruction circuitry 4 that obtains a plurality of reconstructed patches by reconstructing the plurality of patches by processing of an encoder and a decoder in a transformer serving as a deep learning model by using visible patches other than the masked patches among the plurality of patches and mask tokens corresponding to the masked patches; and parameter update circuitry 5 that updates a parameter of the encoder and a parameter of the decoder such that the masked patches approach reconstructed patches corresponding to the masked patches. The number of layers of the decoder is three or more.
1 . A learning device comprising:
spectrogram generation circuitry that generates a spectrogram from an input sound signal;
patch generation circuitry that divides the generated spectrogram to generate a plurality of patches;
mask processing circuitry that selects some patches from among the plurality of patches as masked patches;
reconstruction circuitry that obtains a plurality of reconstructed patches by reconstructing the plurality of patches by processing of an encoder and a decoder in a transformer serving as a deep learning model by using visible patches other than the masked patches among the plurality of patches and mask tokens corresponding to the masked patches; and
parameter update circuitry that updates a parameter of the encoder and a parameter of the decoder such that the masked patches approach reconstructed patches corresponding to the masked patches among the plurality of reconstructed patches, wherein
the number of layers of the decoder is three or more.
2 . The learning device according to claim 1 , wherein the reconstruction circuitry includes (1) encoder processing circuitry that inputs the visible patches to the encoder to obtain an encoding result, and (2) decoder processing circuitry that inputs the encoding result and the mask tokens to the decoder to obtain, as a decoding result, the plurality of reconstructed patches generated by reconstructing the plurality of patches.
3 . The learning device according to claim 1 , wherein the spectrogram has a time length longer than a predetermined time length.
4 . The learning device according to claim 1 , wherein the patch has a size smaller than a predetermined size.
5 . A learning method comprising:
a spectrogram generation step of causing spectrogram generation circuitry to generate a spectrogram from an input sound signal;
a patch generation step of causing patch generation circuitry to divide the generated spectrogram to generate a plurality of patches;
a mask processing step of causing mask processing circuitry to select some patches from among the plurality of patches as masked patches;
a reconstruction step of causing reconstruction circuitry to obtain a plurality of reconstructed patches by reconstructing the plurality of patches by processing of an encoder and a decoder in a transformer serving as a deep learning model by using visible patches other than the masked patches among the plurality of patches and mask tokens corresponding to the masked patches; and
a parameter update step of causing parameter update circuitry to update a parameter of the encoder and a parameter of the decoder such that the masked patches approach reconstructed patches corresponding to the masked patches among the plurality of reconstructed patches, wherein
the number of layers of the decoder is three or more.
6 . A non-transitory computer readable medium that stores a program for causing a computer to perform each step of the learning method according to claim 5 .