Learning apparatus, conversion apparatus, learning method and program
An aspect of the present invention provides a learning device including: a neural network that converts an acoustic time series that is a time series representing a sound into feature data expressed in a predetermined format required by a downstream task; and an update unit that updates the neural network on the basis of an execution result of the downstream task using the feature data, in which the neural network includes: a feature extraction unit that converts an input acoustic time series into an intermediate feature tensor that is a third-order tensor indicating features of the acoustic time series and is a tensor having time, frequency, and channel; and an intermediate network that executes processing of converting a representation of the intermediate feature tensor into a representation of a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel, and processing of acquiring, for each direct product amount of the second-order tensor, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data.
1 . A learning device comprising:
a processor includes a neural network that converts an acoustic time series that is a time series representing a sound into feature data that is information regarding the sound obtained on the basis of the acoustic time series and is information expressed in a predetermined format required by a downstream task; and
a storage medium having computer program instructions stored thereon, wherein the computer program instruction, when executed by the processor, perform processing of:
updating the neural network on the basis of an execution result of the downstream task using the feature data,
in which the neural network:
converts an input acoustic time series into an intermediate feature tensor that is a third-order tensor that indicates features of the acoustic time series and has time, frequency, and channel; and
the neural network includes
an intermediate network that executes non-equivalent second-order tensor processing of converting a representation of the intermediate feature tensor into a representation of a planar tensor that is a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel, and vectorization processing of acquiring, for each direct product amount of the planar tensor, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data.
2 . The learning device according to claim 1 , wherein
the statistic is either an average value or a maximum value.
3 . The learning device according to claim 1 , wherein
the feature data is a one-dimensional vector of a vector sum of a one-dimensional vector indicating, for each direct product amount of the planar tensor, an average value in the time axis direction of each direct product amount and a one-dimensional vector indicating, for each direct product amount of the planar tensor, a maximum value in the time axis direction of each direct product amount.
4 . The learning device according to claim 1 , wherein
a non-equivalent second-order tensor processing execution unit updates the planar tensor with a mathematical model of machine learning.
5 . A conversion device comprising:
a processor and
a storage medium having computer program instructions stored thereon, wherein the computer program instruction, when executed by the processor, perform processing of:
acquiring an acoustic time series that is a time series representing a sound; and
converting, into feature data, the acoustic time series acquired by the acquiring using a trained neural network obtained by learning by a learning device, a learning device comprising: a processor includes a neural network that converts an acoustic time series that is a time series representing a sound into feature data that is information regarding the sound obtained on the basis of the acoustic time series and is information expressed in a predetermined format required by a downstream task; and a storage medium having computer program instructions stored thereon, wherein the computer program instruction, when executed by the processor, perform processing of: updating the neural network on the basis of an execution result of the downstream task using the feature data, in which the neural network converts an input acoustic time series into an intermediate feature tensor that is a third-order tensor that indicates features of the acoustic time series and has time, frequency, and channel; and the neural network includes an intermediate network that executes non-equivalent second-order tensor processing of converting a representation of the intermediate feature tensor into a representation of a planar tensor that is a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel, and vectorization processing of acquiring, for each direct product amount of the planar tensor, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data.
6 . A learning method executed by a learning device comprising: a processor includes a neural network that converts an acoustic time series that is a time series representing a sound into feature data that is information regarding the sound obtained on the basis of the acoustic time series and is information expressed in a predetermined format required by a downstream task; and a storage medium having computer program instructions stored thereon, wherein the computer program instruction, when executed by the processor, perform processing of: updating the neural network on the basis of an execution result of the downstream task using the feature data, in which the neural network converts an input acoustic time series into an intermediate feature tensor that is a third-order tensor that indicates features of the acoustic time series and has time, frequency, and channel; and the neural network includes an intermediate network that executes non-equivalent second-order tensor processing of converting a representation of the intermediate feature tensor into a representation of a planar tensor that is a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel, and vectorization processing of acquiring, for each direct product amount of the planar tensor, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data,
the learning method comprising:
converting an input acoustic time series into an intermediate feature tensor that is a third-order tensor that indicates features of the acoustic time series and has time, frequency, and channel;
converting a representation of the intermediate feature tensor into a representation of a planar tensor that is a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel; and
acquiring, for each direct product amount of the planar tensor, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data.
7 . A non-transitory computer readable medium which stores a program for causing a computer to function as the learning device according to claim 1 .
8 . A non-transitory computer readable medium which stores a program for causing a computer to function as the conversion device according to claim 5 .
9 . A learning device comprising:
a processor includes a neural network that converts an acoustic time series that is a time series representing a sound into feature data that is information regarding the sound obtained on the basis of the acoustic time series and is information expressed in a predetermined format required by a downstream task; and
a storage medium having computer program instructions stored thereon, wherein the computer program instruction, when executed by the processor, perform processing of:
updating the neural network on the basis of an execution result of the downstream task using the feature data,
in which the neural network:
acquires, on the basis of an input acoustic time series, an intermediate feature tensor that is a third-order tensor that indicates features of the acoustic time series and has time, frequency, and channel; and
the neural network includes an intermediate network that executes non-equivalent second-order tensor processing of converting a representation of the intermediate feature tensor into a representation of a planar tensor that is a second-order tensor having time and direct product amounts that are amounts indicating a direct product of frequency and channel.
10 . The learning device according to claim 9 , wherein
a plurality of intermediate feature tensors are acquired by the acquiring, and
the intermediate network acquires the planar tensor for each one of a plurality of the intermediate feature tensors.
11 . The learning device according to claim 10 , wherein
the intermediate network executes:
combining processing of combining a plurality of the planar tensors; and
vectorization processing of acquiring, for each direct product amount of a combined planar tensor obtained by the combining processing, a one-dimensional vector indicating a statistic in a time axis direction of each direct product amount as the feature data.
12 . The learning device according to claim 10 , wherein
the intermediate network executes processing of making a plurality of the intermediate feature tensors to have the same number of elements in a time direction.
13 . The learning device according to claim 10 , wherein
the intermediate network executes processing of making a plurality of the planar tensors to be subjected to the combining processing to have the same number of elements in a time direction.