Keyword spotting apparatus, method, and computer-readable recording medium thereof
A keyword spotting apparatus, method, and computer-readable recording medium are disclosed. The keyword spotting method using an artificial neural network according to an embodiment of the disclosure may include obtaining an input feature map from an input voice; performing a first convolution operation on the input feature map for each of n different filters having the same channel length as the input feature map, wherein a width of each of the filters is w1 and the width w1 is less than a width of the input feature map; performing a second convolution operation on a result of the first convolution operation for each of different filters having the same channel length as the input feature map; storing a result of the second convolution operation as an output feature map; and extracting a voice keyword by applying the output feature map to a learned machine learning model.
1. A method of operation of an apparatus for keyword spotting, the method comprising:
obtaining, from an input voice, an input feature map;
wherein lengths in a channel direction of the input feature map are independently determined for a plurality of sections;
wherein the plurality of sections is obtained by dividing the input voice by a predetermined period; and
wherein each length is defined based on frequency data extracted from a corresponding section of the input voice and corresponds to frequency value for the corresponding section;
performing a convolution operation between the input feature map and at least one filter;
wherein performing the convolution operation comprises performing a first convolution operation between the input feature map and each of n different filters;
wherein the n different filters cover a frequency range of the input feature map and each have a channel length that is the same as the channel length for the input feature map; and
wherein the channel length is above zero;
storing a result of the convolution operation as an output feature map; and
extracting a keyword from the input voice based on the output feature map,
wherein each filter of the n different filters used in the first convolution operation is configured to distinguish characteristics of different voices corresponding to letter sounds.
2. The method of claim 1 , wherein the input feature map represents data corresponding to a size of t×1×f (width×height×channel) and the at least one filter represents data corresponding to a size of t′×1×f (width×height×channel), where t and t′ represent the number of activations listed in an axial direction corresponding to time, and f represents the number of activations listed in an axial direction corresponding to frequency.
3. The method of claim 1 , further comprising performing a second convolution operation by using a first convolution block that performs a plurality of sub-convolution operations in which one or more stride values are respectively applied to a result of the first convolution operation.
4. The method of claim 1 , further comprising performing a second convolution based on a plurality of first convolution blocks to which at least one of a number of strides and a number of filters is applied differently.
5. The method of claim 3 , wherein performing the second convolution operation comprises performing a second convolution by using a second convolution block that performs a plurality of preset sub-convolution operations based on a result of an operation of the first convolution block and a stride value.
6. The method of claim 1 ,
wherein obtaining the input feature map includes obtaining an input feature map having a size of t×1×f (width×height×channel) from a result of mel frequency cepstral coefficient (MFCC) processing for the input voice, and
wherein t denotes time and f denotes frequency.
7. An apparatus for extracting a voice keyword using an artificial neural network, the apparatus comprising:
a memory in which at least one program is stored; and
a processor for extracting a voice keyword using the artificial neural network by executing the at least one program, wherein the processor is configured to:
obtain, from an input voice, an input feature map;
wherein lengths in a channel direction of the input feature map are independently determined for a plurality of sections;
wherein the plurality of sections is obtained by dividing the input voice by a predetermined period; and
wherein each length is defined based on frequency data extracted from a corresponding section of the input voice and corresponds to frequency value for the corresponding section;
perform a convolution operation between the input feature map and at least one filter;
wherein performing the convolution operation comprises performing a first convolution operation between the input feature map and each of n different filters;
wherein the n different filters cover a frequency range of the input feature map and each have a channel length that is the same as the channel length for the input feature map; and
wherein the channel length is above zero;
store a result of the convolution operation as an output feature map, and
extract a keyword from the input voice based on the output feature map,
wherein each filter of the n different filters used in the first convolution operation is configured to distinguish characteristics of different voices corresponding to letter sounds.
8. The apparatus of claim 7 , wherein the input feature map represents data corresponding to a size of t×1×f (width×height×channel) and the at least one filter represents data corresponding to a size of t′×1×f (width×height×channel), where t and t′ represent the number of activations listed in an axial direction corresponding to time, and f represents the number of activations listed in an axial direction corresponding to frequency.
9. The apparatus of claim 7 , wherein the processor is further configured to perform a second convolution operation by using a first convolution block that performs a plurality of sub-convolution operations in which one or more stride values are respectively applied to a result of the first convolution operation.
10. The apparatus of claim 7 , wherein the processor is further configured to perform a second convolution based on a plurality of first convolution blocks to which at least one of a number of strides and a number of filters is applied differently.
11. The apparatus of claim 9 , wherein the processor is further configured to perform a second convolution by using a second convolution block that performs a plurality of preset sub-convolution operations based on a result of an operation of the first convolution block and a stride value.
12. The apparatus of claim 7 ,
wherein the processor is configured to obtain an input feature map having a size of t×1×f (width×height×channel) from a result of mel frequency cepstral coefficient (MFCC) processing for the input voice, and
wherein t denotes time and f denotes frequency.
13. A non-transitory computer-readable recording medium for keyword spotting, wherein the program instructions are executable by one or more processors to perform a process that comprises:
obtaining, from an input voice, an input feature map;
wherein lengths in a channel direction of the input feature map are independently determined for a plurality of sections;
wherein the plurality of sections is obtained by dividing the input voice by a predetermined period; and
wherein each length is defined based on frequency data extracted from a corresponding section of the input voice and corresponds to frequency value for the corresponding section;
performing a convolution operation between the input feature map and at least one filter;
wherein performing the convolution operation comprises performing a first convolution operation between the input feature map and each of n different filters; and
wherein the n different filters cover a frequency range of the input feature map and each have a channel length that is the same as the channel length for the input feature map; and
wherein the channel length is above zero;
storing a result of the convolution operation as an output feature map; and
extracting a keyword from the input voice based on the output feature map,
wherein each filter of the n different filters used in the first convolution operation is configured to distinguish characteristics of different voices corresponding to letter sounds.
14. The non-transitory computer-readable recording medium of claim 13 , wherein the input feature map represents data corresponding to a size of t×1×f (width×height×channel) and the at least one filter represents data corresponding to a size of t′×1×f (width×height×channel), where t and t′ represent the number of activations listed in an axial direction corresponding to time, and f represents the number of activations listed in an axial direction corresponding to frequency.
15. The non-transitory computer-readable recording medium of claim 13 , further comprising performing a second convolution operation by using a first convolution block that performs a plurality of sub-convolution operations in which one or more stride values are respectively applied to a result of the first convolution operation.
16. The non-transitory computer-readable recording medium of claim 13 , further comprising performing a second convolution based on a plurality of first convolution blocks to which at least one of a number of strides and a number of filters is applied differently.
17. The non-transitory computer-readable recording medium of claim 15 , wherein performing the second convolution operation comprises performing a second convolution by using a second convolution block that performs a plurality of preset sub-convolution operations based on a result of an operation of the first convolution block and a stride value.