Method and apparatus with keyword detection
A processor-implemented method with keyword detection includes: receiving at least a portion of a voice signal input by a user; extracting a voice feature of the voice signal; inputting an abstract representation sequence of a preset keyword and the voice feature to an end-to-end keyword detection model, and determining a result on whether the preset keyword is present in the voice signal output from the keyword detection model, wherein the keyword detection model predicts whether the preset keyword is present in the voice signal by: determining an abstract representation sequence of the voice signal, based on the voice feature and the abstract representation sequence of the preset keyword; predicting position information of the preset keyword in the voice signal based on the abstract representation sequence of the voice signal; and predicting whether the preset keyword is present in the voice signal, based on the abstract representation sequence of the voice signal and the position information.
1. A processor-implemented method with keyword detection, the method comprising:
receiving at least a portion of a voice signal input by a user;
extracting a voice feature of the voice signal;
inputting an abstract representation sequence of a preset keyword and the voice feature to an end-to-end keyword detection model; and
determining a result on whether the preset keyword is present in the voice signal output from the keyword detection model,
wherein the keyword detection model predicts whether the preset keyword is present in the voice signal by:
determining an abstract representation sequence of the voice signal, based on the voice feature and the abstract representation sequence of the preset keyword;
predicting position information of the preset keyword in the voice signal based on the abstract representation sequence of the voice signal; and
predicting whether the preset keyword is present in the voice signal, based on the abstract representation sequence of the voice signal and the position information.
2. The method of claim 1 , wherein the keyword detection method is performed in real-time for at least a portion of the voice signal input by the user.
3. The method of claim 1 , wherein the preset keyword comprises either one or both of a keyword defined by the user and a preset keyword preset by a system or an application.
4. The method of claim 1 , wherein the determining of the abstract representation sequence of the voice signal, based on the voice feature and the abstract representation sequence of the preset keyword comprises determining the abstract representation sequence of the voice signal by combining the voice feature with the abstract representation sequence of the preset keyword through an attention mechanism.
5. The method of claim 1 , wherein the predicting of whether the preset keyword is present in the voice signal, based on the abstract representation sequence of the voice signal and the position information comprises:
determining an abstract representation sequence of a portion comprising the preset keyword in the voice signal, based on the abstract representation sequence of the voice signal and the position information; and
predicting whether the preset keyword is present in the voice signal by combining the abstract representation sequence of the preset keyword with the abstract representation sequence of the portion comprising the preset keyword in the voice signal through an attention mechanism.
6. The method of claim 4 , wherein the keyword detection model comprises a voice encoder used for predicting an abstract representation sequence of a voice signal, the voice encoder comprises a plurality of submodules connected in series, and each of the submodules inputs the abstract representation sequence of the preset keyword to a hidden layer abstract representation sequence of the voice signal through the attention mechanism.
7. The method of claim 1 , wherein the abstract representation sequence of the preset keyword is generated by a pre-trained keyword encoder based on a phone sequence of the preset keyword.
8. The method of claim 1 , wherein the keyword detection model is determined by multi-objective joint training, and a multi-objective comprises predicting a phone sequence corresponding to the voice signal, predicting a position of a keyword in the voice signal, and predicting whether the keyword is present in the voice signal.
9. The method of claim 8 , wherein an objective loss function corresponding to an object for predicting the position of the keyword in the voice signal is based on a diagonal pattern of an attention matrix.
10. The method of claim 1 , further comprising either one or both of:
waking up a current electronic device in response to the result output from the keyword detection model indicating that the preset keyword is present in the voice signal; and
outputting the result and the position information.
11. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform the method of claim 1 .
12. An apparatus with keyword detection, the apparatus comprising:
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform the method of claim 1 .
13. An apparatus with keyword detection, the apparatus comprising:
a receiver configured to receive at least a portion of a voice signal input by a user;
one or more processors configured to:
extract a voice feature of the voice signal; and
input an abstract representation sequence of a preset keyword and the voice feature to an end-to-end keyword detection model; and
determine a result on whether the preset keyword is present in the voice signal output from the keyword detection model,
wherein the keyword detection model predicts whether the preset keyword is present in the voice signal by:
determining an abstract representation sequence of the voice signal, based on the voice feature and the abstract representation sequence of the preset keyword;
predicting position information of the preset keyword in the voice signal based on the abstract representation sequence of the voice signal; and
predicting whether the preset keyword is present in the voice signal, based on the abstract representation sequence of the voice signal and the position information.
14. The apparatus of claim 11 , wherein the keyword detection apparatus processes at least a portion of the voice signal input by the user in real-time.
15. The apparatus of claim 11 , wherein, for the determining of the abstract representation sequence of the voice signal, the one or more processors are configured to determine the abstract representation sequence of the voice signal by combining the voice feature with the abstract representation sequence of the preset keyword through an attention mechanism.
16. The apparatus of claim 11 , wherein the predicting of whether the preset keyword is present in the voice signal based on the abstract representation sequence of the voice signal and the position information comprises:
determining an abstract representation sequence of a portion comprising the preset keyword in the voice signal based on the abstract representation sequence of the voice signal and the position information; and
predicting whether the preset keyword is present in the voice signal by combining the abstract representation sequence of the preset keyword with the abstract representation sequence of the portion comprising the preset keyword in the voice signal through an attention mechanism.
17. A processor-implemented method with keyword detection, the method comprising:
determining text of a keyword;
determining a phone sequence of the text;
determining whether the keyword satisfies a preset condition based on either one or both of the text and the phone sequence;
inputting, in response to the keyword satisfying the preset condition, the phone sequence to a pre-trained keyword encoder;
generating an abstract representation sequence of the keyword from the pre-trained keyword encoder,
wherein the determining of whether the keyword satisfies the preset condition comprises determining whether a number of syllables of the text is greater than or equal to a predetermined number.
18. The method of claim 17 , further comprising:
receiving at least a portion of a voice signal input by a user;
extracting a voice feature of the voice signal; and
inputting the abstract representation sequence of the keyword and the voice feature to an end-to-end keyword detection model; and
determining a result on whether the keyword is present in the voice signal output from the keyword detection model.
19. The method of claim 17 , wherein the text of the keyword is input by a user.