IP Library › Granted Patent US 11,450,312
Granted Patent B2
US 11,450,312 · App. 16/900,824 · Granted Sep 20, 2022

Speech recognition method, apparatus, and device, and storage medium

Inventors: Shilun Lin (Shenzhen, CN); Xilin Zhang (Shenzhen, CN); Wenhua Ma (Shenzhen, CN); Bo Liu (Shenzhen, CN); Xinhui Li (Shenzhen, CN); Li Lu (Shenzhen, CN); Xiucai Jiang (Shenzhen, CN)
Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
G10L15/16G10L15/02G10L15/05G10L15/142G10L15/22G10L2015/025G10L2015/088G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,450,312
App. No.
16/900,824
Granted
Sep 20, 2022
Kind
B2
Abstract

A speech recognition method includes: obtaining speech information; and determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network. The candidate speech segment is identified as corresponding to a preset keyword. The method also includes clipping the candidate speech segment from the speech information according to the beginning and ending positions of the candidate speech segment; detecting whether the candidate speech segment includes a preset keyword by using a machine learning model; and determining, upon determining that the candidate speech segment comprises the preset keyword, that the speech information comprises the preset keyword.

Claims (67)

1. A speech recognition method, performed by a computing device, and comprising:

obtaining speech information;

determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network, the WFST network including a deep neural network (DNN), a hidden Markov model (HMM), a dictionary, and a language model, and the candidate speech segment being identified as corresponding to a preset keyword;

clipping the candidate speech segment from the speech information according to the beginning and ending positions;

detecting whether the candidate speech segment comprises the preset keyword by using a convolutional neural network (CNN); and

determining, upon determining that the candidate speech segment comprises the preset keyword, that the speech information comprises the preset keyword.

2. The method according to claim 1 , wherein the determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network comprises:

performing framing on the speech information, to obtain a plurality of frames of speech segments; and

inputting the plurality of frames of speech segments into the WFST network.

3. The method according to claim 2 , wherein inputting the plurality of frames of speech segments into the WFST network comprises:

inputting the plurality of frames of speech segments into the DNN, to obtain a posterior probability of a hidden state corresponding to each of the plurality of frames of speech segments; and

obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMM.

4. The method according to claim 3 , wherein the obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMI comprises:

converting the posterior probability of the hidden state corresponding to each frame of speech segment to an emission probability of the hidden state corresponding to each frame of speech segment.

5. The method according to claim 1 , wherein detecting whether the candidate speech segment comprises a preset keyword by using the machine learning model comprises:

inputting the candidate speech segment into the CNN;

performing convolution and pooling on the candidate speech segment by using the CNN, to obtain semantic features of the candidate speech segment through extraction; and

classifying the semantic features of the candidate speech segment by using the CNN, to detect whether the candidate speech segment comprises the preset keyword.

6. The method according to claim 1 , wherein the computing device is a server, and the method further comprises:

obtaining the speech information from a terminal; and

after determining that the speech information comprises the preset keyword, transmitting a wakeup instruction to the terminal, the wakeup instruction being configured to lift a dormant state and/or a lock screen state of the terminal.

7. The method according to claim 1 , wherein the computing device is a terminal, and the method further comprises:

after determining that the speech information comprises the preset keyword, generating a wakeup instruction to lift a dormant state and/or a lock screen state of the terminal.

8. A speech recognition device, comprising a processor and a memory, the processor being configured to perform:

obtaining speech information;

determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network, the WFST network including a deep neural network (DNN), a hidden Markov model (HMM), a dictionary, and a language model, and the candidate speech segment being identified as corresponding to a preset keyword;

clipping the candidate speech segment from the speech information according to the beginning and ending positions;

detecting whether the candidate speech segment comprises the preset keyword by using a convolutional neural network (CNN); and

determining, upon determining that the candidate speech segment comprises the preset keyword, that the speech information comprises the preset keyword.

9. The device according to claim 8 , wherein the determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network comprises:

performing framing on the speech information, to obtain a plurality of frames of speech segments; and

inputting the plurality of frames of speech segments into the WFST network.

10. The device according to claim 9 , wherein inputting the plurality of frames of speech segments into the WFST network comprises:

inputting the plurality of frames of speech segments into the DNN, to obtain a posterior probability of a hidden state corresponding to each of the plurality of frames of speech segments; and

obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMM.

11. The device according to claim 10 , wherein the obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMM comprises:

converting the posterior probability of the hidden state corresponding to each frame of speech segment to an emission probability of the hidden state corresponding to each frame of speech segment.

12. The device according to claim 8 , wherein detecting whether the candidate speech segment comprises a preset keyword by using the machine learning model comprises:

inputting the candidate speech segment into the CNN;

performing convolution and pooling on the candidate speech segment by using the CNN, to obtain semantic features of the candidate speech segment through extraction; and

classifying the semantic features of the candidate speech segment by using the CNN, to detect whether the candidate speech segment comprises the preset keyword.

13. The device according to claim 8 , wherein the processor is further configured to perform:

obtaining the speech information from a terminal; and

after determining that the speech information comprises the preset keyword, transmitting a wakeup instruction to the terminal, the wakeup instruction being configured to lift a dormant state and/or a lock screen state of the terminal.

14. The device according to claim 8 , wherein the processor is further configured to perform:

after determining that the speech information comprises the preset keyword, generating a wakeup instruction to lift a dormant state and/or a lock screen state of the device.

15. A non-transitory computer-readable storage medium, storing at least one instruction, the at least one instruction being loaded and executed by a processor to implement:

obtaining speech information;

determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network, the WFST network including a deep neural network (DNN), a hidden Markov model (HMM), a dictionary, and a language model, and the candidate speech segment being identified as corresponding to a preset keyword;

clipping the candidate speech segment from the speech information according to the beginning and ending positions;

detecting whether the candidate speech segment comprises the preset keyword by using a convolutional neural network (CNN); and

determining, upon determining that the candidate speech segment comprises the preset keyword, that the speech information comprises the preset keyword.

16. The storage medium according to claim 15 , wherein the determining beginning and ending positions of a candidate speech segment in the speech information by using a weighted finite state transducer (WFST) network comprises:

performing framing on the speech information, to obtain a plurality of frames of speech segments; and

inputting the plurality of frames of speech segments into the WFST network.

17. The storage medium according to claim 16 , wherein inputting the plurality of frames of speech segments into the WFST network comprises:

inputting the plurality of frames of speech segments into the DNN, to obtain a posterior probability of a hidden state corresponding to each of the plurality of frames of speech segments; and

obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMI.

18. The storage medium according to claim 17 , wherein the obtaining the hidden state corresponding to each frame of speech segment according to the posterior probability of the hidden state corresponding to each frame of speech segment by using the HMI comprises:

converting the posterior probability of the hidden state corresponding to each frame of speech segment to an emission probability of the hidden state corresponding to each frame of speech segment.

19. The storage medium according to claim 15 , wherein detecting whether the candidate speech segment comprises a preset keyword by using the machine learning model comprises:

inputting the candidate speech segment into the CNN;

performing convolution and pooling on the candidate speech segment by using the CNN, to obtain semantic features of the candidate speech segment through extraction; and

classifying the semantic features of the candidate speech segment by using the CNN, to detect whether the candidate speech segment comprises the preset keyword.

20. The storage medium according to claim 15 , wherein the processor is further configured to perform:

obtaining the speech information from a terminal; and

after determining that the speech information comprises the preset keyword, transmitting a wakeup instruction to the terminal, the wakeup instruction being configured to lift a dormant state and/or a lock screen state of the terminal.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2020
From: LIN, SHILUN; ZHANG, XILIN; MA, WENHUA; LIU, BO; LI, XINHUI; LU, LI; JIANG, XIUCAI
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 052933/0009 →
Priority Claims (1)
CN 201810240076.X · Mar 22, 2018 · national
Continuity (2)
Continuation PCTCN2019076223 · Feb 27, 2019
Related Publication 20200312309A1 · Oct 1, 2020