Granted Patent B2

US 12,437,748 · App. 17/965,869 · Granted Oct 7, 2025

Inventor: Yinhui Zhang (Beijing, CN)

Assignee: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

G10L15/02G06F17/16G06F18/2415G06F18/253G06F40/166G06F40/20G06F40/279G06F40/30G06N3/045G06N3/08G10L15/183

Abstract

Claims (62)

1. A spoken language processing method, comprising:

obtaining spoken text information by collecting speech information and recognizing the speech information through automatic speech recognition;

determining a word feature of a word in the spoken text information;

determining a correlation feature of the word in the spoken text information;

determining an effect of the word on fluency of the spoken text information according to the word feature of the word and the correlation feature of the word; and

correcting the spoken text information according to the effect of the word on the fluency of the spoken text information to obtain target text information so as to improve quality of the spoken text information;

wherein determining the correlation feature of the word in the spoken text information comprises:

determining a word vector matrix of the word in the spoken text information; and

determining the correlation feature of the word according to the word vector matrix of the word; and

wherein determining the correlation feature of the word according to the word vector matrix of the word comprises:

determining a Hadamard product of a word vector matrix of an i-th word and a word vector matrix of a i-th word in the spoken text information;

determining a correlation matrix according to the Hadamard product; and

performing three-dimensional (3D) feature extraction on the correlation matrix to obtain correlation features of the i-th word and the j-th word, and

wherein i and j are natural numbers.

2. The method of claim 1 , wherein determining the correlation matrix according to the Hadamard product comprises:

determining relative position encoding of the i-th word and the j-th word in the spoken text information; and

adding the relative position encoding to the Hadamard product to obtain the correlation matrix of the i-th word and the j-th word.

3. The method of claim 1 , wherein determining the word vector matrix of the word in the spoken text information comprises:

processing the spoken text information by a language representation model to obtain the word vector matrix of the word in the spoken text information;

determining the word feature of the word in the spoken text information comprises:

performing two-dimensional (2D) feature extraction on the word vector matrix of the word to obtain the word feature of the word.

4. A spoken language processing apparatus, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores an instruction executable by the at least one processor to enable the at least one processor to perform the following steps:

obtaining spoken text information by collecting speech information and recognizing the speech information through automatic speech recognition;

determining a word feature of a word in the spoken text information;

determining a correlation feature of the word in the spoken text information;

determining an effect of the word on fluency of the spoken text information according to the word feature of the word and the correlation feature of the word; and

correcting the spoken text information according to the effect of the word on the fluency of the spoken text information to obtain target text information so as to improve quality of the spoken text information;

wherein determining the correlation feature of the word in the spoken text information comprises:

determining a word vector matrix of the word in the spoken text information; and

determining the correlation feature of the word according to the word vector matrix of the word; and

wherein determining the correlation feature of the word according to the word vector matrix of the word comprises:

determining a Hadamard product of a word vector matrix of an i-th word and a word vector matrix of a j-th word in the spoken text information;

determining a correlation matrix according to the Hadamard product; and

performing three-dimensional (3D) feature extraction on the correlation matrix to obtain correlation features of the i-th word and the j-th word, and

wherein i and i are natural numbers.

5. The apparatus of claim 4 , wherein determining the correlation matrix according to the Hadamard product comprises:

determining relative position encoding of the i-th word and the j-th word in the spoken text information; and

adding the relative position encoding to the Hadamard product to obtain the correlation matrix of the word.

6. The apparatus of claim 4 , wherein determining the word vector matrix of the word in the spoken text information comprises:

processing the spoken text information by a language representation model to obtain the word vector matrix of the word in the spoken text information; and

determining the word feature of the word in the spoken text information comprises performing two-dimensional (2D) feature extraction on the word vector matrix of the word to obtain the word feature of the word.

7. A non-transitory computer-readable storage medium, which is configured to store a computer instruction for causing a computer to perform the following steps:

obtaining spoken text information by collecting speech information and recognizing the speech information through automatic speech recognition;

determining a word feature of a word in the spoken text information;

determining a correlation feature of the word in the spoken text information;

determining an effect of the word on fluency of the spoken text information according to the word feature of the word and the correlation feature of the word; and

correcting the spoken text information according to the effect of the word on the fluency of the spoken text information to obtain target text information so as to improve quality of the spoken text information;

wherein determining the correlation feature of the word in the spoken text information comprises:

determining a word vector matrix of the word in the spoken text information; and

determining the correlation feature of the word according to the word vector matrix of the word; and

wherein determining the correlation feature of the word according to the word vector matrix of the word comprises:

determining a Hadamard product of a word vector matrix of an i-th word and a word vector matrix of a j-th word in the spoken text information;

determining a correlation matrix according to the Hadamard product; and

performing three-dimensional (3D) feature extraction on the correlation matrix to obtain correlation features of the i-th word and the j-th word;

wherein i and i are natural numbers.

8. The storage medium of claim 7 , wherein determining the correlation matrix according to the Hadamard product comprises:

determining relative position encoding of the i-th word and the j-th word in the spoken text information; and

adding the relative position encoding to the Hadamard product to obtain the correlation matrix of the i-th word and the j-th word.

9. The storage medium of claim 7 , wherein determining the word vector matrix of the word in the spoken text information comprises:

processing the spoken text information by a language representation model to obtain the word vector matrix of the word in the spoken text information;

determining the word feature of the word in the spoken text information comprises:

performing two-dimensional (2D) feature extraction on the word vector matrix of the word to obtain the word feature of the word.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (48)