IP Library › Granted Patent US 11,862,143
Granted Patent B2
US 11,862,143 · App. 16/996,961 · Granted Jan 2, 2024

Systems and methods for processing speech dialogues

Inventors: Haiyang Xu (Beijing, CN); Kun Han (Mountain View, CA)
Assignee: BEIJING DIDI INFINITY TECHNOLOGY AND DEVELOPMENT CO., LTD.
G10L15/063G06F18/24G06F40/253G10L15/02G10L15/22G10L2015/025G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,862,143
App. No.
16/996,961
Granted
Jan 2, 2024
Kind
B2
Abstract

The present disclosure is related to systems and methods for processing speech dialogue. The method includes obtaining target speech dialogue data. The method includes obtaining a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the target speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively. The method includes determining a representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence into a trained speech dialogue coding model. The method includes determining a summary of the target speech dialogue data by inputting the representation vector into a classification model.

Claims (72)

1. A method for processing speech dialogue implemented on a computing device having at least one processor and at least one storage device, the method comprising:

obtaining target speech dialogue data;

obtaining a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the target speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively;

determining a representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence into a trained speech dialogue coding model;

determining a summary of the target speech dialogue data by inputting the representation vector into a classification model; and

generating an output utilizing the determined summary of the target speech dialogue data.

2. The method of claim 1 , further comprising:

obtaining a sentence text of the summary of the target speech dialogue data; and

performing a grammatical correction operation on the sentence text.

3. The method of claim 1 , wherein the text embedding model includes at least one of:

a word embedding sub-model configured to determine a word vector representation sequence of the target speech dialogue data;

a position embedding sub-model configured to determine a position vector representation sequence of the target speech dialogue data; and

a paragraph embedding sub-model configured to determine a paragraph vector representation sequence of the target speech dialogue data.

4. The method of claim 1 , wherein the determining the representation vector corresponding to the target speech dialogue data includes:

obtaining at least one of a dialect vector representation sequence, an emotion vector representation sequence, or a background text vector representation sequence corresponding to the target speech dialogue data, wherein

the dialect vector representation sequence is determined by performing a vector transformation on the target speech dialogue data based on a dialect embedding model;

the emotion vector representation sequence is determined by performing a vector transformation on the target speech dialogue data based on an emotion embedding model; and

the background text vector representation sequence is determined by performing a vector transformation on a background text of the target speech dialogue data based on a background text embedding model; and

determining the representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, the role vector representation sequence, and at least one of the dialect vector representation sequence, the emotion vector representation sequence, or the background text vector representation sequence into the trained speech dialogue coding model.

5. The method of claim 1 , wherein the speech dialogue coding model is determined according to a training process, the training process including:

obtaining sample speech dialogue data;

obtaining a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the sample speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively; and

obtaining a pre-trained speech dialogue coding model by pre-training the speech dialogue coding model in a self-supervised learning manner based on the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence.

6. The method of claim 5 , wherein the training process further includes:

jointly pre-training the speech dialogue coding model and at least one of the text embedding model, the phonetic symbol embedding model, or the role embedding model.

7. The method of claim 5 , wherein the pre-training the speech dialogue coding model in the self-supervised learning manner includes:

designating at least portion of at least one of the text vector representation sequence, the phonetic symbol vector representation sequence, or the role vector representation sequence as an annotation, the annotation including at least portion of elements in the role vector representation sequence.

8. The method of claim 7 , wherein the annotation further includes one or more keywords in the text vector representation sequence.

9. The method of claim 7 , wherein the annotation further includes an order of sentences embodied in the text vector representation sequence.

10. The method of claim 5 , wherein the obtaining the pre-trained speech dialogue coding model by pre-training the speech dialogue coding model includes:

obtaining at least one of a dialect vector representation sequence, an emotion vector representation sequence, or a background text vector representation sequence corresponding to the sample speech dialogue data, wherein

the dialect vector representation sequence is determined by performing a vector transformation on the sample speech dialogue data based on a dialect embedding model;

the emotion vector representation sequence is determined by performing a vector transformation on the sample speech dialogue data based on an emotion embedding model; and

the background text vector representation sequence is determined by performing a vector transformation on a background text of the sample speech dialogue data based on a background text embedding model; and

obtaining the pre-trained speech dialogue coding model by pre-training the speech dialogue coding model in the self-supervised learning manner based on the text vector representation sequence, the phonetic symbol vector representation sequence, the role vector representation sequence, and at least one of the dialect vector representation sequence, the emotion vector representation sequence, or the background text vector representation sequence.

11. A system for processing speech dialogue, comprising:

at least one storage device including a set of instructions; and

at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is directed to cause the system to:

obtain target speech dialogue data;

obtain a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the target speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively;

determine a representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence into a trained speech dialogue coding model;

determine a summary of the target speech dialogue data by inputting the representation vector into a classification model; and

generate an output utilizing the determined summary of the target speech dialogue data.

12. The system of claim 11 , wherein the at least one processor is further directed to cause the system to:

obtain a sentence text of the summary of the target speech dialogue data; and

perform a grammatical correction operation on the sentence text.

13. The system of claim 11 , wherein the text embedding model includes at least one of:

a word embedding sub-model configured to determine a word vector representation sequence of the target speech dialogue data;

a position embedding sub-model configured to determine a position vector representation sequence of the target speech dialogue data; and

a paragraph embedding sub-model configured to determine a paragraph vector representation sequence of the target speech dialogue data.

14. The system of claim 11 , wherein to determine the representation vector corresponding to the target speech dialogue data, the at least one processor is further directed to cause the system to:

obtain at least one of a dialect vector representation sequence, an emotion vector representation sequence, or a background text vector representation sequence corresponding to the target speech dialogue data, wherein

the dialect vector representation sequence is determined by performing a vector transformation on the target speech dialogue data based on a dialect embedding model;

the emotion vector representation sequence is determined by performing a vector transformation on the target speech dialogue data based on an emotion embedding model; and

the background text vector representation sequence is determined by performing a vector transformation on a background text of the target speech dialogue data based on a background text embedding model; and

determine the representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, the role vector representation sequence, and at least one of the dialect vector representation sequence, the emotion vector representation sequence, or the background text vector representation sequence into the trained speech dialogue coding model.

15. The system of claim 11 , wherein the speech dialogue coding model is determined according to a training process, the training process including:

obtaining sample speech dialogue data;

obtaining a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the sample speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively; and

obtaining a pre-trained speech dialogue coding model by pre-training the speech dialogue coding model in a self-supervised learning manner based on the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence.

16. The system of claim 15 , wherein the training process further includes:

jointly pre-training the speech dialogue coding model and at least one of the text embedding model, the phonetic symbol embedding model, or the role embedding model.

17. The system of claim 15 , wherein the pre-training the speech dialogue coding model in the self-supervised learning manner includes:

designating at least portion of at least one of the text vector representation sequence, the phonetic symbol vector representation sequence, or the role vector representation sequence as an annotation, the annotation including at least portion of elements in the role vector representation sequence.

18. The system of claim 17 , wherein the annotation further includes one or more keywords in the text vector representation sequence.

19. The system of claim 17 , wherein the annotation further includes an order of sentences embodied in the text vector representation sequence.

20. A non-transitory computer readable medium, comprising at least one set of instructions, wherein when executed by at least one processor of a computing device, the at least one set of instructions cause the at least one processor to effectuate a method comprising:

obtaining target speech dialogue data;

obtaining a text vector representation sequence, a phonetic symbol vector representation sequence, and a role vector representation sequence by performing a vector transformation on the target speech dialogue data based on a text embedding model, a phonetic symbol embedding model, and a role embedding model, respectively;

determining a representation vector corresponding to the target speech dialogue data by inputting the text vector representation sequence, the phonetic symbol vector representation sequence, and the role vector representation sequence into a trained speech dialogue coding model;

determining a summary of the target speech dialogue data by inputting the representation vector into a classification model; and

generating an output utilizing the determined summary of the target speech dialogue data.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: HAN, KUN
To: DIDI RESEARCH AMERICA, LLC
Reel/Frame 053557/0815 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: XU, HAIYANG
To: BEIJING DIDI INFINITY TECHNOLOGY AND DEVELOPMENT CO., LTD.
Reel/Frame 053557/0830 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 21, 2020
From: DIDI RESEARCH AMERICA, LLC
To: BEIJING DIDI INFINITY TECHNOLOGY AND DEVELOPMENT CO., LTD.
Reel/Frame 053557/0833 →
Priority Claims (1)
CN 202010731224.5 · Jul 27, 2020 · national
Continuity (1)
Related Publication 20220028371A1 · Jan 27, 2022