IP Library Granted Patent US 10,467,348
Granted Patent B2
US 10,467,348 · App. 13/284,973 · Granted Nov 5, 2019

Speech morphing communication system

Inventor: Fathy Yassa (Soquel, CA)
Assignee: SPEECH MORPHING SYSTEMS, INC.
G06F17/289G06F17/2854G10L13/00G10L15/26
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,467,348
App. No.
13/284,973
Granted
Nov 5, 2019
Kind
B2
Abstract

A networked communication system is described. The communication system including an automatic speech recognizer configured to receive a speech signal from a client over a network and to convert the speech signal into a text sequence. The communication also including a speech analyzer configured to receive the speech signal. The speech analyzer configured to extract paralinguistic characteristics from the speech signal. In addition, the communication system includes a speech output device coupled with the automatic speech recognizer and the speech analyzer. The speech output device configured to convert the text sequence into an output speech signal based on the extracted paralinguistic characteristics.

Claims (54)

1. A networked communication system comprising:

an automatic speech recognizer configured to receive a speech signal of a speaker in a first language from a client over a network and to convert said speech signal into a text sequence in the first language;

a speech analyzer configured to receive said speech signal, said speech analyzer configured to extract all paralinguistic characteristics of the speech signal in the first language from said speech signal; and

a speech output device coupled with said automatic speech recognizer and said speech analyzer, said speech output device comprising a look-up table of paralinguistic characteristics mapping between the paralinguistic characteristics of the speech signal in the first language to paralinguistic characteristics of the speech signal in a second language, and said speech output device configured to transform all of the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language based on the mapping between the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language of the look-up table of paralinguistic characteristics and convert said text sequence into an output speech signal in the second language based on said paralinguistic characteristics of the speech signal in the second language,

wherein the transformation of all the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language comprises transforming at least one paralinguistic characteristic of the speech signal in the first language imparting a connotation in the first language to a first word in the speech signal in the first language to at least one corresponding paralinguistic characteristic of the speech signal in the second language imparting the connotation in the second language to a second word of the speech signal in the second language, the second word in the second language being a translation of the first word in the first language,

wherein said paralinguistic characteristics of the speech signal in the first language comprises at least one of a first pitch of the speech signal in the first language, a first amplitude of the speech signal in the first language, and a first rate of speech of the speech signal in the first language, and

wherein said paralinguistic characteristics of the speech signal in the second language comprises at least one of a second pitch of the speech signal in the second language, a second amplitude of the speech signal in the second language, and a second rate of speech of the speech signal in the second language.

2. The networked communication system of claim 1 , further comprising a translator coupled with said automatic speech recognizer, said translator configured to convert said text sequence from the first language to the second language.

3. The networked communication system of claim 1 , wherein said automatic speech recognizer converts said speech signal into said text sequence based on a first algorithm.

4. The networked communication system of claim 3 , further comprising a second automatic speech recognizer, said second automatic speech recognizer configured to convert said speech signal to a second text sequence based on a second algorithm.

5. The networked communication system of claim 4 , further comprising a text sequence comparator configured to compare said text sequence with said second text sequence.

6. The networked communication system of claim 5 , wherein said text sequence comparator generates an error corrected text sequence based on the comparison of said text sequence and said second text sequence.

7. The networked communication system of claim 2 , wherein said translator includes a plurality of translators.

8. The networked communication system of claim 1 , wherein said output device converts said text sequence into the output speech signal using a text-to-speech algorithm.

9. A networked communication system comprising:

a memory;

one or more processors; and

one or more modules stored in memory and configured for execution by the one or more processors, the one or more modules comprising:

a speech input module configured to receive a speech signal of a speaker in a first language from a client over a network;

an automatic speech recognizer module coupled with said speech input module, said automatic speech recognizer module configured to convert said speech signal into a text sequence in the first language;

a speech analyzer module coupled with said speech input module, said speech analyzer module configured to extract all paralinguistic characteristics of the speech signal in the first language from said speech signal; and

an output speech module comprising a look-up table of paralinguistic characteristics mapping between the paralinguistic characteristics of the speech signal in the first language to paralinguistic characteristics of the speech signal in a second language, and said output speech module configured to transform all of the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language based on the mapping between the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language of the look-up table of paralinguistic characteristics and convert said text sequence into an output speech signal in the second language based on said paralinguistic characteristics of the speech signal in the second language,

wherein the transformation of all the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language comprises transforming at least one paralinguistic characteristic of the speech signal in the first language imparting a connotation in the first language to a first word in the speech signal in the first language to at least one corresponding paralinguistic characteristic of the speech signal in the second language imparting the connotation in the second language to a second word of the speech signal in the second language, the second word in the second language being a translation of the first word in the first language,

wherein said paralinguistic characteristics of the speech signal in the first language comprises at least one of a first pitch of the speech signal in the first language, a first amplitude of the speech signal in the first language, and a first rate of speech of the speech signal in the first language, and

wherein said paralinguistic characteristics of the speech signal in the second language comprises at least one of a second pitch of the speech signal in the second language, a second amplitude of the speech signal in the second language, and a second rate of speech of the speech signal in the second language.

10. The networked communication system of claim 9 , further comprising a translator module coupled with said automatic speech recognizer module, said translator configured to convert said text sequence from the first language to the second language.

11. The networked communication system of claim 9 , wherein said automatic speech recognizer module converts said speech signal into said text sequence based on a first algorithm.

12. The networked communication system of claim 11 , further comprising a second automatic speech recognizer module, said second automatic speech recognizer module configured to convert said speech signal to a second text sequence based on a second algorithm.

13. The networked communication system of claim 12 , further comprising a text sequence comparator module configured to compare said text sequence with said second text sequence.

14. The networked communication system of claim 13 , wherein said text sequence comparator module generates an error corrected text sequence based on the comparison of said text sequence and said second text sequence.

15. The networked communication system of claim 10 , wherein said translator module comprises a plurality of translator modules.

16. The networked communication system of claim 9 , wherein said output speech module converts said text sequence into the output speech signal using a text-to-speech algorithm.

17. A networked speech morphing communication system comprising:

a speech input device configured to receive a speech signal of a speaker in a first language from a client over a network;

a first automatic speech recognizer and a second automatic speech recognizer coupled with said input device, said first automatic speech recognizer configured to convert said speech signal into a first text sequence in the first language and said second automatic speech recognizer configured to convert said speech signal into a second text sequence in the first language;

a text sequence comparator coupled with said first automatic speech recognizer and said second automatic speech recognizer, said text sequence comparator configured to select one text sequence from among said first text sequence and said second text sequence;

a speech analyzer coupled with said speech input device, said speech analyzer configured to extract all paralinguistic characteristics of the speech signal in the first language from said speech signal;

one or more translators coupled with said text analyzer, said one or more translators configured to translate said selected text sequence from the first language to one or more second languages to generate a translated text sequence in the one or more second languages; and

a speech output device comprising a look-up table of paralinguistic characteristics mapping between the paralinguistic characteristics of the speech signal in the first language to paralinguistic characteristics of the speech signal in the second language, and said speech output device configured to transform all of the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the one or more second languages based on the mapping between the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the one or more second languages of the look-up table of paralinguistic characteristics and convert said translated text sequence into an output speech signal in the one or more second languages based on said paralinguistic characteristics of the speech signal in the one or more second languages,

wherein the transformation of all the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the one or more second languages comprises transforming at least one paralinguistic characteristic of the speech signal in the first language imparting a connotation in the first language to a first word in the speech signal in the first language to at least one corresponding paralinguistic characteristic of the speech signal in the one or more second languages imparting the connotation in the one or more second languages to a second word of the speech signal in the one or more second languages, the second word in the one or more second languages being a translation of the first word in the first language,

wherein said paralinguistic characteristics of the speech signal in the first language comprises at least one of a first pitch of the speech signal in the first language, a first amplitude of the speech signal in the first language, and a first rate of speech of the speech signal in the first language, and

wherein said paralinguistic characteristics of the speech signal in the second language comprises at least one of a second pitch of the speech signal in the second language, a second amplitude of the speech signal in the second language, and a second rate of speech of the speech signal in the second language.

18. A method for converting speech into text comprising:

receiving an input speech signal of a speaker in a first language from a client over a network;

extracting all paralinguistic characteristics of the speech signal in the first language from said input speech signal;

converting said input speech signal to a text sequence in the first language;

transforming all of the paralinguistic characteristics of the speech signal in the first language to paralinguistic characteristics of the speech signal in a second language; and

transforming said text sequence into an output speech signal in the second language based on said paralinguistic characteristics of the speech signal in the second language,

wherein the transforming of the paralinguistic characteristics comprises:

reading from a look-up table of paralinguistic characteristics mapping between the paralinguistic characteristics of the speech signal in the first language to paralinguistic characteristics of the speech signal in the second language; and

transforming at least one paralinguistic characteristic of the speech signal in the first language imparting a connotation in the first language to a first word in the speech signal in the first language to at least one corresponding paralinguistic characteristic of the speech signal in the second language imparting the connotation in the second language to a second word of the speech signal in the second language based on the mapping between the paralinguistic characteristics of the speech signal in the first language to the paralinguistic characteristics of the speech signal in the second language of the look-up table of paralinguistic characteristics, the second word in the second language being a translation of the first word in the first language,

wherein said paralinguistic characteristics of the speech signal in the first language comprises at least one of a first pitch of the speech signal in the first language, a first amplitude of the speech signal in the first language, and a first rate of speech of the speech signal in the first language, and

wherein said paralinguistic characteristics of the speech signal in the second language comprises at least one of a second pitch of the speech signal in the second language, a second amplitude of the speech signal in the second language, and a second rate of speech of the speech signal in the second language.

19. The method of claim 18 , further comprising translating said text sequence from the first language to the second language.

Assignments (2)
CHANGE OF NAME Recorded Mar 29, 2016
From: SPEECH MORPHING, INC.
To: SPEECH MORPHING SYSTEMS, INC.
Reel/Frame 038123/0026 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2014
From: YASSA, FATHY
To: SPEECH MORPHING, INC.
Reel/Frame 033889/0079 →
Continuity (2)
Provisional Application 61408634 · Oct 31, 2010
Related Publication 20120109627A1 · May 3, 2012