IP Library Granted Patent US 11,227,129
Granted Patent B2
US 11,227,129 · App. 16/866,406 · Granted Jan 18, 2022

Language translation device and language translation method

Inventors: Sangil Ahn (Cheongju-si, KR); Kangsik Jung (Seoul, KR); Hyountaek Yong (Seoul, KR); Hyeok Choi (Seoul, KR)
Assignee: Hyperconnect, Inc.
G06F40/58G06F40/51G10L15/26H04N7/147H04N7/15G06F9/454G06F40/40G06F40/42G10L15/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,227,129
App. No.
16/866,406
Granted
Jan 18, 2022
Kind
B2
Abstract

A method of providing real-time translation for video chat is provided. The method includes: continuously receiving first-language voice data and at least one second-language word from a first terminal; continuously displaying the at least one second-language word at the same time as reproduction of the voice data; acquiring a second-language translation of an ended sentence included in a voice recognition result for the voice data; and substituting at least one word, which corresponds to the ended sentence in the displayed at least one second-language word, with the acquired translation. The at least one second-language word corresponds to respective words included in the voice recognition result for the voice data.

Claims (49)

1. A language translation method at a transmitting terminal, the method comprising:

continuously receiving a stream of voice in a first-language;

continuously acquiring a voice recognition result for the stream of voice;

continuously acquiring second-language words, at least one of the second-language words being converted based on each of first-language words included in the voice recognition result;

continuously transmitting the acquired second-language words to a receiving terminal;

identifying an ended sentence included in the voice recognition result;

acquiring a second-language translation of the identified ended sentence, the identified ended sentence including the first-language words included in the voice recognition result; and

transmitting the acquired second-language translation to the receiving terminal to substitute at least one word of the transmitted second-language words for being displayed on the receiving terminal with at least a portion of the transmitted second-language translation.

2. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:

acquiring a similarity between a word order of the first-language and a word order of the second-language; and

when the acquired similarity is greater than a reference value, continuously converting each first-language word included in the voice recognition result into a second-language word having an identical or similar meaning thereto, and

wherein the continuously transmitting to the receiving terminal includes, when the acquired similarity is greater than the reference value, continuously transmitting the stream of voice and the acquired second-language words to the receiving terminal.

3. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:

transmitting the first-language words included in the voice recognition result to a first server; and

receiving the at least one second-language word having an identical or similar meaning to each of the transmitted first-language words from the first server.

4. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:

retrieving each of the first-language words included in the voice recognition result from a second database; and

when the second-language words corresponding to the retrieved first-language word is present in the second database, acquiring the second-language word corresponding to the retrieved first-language word from the second database as a result of the retrieving.

5. The language translation method of claim 1 , wherein the continuously acquiring the second-language words comprises:

retrieving one of the first language words included in the voice recognition result from a second database;

when a second-language word corresponding to the retrieved first-language word is not present in the second database, transmitting the retrieved first-language word to a second server;

receiving a second-language word having an identical or similar meaning to the transmitted first-language word from the second server; and

caching the transmitted retrieved first-language word and the received second-language word in the second database.

6. The language translation method of claim 1 , wherein the ended sentence included in the voice recognition result is identified based on

an end point using an intonation included in the received voice, a blank period included in the received voice, or a sentence constituent of the first-language word included in the voice recognition result; and

wherein the ended sentence is defined based on, section from a portion subsequent to a portion corresponding to a previous end point to a portion corresponding to the identified end point in the voice recognition result.

7. The language translation method of claim 6 , wherein the end point is identified based on an intonation corresponding to an end of a sentence in the received voice, a blank period exceeding a specific time in the received voice, or whether the voice recognition result includes first-language words corresponding to all sentence constituents necessary for constituting a full sentence.

8. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:

transmitting the identified ended sentence to a first server; and

receiving the second-language translation of the ended sentence from the first server.

9. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:

retrieving the identified ended sentence from a first database; and

when the second-language translation corresponding to the ended sentence is present in the first database, acquiring the second-language translation corresponding to the ended sentence from the first database as a result of the retrieving.

10. The language translation method of claim 1 , wherein the acquiring of the second-language translation comprises:

retrieving the identified ended sentence from a first database;

when the second-language translation corresponding to the ended sentence is not present in the first database, transmitting the ended sentence to a first server;

receiving the second-language translation of the ended sentence from the first server; and

caching the ended sentence and the received second-language translation of the ended sentence in the first database.

11. A terminal for transmitting information related to language translation, the terminal comprising;

a communication interface;

a display; and

a controller in data communication with the communication interface and the display, the controller configured to:

continuously receive a stream of voice in a first-language;

continuously acquire a voice recognition result for the stream of voice;

continuously acquire second-language words, at least one of the second-language words being converted based on each of first-language words included in the voice recognition result;

continuously transmit the acquired second-language words to a receiving terminal;

identify an ended sentence included in the voice recognition result;

acquire a second-language translation of the identified ended sentence, the identified ended sentence including the first-language words included in the voice recognition result; and

transmit the acquired second-language translation to the receiving terminal to substitute at least one word of the transmitted second-language words for being displayed on the receiving terminal with at least a portion of the transmitted second-language translation.

Assignments (4)
CONFIRMATION OF ASSIGNMENT Recorded Nov 30, 2022
From: AHN, SANG IL
To: HYPERCONNECT INC.
Reel/Frame 062026/0401 →
CONFIRMATION OF ASSIGNMENT Recorded Nov 28, 2022
From: JUNG, KANG SIK
To: HYPERCONNECT INC.
Reel/Frame 062005/0483 →
CONFIRMATION OF ASSIGNMENT Recorded Nov 21, 2022
From: YONG, HYOUN TAEK
To: HYPERCONNECT INC.
Reel/Frame 061975/0837 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2021
From: AHN, SANGIL; JUNG, KANGSIK; YONG, HYOUNTAEK; CHOI, HYEOK
To: HYPERCONNECT, INC.
Reel/Frame 056419/0956 →
Priority Claims (1)
KR 10-2016-0104579 · Aug 18, 2016 · national
Continuity (2)
Continuation 15681221 · Aug 18, 2017
Related Publication 20200265197A1 · Aug 20, 2020