Granted Patent B2

US 12,437,673 · App. 18/101,904 · Granted Oct 7, 2025

Inventors: Daryl Luciano Peralta (Metro Manila, PH); Shakira Arguelles (Rizal, PH); Williard Joshua Decena Jose (Metro Manila, PH)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G09B21/009G06T9/00G06T13/40G10L13/00

Abstract

Claims (34)

1. A system for bidirectional automatic sign language translation and production, the system comprising:

at least one communication-capable device in communication with another communication-capable device;

at least one visual sensor disposed on the at least one communication-capable device for acquiring input visual feed;

at least one audio sensor disposed on the at least one communication-capable device for acquiring input audio feed;

at least one text interface disposed on the at one least communication-capable device for acquiring input text feed;

the at least one communication-capable device further comprising:

at least one visual display; and

at least one auditory display;

a translation block for processing the input visual feed, the translation block comprising:

an input processing module;

a frame encoder in communication with the input processing module;

a sequence encoder in communication with the frame encoder;

a word-level decoder in communication with the sequence encoder;

a sentence-level decoder in communication with the sequence encoder;

a text-to-speech module in communication with the sentence-level decoder; and

a first output processor in communication with the word-level decoder, the sentence-level decoder, and the text-to-speech module;

a production block for processing the audio feed and text feed, the production block comprising:

a speech recognition module;

an input processor in communication with the speech recognition module;

an input-to-pose generator in communication with the input processor the input-to-pose genera of figured to e f poses;

a pose sequence buffer in communication with the input-to-pose generator, the pose sequence buffer being configured to store the sequence of poses, check when the pose sequence buffer is empty, and generate an end-of-pose signal indicating an end of the sequence of poses in the pose sequence buffer; and

a second output processor in communication with the pose sequence buffer to receive the sequence of poses, the second output processor being configured to receive the end-of-pose signal;

wherein a production model in the production block and a translation model in the translation block are trained simultaneously by machine learning methods.

2. The system according to claim 1 , wherein the production block comprises a sign language identification method module for language identification.

3. The system according to claim 1 , wherein the at least one communication-capable device is in communication with the another communication-capable device through a communication protocol.

4. The system according to claim 1 , wherein a first output from the production model trains the translation model while a second output from the translation model trains the production model.

5. A method for bidirectional automatic sign language production, the method comprising:

receiving an audio feed;

converting the received audio feed into another data format for a sign language;

generating a sequence of poses to refer to the sign language having been translated from the audio feed;

storing the sequence of poses in a pose sequence buffer, wherein a check is performed to determine when the pose sequence buffer is empty, and wherein an end-of-pose signal is generated to indicate an end of the sequence of poses in the pose sequence buffer; and

displaying the sequence of poses as the sign language;

wherein machine learning methods are utilized to optimize generating the sequence of poses.

6. The method according to claim 5 , wherein the sequence of poses comprises photorealistic videos, 3D animated videos, or a combination of the photorealistic videos and the 3D animated videos.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (26)