Granted Patent B1

US 10,176,366 · App. 15/801,190 · Granted Jan 8, 2019

Abstract

Claims (45)

1. A video relay service for providing automatic translation services during a real-time communication session, the video relay service comprising:

at least one server configured to:

receive a video stream containing sign language content from a video communication device associated with a hearing-impaired user;

automatically translate the sign language content into a verbal language translation during the real-time communication session without assistance of a human sign language interpreter by comparing data chunks including multiple frames of the video stream with stored video files in an artificial intelligence (AI) database; and

transmit the verbal language translation to a far-end communication device associated with a hearing-capable user during the real-time communication session.

2. The video relay service of claim 1 , wherein the at least one server is configured to transmit the verbal language translation as synthesized audio.

3. The video relay service of claim 1 , wherein the at least one server configured to receive new entries to add stored video files to the AI database from the video communication device that are available to other users during subsequent communication sessions facilitated by the video relay service.

4. The video relay service of claim 1 , wherein the at least one server is further configured to:

record portions of the communication sessions facilitated by the video relay service; and

add new video files to the stored video files to the AI database generated from the recorded portions of the communication sessions by a training station that are available to other users during subsequent communication sessions facilitated by the video relay service.

5. The video relay service of claim 1 , wherein the at least one server is further configured to:

receive an audio stream from the far-end communication device during the real-time communication session;

automatically translate voice content from the audio stream into another language translation during the real-time communication session; and

transmit the another language translation to the video communication device associated with the hearing-impaired user during the real-time communication session.

6. The video relay service of claim 5 , wherein the another language translation is first generated in a textual format generated by a speech-to-text voice recognition system.

7. The video relay service of claim 6 , wherein the at least one server is configured to transmit the another language translation in the textual format.

8. The video relay service of claim 6 , wherein the at least one server is configured to transmit the another language translation in simulated avatar format depicting the another language translation in sign language without assistance of a human sign language interpreter.

9. The video relay service of claim 6 , wherein the at least one server is configured to transmit the another language translation in both the textual format and a simulated avatar format.

10. The video relay service of claim 6 , wherein the at least one server is configured to transmit the verbal language translation to back to the video communication device associated with a hearing-impaired user during the real-time communication session.

11. The video relay service of claim 6 , wherein the verbal language translation and the another is language translation are the same language.

12. The video relay service of claim 6 , wherein the verbal language translation and the another is language translation are different languages.

13. A method of performing automated translation services during a real-time communication session, the method comprising:

establishing a communication session between a video communication device associated with a hearing-impaired user and a far-end communication device associated with a hearing-capable user through an artificial intelligence (AI) translation engine;

automatically generating, with the AI translation engine, a first translation of sign language content from a video stream from the video communication device without a human sign language interpreter performing the sign language translation for the communication session by dividing the video stream into data chunks of different lengths depending on phrases detected in the sign language content, and comparing the data chunks to stored video files in an AI translation database; and

transmitting the first translation from the AI translation engine to the far-end communication device during the real-time communication session.

14. The method of claim 13 , further comprising:

automatically generating, with the AI translation engine, a second translation of voice content from an audio stream from the far-end communication device; and

transmitting the second translation from the AI translation engine to the video communication device during the real-time communication session.

15. The method of claim 13 , wherein transmitting the first translation from the AI translation engine to the far-end communication device includes synthesized audio of the first translation.

16. The method of claim 13 , further comprising transmitting the first translation from the AI translation engine to the video communication device during the real-time communication session.

17. The method of claim 16 , wherein transmitting the first translation from the AI translation engine to the video communication device includes textual content of the first translation.

18. A video communication system comprising:

a plurality of video communication devices configured for hearing-impaired users to engage in communication sessions with hearing-capable users; and

a video relay service configured to:

establish communication sessions between video communication devices associated with hearing-impaired users and far-end communication device associated with hearing-capable users;

automatically generate translations of sign language content from a video stream from a corresponding video communication device associated with a hearing-impaired user during real-time communication sessions without a human sign language interpreter performing the sign language translation for the communication session by comparing data chunks including multiple frames of the video stream with stored video files in an artificial intelligence (AI) database of an AI translation engine;

transmit the translation from the AI translation engine to a corresponding far-end communication device associated with a hearing-capable user;

automatically generate with the AI translation engine, a second translation of voice content from an audio stream from the corresponding far-end communication device; and

transmit the second translation from the AI translation engine to the video communication device during the real-time communication session.

19. The video relay service of claim 1 , wherein the at least one server is configured to detect a pause between a word or phrase to extract the data chunks from the video stream for comparison with the stored video files in the AI database.

20. The video relay service of claim 1 , wherein a size of the data chunks are hard coded by the at least one server.

21. The video relay service of claim 1 , wherein the at least one server is configured to determine the size of the data chunks based, at least in part, on determining a signing speed for the hearing-impaired user.

22. The video relay service of claim 1 , wherein the at least one server is configured to continuously form the multiple frames of data into the data chunks on a rolling basis.

23. The video relay service of claim 1 , wherein the at least one server is configured to compare the data chunks first with a selected portion of the AI database responsive to determining contextual relationships from surrounded translated words or phrases.

24. The method of claim 13 , wherein the different lengths of the data chunks are dynamically changing during the communication session.

Assignments (11)