IP Library Granted Patent US 10,943,158
Granted Patent B2
US 10,943,158 · App. 16/516,813 · Granted Mar 9, 2021

Translation and display of text in picture

Inventors: Hakan Mats Englund (Stockholm, SE); Simon Lessing (Malmo, SE)
Assignee: Sony Corporation
G06K9/78G06F40/58G06K9/228G06K9/325G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,943,158
App. No.
16/516,813
Granted
Mar 9, 2021
Kind
B2
Abstract

A method performed by a mobile terminal may include receiving an image that includes text, translating the text into another language and superimposing and displaying the translated text over the received image.

Claims (37)

1. A method of translating text, comprising: performing character recognition on each frame of a video stream; determining, based on the character recognition, a line of text in a first frame of the video stream; determining a word of a first language in the line of text; tracking a position of the word in a second frame of the video stream, wherein the second frame is subsequent to the first frame; translating the word from the first language into a corresponding word from a second language; and displaying the corresponding word from the second language in the second frame of the video stream over the position of the word in the second frame of the video stream.

2. The method of claim 1 , wherein the second language is determined from a user input where a user selects the second language from a plurality of languages, the plurality of languages including the first language and the second language.

3. The method of claim 1 , wherein the first language is determined by an evaluation of the word.

4. The method of claim 1 , wherein the first language is determined from a user input where a user selects the first language from a plurality of languages, the plurality of languages including the first language and the second language.

5. The method of claim 1 , further comprising:

displaying the corresponding word using text that is a same size as the text of the word.

6. The method of claim 1 , further comprising:

storing, in a memory of the mobile device, a first dictionary of words from the first language and a second dictionary of words from the second language.

7. The method of claim 1 , wherein the video stream is a real-time video stream.

8. A mobile device comprising:

a display, a camera, a memory and a processor, wherein the processor is configured to:

perform character recognition on each frame of a video stream;

determine, based on the character recognition, a line of text in a first frame of the video stream;

determine a word of a first language in the line of text;

track a position of the word in a second frame of the video stream, wherein the second frame is subsequent to the first frame;

translate the word from the first language into a corresponding word from a second language; and

display, via the display of the mobile device, the corresponding word from the second language in the second frame of the video stream over the position of the word in the second frame of the video stream.

9. The device of claim 8 , wherein the second language is determined from a user input where a user selects the second language from a plurality of languages, the plurality of languages including the first language and the second language.

10. The device of claim 8 , wherein the first language is determined by an evaluation of the word.

11. The device of claim 8 , wherein the first language is determined from a user input where a user selects the first language from a plurality of languages, the plurality of languages including the first language and the second language.

12. The device of claim 8 , wherein the corresponding word is displayed using text that is approximately the same size as the text of the word.

13. The device of claim 8 , wherein the memory stores a first dictionary of words from the first language and a second dictionary of words from the second language.

14. The device of claim 8 , wherein the video stream is a real-time video stream.

15. A non-transitory computer-readable medium having instructions stored thereon for translating text, which instructions, when executed by a processor of a mobile device, cause the mobile device to perform the steps comprising:

performing character recognition on each frame of a video stream;

determining, based on the character recognition, a line of text in a first frame of the video stream;

determining a word of a first language in the line of text;

tracking a position of the word in a second frame of the video stream, wherein the second frame is subsequent to the first frame;

translating the word from the first language into a corresponding word from a second language; and

displaying the corresponding word from the second language in the second frame of the video stream over the position of the word in the second frame of the video stream.

16. The non-transitory computer-readable medium of claim 15 , wherein the second language is determined from a user input where a user selects the second language from a plurality of languages, the plurality of languages including the first language and the second language.

17. The non-transitory computer-readable medium of claim 15 , wherein the first language is determined by an evaluation of the word.

18. The non-transitory computer-readable medium of claim 15 , further comprising:

displaying the corresponding word using text that is a same size as the text of the word.

19. The non-transitory computer-readable medium of claim 15 , further comprising:

storing, in a memory of the mobile device, a first dictionary of words from the first language and a second dictionary of words from the second language.

20. The non-transitory computer-readable medium of claim 15 , wherein the video stream is a real-time video stream.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2020
From: ENGLUND, HAKAN MATS; LESSING, SIMON
To: SONY ERICSSON MOBILE COMMUNICATIONS AB
Reel/Frame 052926/0760 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2020
From: SONY MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS INC.
Reel/Frame 052927/0130 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2020
From: SONY MOBILE COMMUNICATIONS INC.
To: SONY CORPORATION
Reel/Frame 052927/0181 →
CHANGE OF NAME Recorded Jun 12, 2020
From: SONY ERICSSON MOBILE COMMUNICATIONS AB
To: SONY MOBILE COMMUNICATIONS AB
Reel/Frame 052929/0530 →
Continuity (4)
Division 15694600 · Sep 1, 2017
Continuation 13412827 · Mar 6, 2012
Continuation 11689673 · Mar 22, 2007
Related Publication 20190340474A1 · Nov 7, 2019
Cited By (1)
US 12,675,652