IP Library › Granted Patent US 9,762,889
Granted Patent B2
US 9,762,889 · App. 14/262,093 · Granted Sep 12, 2017

Subtitle detection for stereoscopic video contents

Inventors: Chao Huang (Esslingen, DE); Takuto Motoyama (Stuttgart, DE); Yalcin Incesu (Stuttgart, DE)
Assignee: SONY CORPORATION
H04N13/007G06K9/00208G06K9/3266H04N21/4884H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,762,889
App. No.
14/262,093
Granted
Sep 12, 2017
Kind
B2
Abstract

A right image and a left image is provided. Then a disparity estimation map relating to both images is provided and a left subtitle detection map for the left image and a right subtitle detection map for the right image are generated. Each map indicates subtitle areas within an image. For said subtitle areas and based on said disparity estimation map a subtitle disparity value for X and Y directions common for all subtitle areas is determined. Said left and right subtitle maps and said subtitle X and Y disparity values are used in an image interpolation process.

Claims (36)

1. A method comprising:

providing a right image and a left image in circuitry,

providing, in the circuitry, a disparity estimation map relating to both images,

generating, using the circuitry, a left subtitle detection map for the left image and a right subtitle detection map for the right image, each map indicating subtitle areas within an image,

for said subtitle areas and based on said disparity estimation map, determining, using the circuitry, a subtitle disparity value for X and Y directions common for all of the subtitle areas,

evaluating, using the circuitry, a consistency between the left subtitle detection map and the right subtitle detection map by comparing image data of the subtitle areas in the right image to image data of the subtitle areas in the left image when the subtitle areas in the right image and the subtitle areas in the left image are shifted relative to each other by an amount determined by the subtitle disparity value for the X and Y directions;

detecting, using the circuitry, false positives in generation of the left subtitle detection map and the right subtitle detection map in accordance with the consistency, and

using said left and right subtitle maps and said subtitle X and Y disparity values in an image interpolation process, using the circuitry.

2. A non-transitory computer-readable recording medium that stores therein a computer program product, which, when executed by a processor, causes the method according to claim 1 to be performed.

3. A device for detecting subtitle areas in 3D image content, the device comprising

circuitry configured to

provide a disparity estimation map relating to a left image and a right image,

detect subtitle text areas in left and right images for providing left and right subtitle detection maps,

a disparity extracting unit adapted to extract a common subtitle disparity X- and Y-value for the detected text areas,

evaluate a consistency between the left subtitle detection map and the right subtitle detection map by comparing image data of the subtitle areas in the right image to image data of the subtitle areas in the left image when the subtitle areas in the right image and the subtitle areas in the left image are shifted relative to each other by an amount determined by the subtitle disparity value for the X and Y directions;

detect false positives in generation of the left subtitle detection map and the right subtitle detection map in accordance with the consistency, and

use said left and right subtitle maps and said subtitle X and Y disparity values in an image interpolation process.

4. The device of claim 3 , wherein the circuitry is configured to generate the subtitle detection map by

applying a text edge detection on said image for providing a binary text edges map indicating text specific edges, and

applying a motion detection on said image and a previous image for providing a still area map indicating areas with motion and without motion.

5. The device of claim 4 , wherein the circuitry is configured to AND gate said binary text edges map and said still area map to provide a binary final edge map.

6. The device of claim 4 , wherein the circuitry is configured to perform said text edge detection by detecting up-down edge pairs in said image indicating up-down changes of luminance level.

7. The device of claim 6 , wherein the circuitry is configured to perform said detecting by applying a gradient filter.

8. The device of claim 7 , wherein the circuitry is configured to classify said luminance level changes.

9. The device of claim 5 , wherein the circuitry is configured to apply a temporal stabilization to said binary final edge map.

10. The device of claim 9 , wherein the circuitry is configured to perform said temporal stabilization by evaluating a continuity of successive binary final edge maps.

11. The device of claim 3 , wherein the circuitry is configured to merge contiguous subtitle areas to rectangular bounding areas.

12. The device of claim 11 , wherein the circuitry is configured to evaluate said bounding areas and discard bounding areas not fulfilling a predetermined criterion.

13. The device of claim 12 , wherein the circuitry is configured such that said criterion is the size of the rectangular bounding area.

14. The device of claim 13 , wherein the circuitry is configured to discard a bounding area as invalid if its area size is smaller than a predetermined value.

15. The device of claim 12 , wherein the circuitry is configured to determine the subtitle disparity value by

evaluating the disparity estimation map in the bounding areas and

determining a common disparity value for X- and Y-direction on the basis of the evaluating.

16. The device of claim 15 , wherein the circuitry is configured to perform the evaluating by applying a histogram analysis of the disparity values in the bounding areas and selecting the disparity value of the highest frequency.

17. The device of claim 15 , wherein

the circuitry is configured such that an output of evaluating the consistency is a binary value indicating a match.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2014
From: HUANG, CHAO; MOTOYAMA, TAKUTO; INCESU, YALCIN
To: SONY CORPORATION
Reel/Frame 032760/0364 →
Priority Claims (1)
EP 13167084 · May 8, 2013 · regional
Continuity (1)
Related Publication 20140333720A1 · Nov 13, 2014