IP Library › Granted Patent US 8,436,941
Granted Patent B2
US 8,436,941 · App. 12/479,198 · Granted May 7, 2013

Information presenting device and information presenting method

Inventors: Tetsujiro Kondo (Tokyo, JP); Kazutaka Uchida (Tokyo, JP); Ryotaku Hayashi (Saitama, JP)
Assignee: Sony Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,436,941
App. No.
12/479,198
Granted
May 7, 2013
Kind
B2
Abstract

An information presenting device includes: a first image generating unit configured to extract, from video data input by way of an input unit, a portion of the video data, and generate multiple main images; a second image generating unit configured to generate multiple sub-images from information correlated to the video data; and an image output unit configured to generate a display image, wherein the plurality of main images are placed in time sequence, and each of the plurality of sub-images are displayed in time sequence corresponding to the main images.

Claims (50)

1. An information presenting device comprising:

a first image generating unit configured to extract, from video data input by way of an input unit, a portion of said video data, and generate a plurality of main images;

a second image generating unit configured to generate multiple sub-images from information correlated to said video data; and

an image output unit configured to generate a display image, wherein said plurality of main images are placed in time sequence, and each of said plurality of sub-images are displayed in time sequence corresponding to said main images; and

a current image generating unit configured to generate a small image of a display size smaller than the display size of one of said plurality of main images, which is at a current position, generated by said first image generating unit, the display size of the small image being proportionately smaller with respect to an increasing distance from the one of said plurality of main images at said current position, the display size of the small image increasing proportionately, with respect to a decreasing distance, as the small image moves toward said current position, wherein

the image output unit comprises: a virtual space generating unit configured to generate a main image object wherein said plurality of main images are arrayed in time sequence, generate sub-image objects from each of the plurality of sub-images, and generate said virtual space by placing said plurality of sub-image objects in time sequence, above said main image object and in parallel with the placement direction of said main images; and a rendering unit configured to set a viewpoint between said main image object and said plurality of sub-image objects, so as to generate said display image from said virtual space, such that said viewpoint moves from the present toward the future in a manner parallel to the array direction of said main images as time progress.

2. The information presenting device according to claim 1 , wherein said image output unit

generates a first virtual object in accordance with a metaphor in which said plurality of main images are represented as a road viewed through a car windshield,

generates a second virtual object in accordance with a metaphor in which said plurality of sub-images are represented as road signs viewed through a car windshield, and

generates said display image from a virtual space including said first and second virtual objects.

3. The information presenting device according to claim 1 , said second image generating unit further comprising:

a caption image generating unit configured to generate caption images as said sub-images, based on caption information included in the video data input to said input unit.

4. The information presenting device according to claim 1 , said second image generating unit further comprising:

a superimposed text image detecting unit configured to detect superimposed text images as said sub-images, from the video data input to said input unit.

5. The information presenting device according to claim 1 , said second image generating unit further comprising:

a face image detecting unit configured to detect face images as said sub-images, from the video data input to said input unit.

6. The information presenting device according to claim 1 , said second image generating unit further comprising:

an inset screen detecting unit configured to detect an image displayed in an insert screen region which has been composited in a superimposed manner on the image of said video data, as a sub-image, from the video data input to said input unit.

7. The information presenting device according to claim 1 , in which a plurality of said rendering unit is provided, said information presenting device further comprising:

an image compositing configured to composite a plurality of images including said main image object and said plurality of sub-image objects supplied from each of the rendering units, so as to generate a single display image.

8. The information presenting device according to claim 1 , said second image generating unit further comprising:

a caption image generating unit configured to

generate a caption image as a first sub-image, based on caption information included in the video data input to said input unit

calculate a position in said caption image which corresponds to a content of speech in said main image at said current position, and

output said caption image in a different form with said current position as a base point; and

an audio waveform image generating unit configured to

generate an audio waveform image of the audio data, input to said input unit along with said video data, as a second sub-image,

calculate a position in said audio waveform image corresponding to the content of speech in said main image at said current position, and

output said audio waveform image in a different form with said current position as a base point;

wherein said image output unit generates said display image by compositing each of said plurality of main images, said caption image, and said audio waveform image, so as to be placed in time sequence of past, present, and future, with said current position as a base point.

9. The information presenting device according to claim 8 , wherein said caption image generating unit changes the display color of captions corresponding to the past and captions corresponding to the future, with said current position as a base point;

and wherein said audio waveform image generating unit changes the display color of audio waveforms corresponding to the past and audio waveforms corresponding to the future, with said current position as a base point;

10. The information presenting device according to claim 9 , said second image generating unit including the current image generating unit,

wherein said image output unit places the small image generated at said current image generating unit, at said current position within said caption image output form said caption image generating unit.

11. The information presenting device according to claim 1 , wherein said image output unit generates a display image in which, of said plurality of main images, a main image of interest and an image further ahead in the temporal direction are arrayed.

12. The information presenting device according to claim 11 , wherein said image output unit generates a display image in which, of said plurality of main images, a main image of interest and an image already behind in the temporal direction are arrayed.

13. The information presenting device according to claim 1 , wherein said image output unit generates a display image in which, of said plurality of main images, a main image of interest and an image already behind in the temporal direction are arrayed.

14. The information presenting device according to claim 1 , wherein the video data corresponds to a television broadcast of a news program.

15. The information presenting device according to claim 1 , wherein the one of said plurality of main images corresponds to a cropped face image of a news anchor.

16. The information presenting device according to claim 1 , wherein one of said plurality of sub-images corresponds to a caption for a television broadcast.

17. The information presenting device according to claim 16 , wherein the caption, when positioned at a foreground position, corresponds to words spoken by a person on the television broadcast and when positioned at one or more background positions, corresponds to words that will be spoken by the person at one or more future points in time.

18. An information presenting method comprising the steps of:

extracting, from video data, a portion of said video data, and generating multiple main images;

generating multiple sub-images from information correlated to said video data;

generating a display image, wherein said plurality of main images are placed in time sequence, and each of said plurality of sub-images are displayed in time sequence corresponding to said main images;

generating a small image of a display size smaller than the display size of one of said plurality of main images, which is at a current position, the display size of the small image being proportionately smaller with respect to an increasing distance from the one of said plurality of main images at said current position, the display size of the small image increasing proportionately, with respect to a decreasing distance, as the small image moves toward said current position;

generating a main image object wherein said plurality of main images are arrayed in time sequence;

generate sub-image objects from each of the plurality of sub-images;

generating said virtual space by placing said plurality of sub-image objects in time sequence, above said main image object and in parallel with the placement direction of said main images; and

setting a viewpoint between said main image object and said plurality of sub-image objects, so as to generate said display image from said virtual space, such that said viewpoint moves from the present toward the future in a manner parallel to the array direction of said main images as time progress.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 24, 2009
From: KONDO, TETSUJIRO; UCHIDA, KAZUTAKA; HAYASHI, RYOTAKU
To: SONY CORPORATION
Reel/Frame 023139/0265 →
Priority Claims (2)
JP P2008-150962 · Jun 9, 2008 · national
JP P2009-132489 · Jun 1, 2009 · national
Continuity (1)
Related Publication 20090310021A1 · Dec 17, 2009