IP Library Granted Patent US 9,426,412
Granted Patent B2
US 9,426,412 · App. 13/819,770 · Granted Aug 23, 2016

Rendering device and rendering method

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,426,412
App. No.
13/819,770
Granted
Aug 23, 2016
Kind
B2
Abstract

A rendering device provides improved rendering responsiveness in multi-window display for rendering scenarios in which display sizes of images vary over time, while also reducing required memory bandwidth. The device comprises: a scenario processor 101 for interpreting a rendering scenario and calculating for each frame period a scale-down ratio for each of a plurality of pictures; a plurality of decoders 107 for decoding encoded data of a plurality of videos; a plurality of first scalers for scaling-down the decoded pictures using the scale-down ratios calculated by the scenario processor 101 ; a memory 106 for storing the scaled-down pictures; a plurality of second scalers 113 for reading the scaled-down pictures from the memory and re-scaling the scaled-down pictures to match the scale-down ratios calculated by the scenario processor for a current frame period; and, a composing unit 115 for composing the re-scaled pictures.

Claims (72)

1. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period; and

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

the rendering scenario includes a scaling function for each of the image objects, the scaling function designating as an argument the scale-down ratio in any of an X-axis direction, a Y-axis direction, and a Z-axis direction,

after n predetermined frame periods have passed, the scenario processor displays each of the image objects at the scale-down ratio designated by the argument of the scaling function,

from when a 1 st frame period passes to when an n− 1 st frame period passes, the scenario processer changes the scale-down ratio in stages in accordance with the number of passed frame periods, and

the target frame is one of frame periods to which the scale-down ratio which is changed in stages is allocated.

2. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period;

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object; and

a plurality of third scalers that are in one-to-one correspondence with the first scalers and are each configured, when a scale-down ratio used by a corresponding first scaler on the acquired image object is smaller than a predetermined scale-down ratio, to create an additionally scaled-down image object, corresponding to the scaled-down image object, using a smaller scale-down ratio than the scale-down ratio used by the corresponding first scaler, and to store both the additionally scaled-down image object and the scaled-down image object in the memory, wherein

each of the additionally scaled-down image objects stored in the memory is read from the memory and re-scaled by the second scalers to create a re-scaled image object, and

each of the scaled-down image objects stored in the memory is only read from the memory and re-scaled by the second scalers to create a re-scaled image object when no additionally scaled-down image object corresponding to the scaled-down image object is stored in the memory.

3. The rendering device in claim 2 , wherein

the additionally scaled-down image object is a mipmap of the acquired image object.

4. The rendering device in claim 3 , wherein

the predetermined scale-down ratio is 3/4.

5. A rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the rendering device comprising:

a scenario processor configured to analyze a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

a plurality of acquiring units each configured to acquire an image object for the frame period;

a plurality of first scalers configured to scale-down the acquired image objects;

a memory configured to store therein the scaled-down image objects;

a plurality of second scalers configured to, when the target frame period is a current frame period, read from the memory the scaled-down image objects and re-scale each of the scaled-down image objects to match a scale-down ratio calculated by the scenario processor for the current frame period; and

a composing unit configured to compose each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

a delay equivalent to a frame period k occurs for combined actions of writing to and reading from the memory each of the scaled-down image objects, and

when one of the acquired image objects is scaled-down by one of the first scalers using a scale-down ratio S 1 at a frame period t-k temporally in the past, one of the second scalers reads from the memory the scaled-down image object scaled-down using the ratio S 1 and re-scales the scaled-down image object for a current frame period t where the scale-down ratio calculated by the scenario processor is S 2 , by using a factor of S 2 /S 1 .

6. The rendering device in claim 5 , wherein

the scale-down ratio used by each of the first scalers is the scale-down ratio calculated by the scenario processor with respect to each of the image objects for the current frame period.

7. The rendering device in claim 5 , wherein

the smaller a shape specified by the rendering scenario for each of the image objects, the smaller a scale-down ratio used by each of the first scalers.

8. The rendering device in claim 5 , wherein

the larger a distortion in an image object pasting shape specified by the rendering scenario for each of the image objects compared to an original shape of the image object, the smaller a scale-down ratio used by each of the first scalers.

9. The rendering device in claim 5 , wherein

the larger a variation over time in an image object pasting position specified by the rendering scenario for each of the image objects, the smaller a scale-down ratio used by each of the first scalers.

10. The rendering device in claim 5 , further comprising

a plurality of image feature detection units each configured to detect a human figure image from an image object, wherein

the smaller the human figure image detected in each of the image objects by the feature detection units, the smaller a scale-down ratio used by each of the first scalers.

11. The rendering device in claim 5 , wherein

the plurality of image objects include at least one pair of viewpoint images to be used in stereoscopic playback, and

a scale-down ratio used by one of the first scalers on one of the pair of viewpoint images is larger than a scale-down ratio used by another one of the first scalers on the other of the pair of viewpoint images.

12. An integrated circuit that is incorporatable into the rendering device in claim 5 , the integrated circuit comprising:

the scenario processor;

the acquiring units;

the first scalers;

the memory;

the second scalers; and

the composing unit.

13. The rendering device in claim 5 , being a television device,

wherein

the acquiring units each include:

a tuner configured to acquire a plurality of pieces of encoded data of a video; and

a decoder configured to acquire a plurality of image objects by decoding the acquired plurality of pieces of encoded data of the videos, and

the television device further comprises

a display configured to display the virtual space video composed by the composing unit using the plurality of image objects.

14. A method for controlling a rendering device that renders a virtual space video of a projected virtual space object varying in at least one of shape and position over a plurality of frame periods, and having a plurality of image objects pasted on a plurality of surfaces of the virtual space object in one-to-one correspondence, the method comprising:

an analyzing step of analyzing a rendering scenario of the virtual space video to calculate a scale-down ratio with respect to each of a plurality of image objects for a target frame period included in the frame periods, the scale-down ratio being for fitting the image object to a region in the virtual space video corresponding to one of the plurality of surfaces of the virtual space object, the target frame period being a period in which the virtual space video is to be displayed;

an acquiring step of acquiring a plurality of image objects for the frame period;

a first scaling step of scaling-down the acquired image objects;

a storing step of storing the scaled-down image objects in a memory of the rendering device;

a second scaling step of, when the target frame period is a current frame period, reading the scaled-down image objects stored in the memory and re-scaling each of the scaled-down image objects to match a scale-down ratio calculated for the current frame period; and

a composing step of composing each of the re-scaled image objects on a region in the virtual space video corresponding to one of the surfaces of the virtual space object, wherein

after n predetermined frame periods have passed, the analyzing step displays each of the image objects at the scale-down ratio designated by an argument of a scaling function,

from when a 1 st frame period passes to when an n− 1 st frame period passes, the analyzing step changes the scale-down ratio in stages in accordance with the number of passed frame periods, and

the target frame is one of frame periods to which the scale-down ratio which is changed in stages is allocated.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE ERRONEOUSLY FILED APPLICATION NUMBERS 13/384239, 13/498734, 14/116681 AND 14/301144 PREVIOUSLY RECORDED ON REEL 034194 FRAME 0143. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 24, 2020
From: PANASONIC CORPORATION
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 056788/0362 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2014
From: PANASONIC CORPORATION
To: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Reel/Frame 034194/0143 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2013
From: NISHIDA, HIDESHI
To: PANASONIC CORPORATION
Reel/Frame 030462/0106 →