IP Library Granted Patent US 9,529,427
Granted Patent B2
US 9,529,427 · App. 14/072,922 · Granted Dec 27, 2016

System and method of image rendering

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,529,427
App. No.
14/072,922
Granted
Dec 27, 2016
Kind
B2
Abstract

A method of rendering an image based upon a first stereoscopic image comprising a pair of images is provided. The method includes generating a virtual three-dimensional model of the scene depicted in the first stereoscopic image responsive to distances derived from the first stereoscopic image, detecting one or more free edges in the three dimensional model, and generating one or more textures for the virtual three-dimensional model from at least one of the pair of images of the first stereoscopic image. The method also includes applying at least one texture to a respective part of the three dimensional model, and rendering the virtual three dimensional model from a different viewpoint to that of the first stereoscopic image. Rendering the virtual three dimensional model comprises modifying a transparency of rendered pixels of an applied texture as a function of the pixel's distance from that free edge.

Claims (62)

1. A method of rendering an image based upon a first stereoscopic image comprising a pair of images, the method comprising the steps of:

generating a virtual three-dimensional model of a scene depicted in the first stereoscopic image responsive to distances derived from the first stereoscopic image;

detecting one or more free edges in the virtual three dimensional model;

generating, by one or more processors, one or more textures for the virtual three-dimensional model from at least one of the pair of images of the first stereoscopic image;

applying, by the one or more processors, at least one of the one or more textures to a respective part of the virtual three dimensional model; and

rendering, by the one or more processors, the virtual three dimensional model from a different viewpoint to that of the first stereoscopic image;

wherein rendering the virtual three dimensional model comprises modifying a transparency of rendered pixels of an applied texture as a function of each pixel's distance from a given one of the free edges; and

wherein the rendering further includes generation of a texture for at least one polygon comprising color information from foreground and background elements of the scene;

in which generating a virtual three-dimensional model of the scene depicted in the first stereoscopic image comprises:

generating a disparity map from the pair of images of the first stereoscopic image, the disparity map being indicative of distances in the first stereoscopic image;

defining a series of value ranges corresponding to disparity values of the disparity map, each value range in the series having an end point corresponding to a greater disparity than an end point of preceding value ranges in the series;

selecting points in the disparity map falling within the respective value range;

generating a respective mesh responsive to those selected points; and

merging generated meshes to form the 3D model of the scene.

2. A method according to claim 1 , in which the step of detecting one or more free edges in the virtual three dimensional model comprises detecting at least a first polygon with a side that is not shared with another polygon.

3. A method according to claim 1 , in which the step of detecting one or more free edges in the virtual three dimensional model comprises detecting at least a first polygon with a vertex that is not shared with another polygon.

4. A method according to claim 1 , in which the step of modifying the transparency comprises:

generating a gradient of pixel transparency values over a predetermined distance from the given one of the free edges of the virtual three dimensional model such that pixels are more transparent at the edge of the virtual three dimensional model.

5. A method according to claim 1 , in which the step of modifying the transparency comprises:

generating a spline fit to the given one of the free edges of the virtual three dimensional model; and

generating a gradient of pixel transparency values over a predetermined distance from that spline, such that pixels are more transparent at the spline.

6. A method according to claim 1 , comprising the step of modelling an interaction of a virtual object with the virtual three dimensional model; and in which the step of rendering the virtual three dimensional model comprises rendering a visible effect of the interaction of the virtual object with the three dimensional model.

7. A method according to claim 1 , in which the step of rendering the virtual three dimensional model from a different viewpoint comprises the steps of:

tracking a position of a user's head with respect to a display; and

calculating one or more viewpoints for rendering responsive to a deviation of the user's head from a default viewpoint.

8. A non-transitory computer program product comprising computer implementable instructions that when run cause a computer to implement a method of rendering an image based upon a first stereoscopic image comprising a pair of images, the method comprising the steps of:

generating a virtual three-dimensional model of a scene depicted in the first stereoscopic image responsive to distances derived from the first stereoscopic image;

detecting one or more free edges in the virtual three dimensional model;

generating one or more textures for the virtual three-dimensional model from at least one of the pair of images of the first stereoscopic image;

applying at least one of the one or more textures to a respective part of the virtual three dimensional model; and

rendering the virtual three dimensional model from a different viewpoint to that of the first stereoscopic image;

wherein rendering the virtual three dimensional model comprises modifying a transparency of rendered pixels of an applied texture as a function of each pixel's distance from a given one of the free edges; and

wherein the rendering further includes generation of a texture for at least one polygon comprising color information from foreground and background elements of the scene;

in which generating a virtual three-dimensional model of the scene depicted in the first stereoscopic image comprises:

generating a disparity map from the pair of images of the first stereoscopic image, the disparity map being indicative of distances in the first stereoscopic image;

defining a series of value ranges corresponding to disparity values of the disparity map, each value range in the series having an end point corresponding to a greater disparity than an end point of preceding value ranges in the series;

selecting points in the disparity map falling within the respective value range;

generating a respective mesh responsive to those selected points; and

merging generated meshes to form the 3D model of the scene.

9. An entertainment device for rendering an image based upon a first stereoscopic image comprising a pair of images, the entertainment device comprising:

virtual modelling means for generating a virtual three-dimensional model of a scene depicted in the first stereoscopic image, responsive to distances derived from the first stereoscopic image;

model edge detection means for detecting one or more free edges in the virtual three dimensional model;

texture generation means for generating one or more textures for the virtual three-dimensional model from at least one of the pair of images of the first stereoscopic image;

texture application means for applying at least one of the textures to a respective part of the virtual three dimensional model; and

rendering means for rendering the virtual three dimensional model from a different viewpoint to that of the first stereoscopic image;

wherein the rendering means is operable to modify a transparency of rendered pixels of an applied texture as a function of each pixel's distance from a given one of the free edges; and

wherein the rendering means is further operable to generate a texture for at least one polygon comprising color information from foreground and background elements of the scene

in which the virtual modelling means comprises:

disparity map generating means for generating a disparity map from the pair of images of the first stereoscopic image, the disparity map being indicative of distances in the first stereoscopic image;

range setting means for defining a series of value ranges corresponding to disparity values of the disparity map, each value range in the series having an end point corresponding to a greater disparity than an end point of preceding value ranges in the series;

selection means for selecting points in the disparity map falling within the respective value range;

mesh generating means for generating a respective mesh responsive to those selected points; and

mesh merging means for merging generated meshes to form the 3D model of the scene.

10. An entertainment device according to claim 9 , in which the rendering means is operable to generate a gradient of pixel transparency values over a predetermined distance from the given one of the free edges of the virtual three dimensional model such that pixels are more transparent at the edge of the virtual three dimensional model.

11. An entertainment device according to claim 9 , in which the rendering means is operable to generate a spline fit to the given one of the free edges of the virtual three dimensional model, and to generate a gradient of pixel transparency values over a predetermined distance from that spline, such that pixels are more transparent at the spline.

12. An entertainment device according to claim 9 , comprising:

virtual object interaction modelling means for modeling an interaction of a virtual object with the virtual three dimensional model; and in which

the rendering means is operable to render a visible effect of the interaction of the virtual object with the three dimensional model.

13. An entertainment device according to claim 9 , comprising:

input means for head tracking information for a user's head; and

calculating means for calculating one or more viewpoints for rendering responsive to a deviation of the user's head from a default viewpoint; and in which

the one or more viewpoints from which the virtual three dimensional model is rendered is based upon respective calculated viewpoints.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 052167/0398 →
CHANGE OF NAME Recorded Jul 14, 2017
From: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
To: SONY INTERACTIVE ENTERTAINMENT EUROPE LIMITED
Reel/Frame 043198/0110 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 24, 2014
From: RAGHOEBARDAYAL, SHARWIN WINESH; BENSON, SIMON MARK; BICKERSTAFF, IAN HENRY
To: SONY COMPUTER ENTERTAINMENT EUROPE LIMITED
Reel/Frame 032039/0396 →