IP Library Granted Patent US 10,739,849
Granted Patent B2
US 10,739,849 · App. 16/235,727 · Granted Aug 11, 2020

Selective peripheral vision filtering in a foveated rendering system

Inventors: Jeffrey Roger Stafford (Redwood City, CA); Andrew Young (San Mateo, CA)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
G06F3/012G02B27/017G02B27/0179G06F1/163G06F1/1694G06F3/011G06F3/013G06F3/0304G06T17/20G02B2027/014G02B2027/0118G02B2027/0147G02B2027/0187G06K9/00597G06T2210/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,739,849
App. No.
16/235,727
Granted
Aug 11, 2020
Kind
B2
Abstract

Gaze tracking data representing a viewer's gaze with respect to one or more images presented to the viewer is used to generate foveated image data representing one or more foveated images characterized by a higher level of detail within one or more regions of interest and a lower level of detail outside the regions of interest. The image data for portions outside the one or more regions of interest is selectively filtered to reduce visual artifacts due to contrast resulting from the lower level of detail before compositing foveated images for presentation.

Claims (34)

1. A graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to a viewer;

generating foveated digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of detail for one or more portions of the foveated images outside the one or more regions of interest;

filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail by reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest, wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes using information about a relative importance of objects outside the one or more region to apply a filter to the digital image data; and

compositing the one or more filtered foveated images; and

displaying the one or more filtered foveated images to the viewer on a display device.

2. The method of claim 1 , wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes changing a color mapping used in the low resolution peripheral region to compress the contrast in a color space in the at least the one or more portions of the foveated images outside the one or more regions of interest.

3. The method of claim 1 , wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes filtering the image with a filter to reduce the contrast in the at least the one or more portions of the foveated images outside the one or more regions of interest.

4. The method of claim 1 , wherein information about the relative importance of objects is stored in a stencil buffer.

5. The method of claim 1 , wherein the filter is a bilinear filter.

6. The method of claim 1 , wherein the filter is a Gaussian filter.

7. The method of claim 1 wherein filtering the image data comprises applying an anti-aliasing filter to one or more portions of the one or more foveated images outside the one or more regions of interest.

8. The method of claim 1 , wherein filtering the digital image data includes inserting filtering information into the image data.

9. The method of claim 8 , wherein filtering the digital image data includes inserting the filtering information into the digital image data by inserting filtering data into a color buffer via the alpha channel, depth buffer, stencil buffer, or an auxiliary buffer for each pixel in the one or more foveated images.

10. A system, comprising:

a processor;

a memory; and

computer-readable instructions embodied in the memory, the computer-readable instructions being configured to implement a graphics processing method when executed, the graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to the viewer;

generating digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of detail for one or more portions of the foveated images outside the one or more regions of interest;

generating filtered digital image data by filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail by reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest, wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes using information about a relative importance of objects outside the one or more region to apply a filter to the digital image data; and

displaying a final composited image comprised of one or more filtered foveated images to the viewer on a display device.

11. The system of claim 10 , wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes changing a color mapping used in the low resolution peripheral region to compress the contrast in a color space in the at least the one or more portions of the foveated images outside the one or more regions of interest.

12. The system of claim 10 , wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes filtering the image with a filter to reduce the contrast in the at least the one or more portions of the foveated images outside the one or more regions of interest.

13. The system of claim 10 , wherein the filter is a bilinear filter.

14. The system of claim 10 , wherein the filter is a Gaussian filter.

15. The system of claim 10 wherein filtering the image data comprises applying an anti-aliasing filter to one or more portions of the one or more foveated images outside the one or more regions of interest.

16. The system of claim 10 , wherein filtering the digital image data includes inserting filtering information into the image data.

17. The system of claim 16 , wherein filtering the digital image data includes inserting the filtering information into the digital image data by inserting filtering data into a color buffer via the alpha channel, depth buffer, stencil buffer, or an auxiliary buffer for each pixel in the one or more foveated images.

18. A non-transitory computer-readable medium having computer-readable instructions embodied therein, the computer-readable instructions being configured to implement a graphics processing method when executed, the graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to the viewer;

generating digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of to detail for one or more portions of the foveated images outside the one or more regions of interest;

generating filtered digital image data by filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail by reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest, wherein reducing scintillation, aliasing and high contrast pixels in the at least the one or more portions of the foveated images outside the one or more regions of interest includes using information about a relative importance of objects outside the one or more region to apply a filter to the digital image data; and

displaying a final composited image comprised of one or more filtered foveated images to the viewer on a display device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 30, 2019
From: STAFFORD, JEFFREY ROGER; YOUNG, ANDREW
To: SONY COMPUTER ENTERTAINMENT INC.
Reel/Frame 048196/0640 →
CHANGE OF NAME Recorded Jan 30, 2019
From: SONY COMPUTER ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 048196/0699 →
Continuity (2)
Continuation 15087629 · Mar 31, 2016
Related Publication 20190384381A1 · Dec 19, 2019
Cited By (1)
US 12,731,334