IP Library Granted Patent US 10,169,846
Granted Patent B2
US 10,169,846 · App. 15/087,629 · Granted Jan 1, 2019

Selective peripheral vision filtering in a foveated rendering system

Inventors: Jeffrey Roger Stafford (Redwood City, CA); Andrew Young (San Mateo, CA)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
G06T5/00G02B27/017G02B27/0179G06F1/163G06F1/1694G06F3/011G06F3/012G06F3/013G06F3/0304G06T17/20G02B2027/014G02B2027/0118G02B2027/0147G02B2027/0187G06T2207/20024G06T2207/20192G06T2207/20221G06T2210/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,169,846
App. No.
15/087,629
Granted
Jan 1, 2019
Kind
B2
Abstract

Gaze tracking data representing a viewer's gaze with respect to one or more images presented to the viewer is used to generate foveated image data representing one or more foveated images characterized by a higher level of detail within one or more regions of interest and a lower level of detail outside the regions of interest. The image data for portions outside the one or more regions of interest is selectively filtered to reduce visual artifacts due to contrast resulting from the lower level of detail before compositing foveated images for presentation.

Claims (27)

1. A graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to a viewer;

generating foveated digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of detail for one or more portions of the foveated images outside the one or more regions of interest;

filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail, wherein one or more objects within the one or more foveated images but outside the regions of interest are tagged with information reflecting a relative importance of the one or more objects by automatically tagging objects in the one or more foveated images that are moving as important based on motion vectors computed from the digital image data; and

compositing the one or more filtered foveated images; and

displaying the one or more filtered foveated images to the viewer on a display device.

2. The method of claim 1 , wherein filtering the digital image data for the at least the one or more portions of the one or more foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail includes selectively filtering of the one or more foveated images outside the one or more regions of interest that correspond to selected objects to maintain or increase the contrast.

3. The method of claim 1 , wherein important objects outside the one or more regions of interest are subjected to less filtering to reduce artifacts than other portions of the one or more images outside the one or more regions of interest.

4. The method of claim 1 , wherein tagging the one or more objects includes associating metadata with the one or more objects.

5. The method of claim 4 , wherein associating the metadata with the one or more objects is done subjectively through an application programming interface (API).

6. The method of claim 4 , wherein associating the metadata with the one or more objects is done objectively by an algorithm.

7. The method of claim 1 , wherein filtering the digital image data includes inserting filtering information into the digital image data.

8. The method of claim 7 , wherein filtering the digital image data includes inserting the filtering information into the digital image data by inserting filtering data into a color buffer via the alpha channel, depth buffer, stencil buffer, or an auxiliary buffer for each pixel in the one or more foveated images.

9. The graphics processing method of claim 1 wherein filtering the image data comprises applying an anti-aliasing filter to one or more portions of the one or more foveated images outside the one or more regions of interest.

10. A system, comprising:

a processor;

a memory; and

computer-readable instructions embodied in the memory, the computer-readable instructions being configured to implement a graphics processing method when executed, the graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to a viewer;

generating digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of detail for one or more portions of the foveated images outside the one or more regions of interest;

generating filtered digital image data by filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail, wherein one or more objects within the one or more foveated images but outside the regions of interest are tagged with information reflecting a relative importance of the one or more objects by automatically tagging objects in the one or more foveated images that are moving as important based on motion vectors computed from the digital image data; and

displaying a final composited image comprised of one or more filtered foveated images to the viewer on a display device.

11. A non-transitory computer-readable medium having computer-readable instructions embodied therein, the computer-readable instructions being configured to implement a graphics processing method when executed, the graphics processing method comprising:

obtaining gaze tracking data representing a viewer's gaze with respect to one or more images presented to a viewer;

generating digital image data representing one or more foveated images using the gaze tracking data, wherein the one or more foveated images are generated with a higher level of detail within one or more regions of interest than a lower level of detail for one or more portions of the foveated images outside the one or more regions of interest;

generating filtered digital image data by filtering the digital image data for the at least the one or more portions of the foveated images outside the one or more regions of interest to reduce visual artifacts due to contrast resulting from the lower level of detail, wherein one or more objects within the one or more foveated images but outside the regions of interest are tagged with information reflecting a relative importance of the one or more objects by automatically tagging objects in the one or more foveated images that are moving as important based on motion vectors computed from the digital image data; and

displaying a final composited image comprised of one or more filtered foveated images to the viewer on a display device.

Assignments (2)
CHANGE OF NAME Recorded Jul 6, 2016
From: SONY COMPUTER ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 039276/0023 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2016
From: YOUNG, ANDREW; STAFFORD, JEFFREY ROGER
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 038415/0083 →
Continuity (1)
Related Publication 20170287112A1 · Oct 5, 2017
Cited By (2)
US 12,243,186 US 12,572,204