IP Library › Granted Patent US 11,748,956
Granted Patent B2
US 11,748,956 · App. 17/500,281 · Granted Sep 5, 2023

Device and method for foveated rendering

Inventors: Kota Kitamura (Tokyo, JP); Goro Sakamaki (Tokyo, JP); Tomoo Minaki (Tokyo, JP)
Assignee: Synaptics Incorporated
G06T19/006G06T3/0056G06T3/4053G06T7/90G06T15/005G06T15/503
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,748,956
App. No.
17/500,281
Granted
Sep 5, 2023
Kind
B2
Abstract

A display driver includes interface circuitry, image processing circuitry, and drive circuitry. The interface circuitry is configured to receive a full frame image and a foveal image from a source external to the display driver. The image processing circuitry is configured to: upscale the full frame image; render a foveated image from the upscaled full frame image and the foveal image. The foveated image includes a foveal area based on the foveal image, a peripheral are based on the upscaled full frame image, and a border area based on the foveal image and the upscaled full frame image. The border area being located between the foveal area and the peripheral area. The drive circuitry is configured to drive a display panel using the foveated image.

Claims (68)

1. A display driver, comprising:

interface circuitry configured to receive a full frame image and a foveal image from a source external to the display driver;

image processing circuitry configured to:

upscale the full frame image;

render a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image comprises:

a foveal area based on the foveal image;

a peripheral area based on the upscaled full frame image; and

a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and

drive circuitry configured to drive a display panel using the foveated image.

2. The display driver of claim 1 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image.

3. The display driver of claim 2 , wherein a weight is assigned to the first pixel in the weighted sum based on a position of the target pixel in the border area.

4. The display driver of claim 3 , wherein the weight assigned to the first pixel increases as a distance of the target pixel from the foveal area decreases.

5. The display driver of claim 1 , wherein the border area comprises:

a first section that surrounds the foveal area; and

a second section that surrounds the first section,

wherein rendering the foveated image comprises:

determining a graylevel of a first target pixel in the first section based on a first weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image; and

determining a graylevel of a second target pixel in the second section based on a second weighted sum of a graylevel of a third pixel in the foveal image and a graylevel of a fourth pixel in the upscaled full frame image, and

wherein a first weight is assigned to the first pixel in the first weighted sum and a second weight is assigned to the third pixel in the second weighted sum, the first weight being larger than the second weight.

6. The display driver of claim 5 , wherein the border area further comprises a third section that surrounds the second section, and

wherein the rendering of the border area further comprises:

determining a graylevel of a third target pixel in the third section based on a third weighted sum of a graylevel of a fifth pixel in the foveal image and a graylevel of a sixth pixel in the upscaled full frame image,

wherein a third weight is assigned to the fifth pixel in the third weighted sum, the second weight being larger than the third weight.

7. The display driver of claim 1 , further comprising:

a register configured to store a border area dimension data set that indicates one or more dimensions of the border area,

wherein the border area is rendered based on the border area dimension data set.

8. The display driver of claim 7 , wherein the border area dimension data set is rewritable by the source.

9. The display driver of claim 1 , wherein the image processing circuitry is further configured to:

receive, via the interface circuitry, control data from the source; and

modify at least one dimension of the border area based on the control data received from the source.

10. The display driver of claim 1 , further comprising:

register circuitry configured to store a plurality of border area dimension data sets each indicating one or more dimensions of the border area,

wherein the image processing circuitry is further configured to:

receive, via the interface circuitry, control data from the source; and

select one of the plurality of border area dimension data sets based on the control data, wherein the border area is rendered based on the selected one of the plurality of border area dimension data sets.

11. The display driver of claim 1 , wherein the full frame image has the same resolution as the foveal image and encompasses a larger display area than the foveal image.

12. The display driver of claim 1 , wherein the rendering the foveated image comprises rendering the border area by alpha blending respective portions of the upscaled full frame image and the foveal image.

13. The display driver of claim 1 , wherein the interface circuitry, the image processing circuitry, and the drive circuitry are monolithically integrated in a single semiconductor chip.

14. A display system, comprising:

a source configured to render a full frame image and a foveal image;

a display panel; and

a display driver configured to:

receive the full frame image and the foveal image from the source;

upscale the full frame image;

render a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image includes:

a foveal area based on the foveal image;

a peripheral area based on the upscaled full frame image; and

a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and

drive the display panel using the foveated image.

15. The display system of claim 14 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image.

16. The display system of claim 15 , wherein a weight is assigned to the first pixel in the weighted sum based on a position of the target pixel in the border area.

17. The display system of claim 14 , wherein the border area comprises:

a first section that surrounds the foveal area; and

a second section that surrounds the first section,

wherein rendering the foveated image comprises:

determining a graylevel of a first target pixel in the first section based on a first weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image; and

determining a graylevel of a second target pixel in the second section based on a second weighted sum of a graylevel of a third pixel in the foveal image and a graylevel of a fourth pixel in the upscaled full frame image,

wherein a first weight is assigned to the first pixel in the first weighted sum and a second weight is assigned to the third pixel in the second weighted sum, the first weight being larger than the second weight.

18. The display system of claim 14 , wherein the source is configured as a separate integrated circuit from the display driver.

19. A method of operating a display driver, comprising:

receiving a full frame image and a foveal image from a source external to the display driver;

upscaling the full frame image; and

rendering a foveated image from the upscaled full frame image and the foveal image, wherein the foveated image comprises:

a foveal area based on the foveal image;

a peripheral area based on the upscaled full frame image; and

a border area based on the foveal image and the upscaled full frame image, the border area being located between the foveal area and the peripheral area; and

driving a display panel using the foveated image.

20. The method of claim 19 , wherein the rendering of the foveated image comprises determining a graylevel of a target pixel in the border area based on a weighted sum of a graylevel of a first pixel in the foveal image and a graylevel of a second pixel in the upscaled full frame image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: KITAMURA, KOTA; SAKAMAKI, GORO; MINAKI, TOMOO
To: SYNAPTICS INCORPORATED
Reel/Frame 057780/0990 →
Continuity (1)
Related Publication 20230111826A1 · Apr 13, 2023
Cited By (1)
US 12,585,331