IP Library Granted Patent US 12,141,935
Granted Patent B2
US 12,141,935 · App. 18/133,568 · Granted Nov 12, 2024

Foveated rendering using eye motion

Inventors: Behnam Bastani (Palo Alto, CA); Tianxin Ning (San Mateo, CA); Haomiao Jiang (Cupertino, CA)
Assignee: Meta Platforms Technologies, LLC
G06T3/04G06F3/013G06T1/60G06T11/00G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,141,935
App. No.
18/133,568
Granted
Nov 12, 2024
Kind
B2
Abstract

A method for providing imagery to a user on a display includes receiving eye tracking data. The method also includes determining a current gaze location and a relative distance between the current gaze location and an edge of the display using the eye tracking data. The method also includes defining a first tile centered at the current gaze location and multiple tiles that surround the first tile using the current gaze location and the relative distance between the current gaze location and the edge of the display. The method includes providing a foveated rendered image using the first tile and the multiple tiles.

Claims (32)

1. A system comprising:

a display configured to provide imagery to a user;

an imaging device configured to obtain eye tracking data; and

processing circuitry configured to:

identify, based on at least the eye tracking data, a distance between a gaze location and an edge of the display; and

render a foveated image using a plurality of tiles around a first tile centered at the gaze location at least partially in response to the distance between the gaze location and the edge of the display.

2. The system of claim 1 , wherein the first tile is configured to provide imagery at an image quality greater than or equal to one or more of the plurality of tiles, wherein a fall-off of a fovea region defines a rate of change of image quality of the first tile and the plurality of tiles with increased distance from a center of the first tile.

3. The system of claim 1 , wherein the processing circuitry is configured to increase a size of one or more of the plurality of tiles and the first tile in response to a decrease in a confidence factor of the gaze location.

4. The system of claim 1 , wherein the processing circuitry is configured to define or adjust one or more tiles of the plurality of tiles that are proximate the edge of the display or are in peripheral vision of the user to have a larger size than at least one tile of the plurality of tiles and the first tile that are closer to a centerpoint of the display.

5. The system of claim 1 , wherein one or more tiles of the plurality of tiles or the first tile that are proximate or at the gaze location have a smaller size than one or more tiles that are distal from the gaze location.

6. The system of claim 1 , wherein the processing circuitry is configured to increase a size of one or more tiles of the plurality of tiles and the first tile in response to an increase in a speed of a change of the gaze location.

7. The system of claim 1 , wherein the processing circuitry is configured to provide a first portion of the foveated image via the first tile at a first image quality and a second portion of the foveated image via the plurality of tiles around the first tile at second image qualities, the first image quality greater than the second image qualities.

8. The system of claim 1 , wherein the processing circuitry is configured to render the foveated image at an image quality based on the distance between the gaze location and the edge of the display, the processing circuitry configured to decrease the image quality with respect to decreased distance between the gaze location and the edge of the display.

9. The system of claim 1 , wherein the processing circuitry is configured to determine a fall-off of a fovea region based on the distance between the gaze location and the edge of the display and establish the first tile and the plurality of tiles based on the gaze location and the fall-off of the fovea region.

10. A method, comprising:

obtaining eye tracking data from an imaging device;

identifying a distance between a gaze location and an edge of a display based on at least the eye tracking data; and

rendering a foveated image using a plurality of tiles around a first tile centered at the gaze location at least partially in response to the distance between the gaze location and the edge of the display.

11. The method of claim 10 , wherein the first tile is configured to provide imagery at an image quality greater than or equal to one or more of the plurality of tiles, wherein a fall-off of a fovea region defines a rage of change of image quality of the first tile and the plurality of tiles with increased distance from a center of the first tile.

12. The method of claim 10 , further comprising increasing a size of one or more of the plurality of tiles and the first tile in response to a decrease in a confidence factor of the gaze location.

13. The method of claim 10 , further comprising defining or adjusting one or more tiles of the plurality of tiles that are proximate the edge of the display or are in peripheral vision of the user to have a larger size than at least one tile of the plurality of tiles and the first tile that are closer to a centerpoint of the display.

14. The method of claim 10 , wherein one or more tiles of the plurality of tiles or the first tile that are proximate or at the gaze location have a smaller size than one or more tiles that are distal from the gaze location.

15. The method of claim 10 , further comprising increasing a size of one or more tiles of the plurality of tiles and the first tile in response to an increase in a speed of a change of the gaze location.

16. The method of claim 10 , further comprising providing a first portion of the foveated image via the first tile at a first image quality and a second portion of the foveated image via the plurality of tiles around the first tile at second image qualities, the first image quality greater than the second image qualities.

17. The method of claim 10 , wherein rendering the foveated image comprises rendering the foveated image at an image quality based on the distance between the gaze location and the edge of the display, the image quality decreased with respect to decreased distance between the gaze location and the edge of the display.

18. The method of claim 10 , further comprising determining a fall-off of a fovea region based on the distance between gaze location and the edge of the display and establishing the first tile and the plurality of tiles based on the gaze location and the fall-off of the fovea region.

19. A device, comprising:

processing circuitry configured to:

identify, based on eye tracking data of a user's eye, a distance between a gaze location and an edge of a display;

establish a first tile centered at the gaze location and a plurality of tiles around the first tile at least partially in response to the distance between the gaze location and the edge of the display; and

render a foveated image using the first tile and the plurality of tiles.

20. The device of claim 19 , wherein the device is a head wearable display device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2023
From: BASTANI, BEHNAM; NING, TIANXIN; JIANG, HAOMIAO
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 063502/0993 →
CHANGE OF NAME Recorded May 2, 2023
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 063955/0058 →
Continuity (3)
Continuation 17526681 · Nov 15, 2021
Continuation 16702179 · Dec 3, 2019
Related Publication 20230245261A1 · Aug 3, 2023
Cited By (1)
US 12,380,529