IP Library Granted Patent US 11,881,143
Granted Patent B2
US 11,881,143 · App. 17/499,633 · Granted Jan 23, 2024

Display peak power management for artificial reality systems

Inventors: Nilanjan Goswami (Livermore, CA); Eugene Gorbatov (Sammamish, WA); Steve John Clohset (San Francisco, CA); Michael Yee (Woodinville, WA)
Assignee: Meta Platforms Technologies, LLC
G09G3/2007G06F3/012G06F3/013G09G2320/0686G09G2330/023G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,881,143
App. No.
17/499,633
Granted
Jan 23, 2024
Kind
B2
Abstract

In particular embodiments, a computing system of a device may determine a display peak power budget allocated for a display component of the device. The system may determine display information including display workload and display telemetry associated with the display component. The system may determine, in accordance with a display peak power management policy applied to the display peak power budget and the display information, one or more display-controlling parameters for maintaining the display component to operate within the display peak power budget. The system may determine, based on the one or more display-controlling parameters, a plurality of grayscales for a plurality of regions on a display screen of the device. The system may adjust a rendered frame based on the plurality of grayscales and output the adjusted rendered frame on the display screen of the device.

Claims (67)

1. A method comprising, by a computing system of a device:

determining a display peak power budget allocated for a display component of the device;

determining display information comprising display workload and display telemetry associated with the display component;

prior to performing eye tracking, determining, based on the display peak power budget allocated for the display component of the device and the display information, a plurality of display-controlling parameters comprising instructions for the display component of the device on how to render a display at run time to meet the display peak power budget, wherein the plurality of display-controlling parameters, determined prior to performing the eye tracking, defines (1) particular size of a fovea region of an eye of a user at run time, (2) a particular number of periphery regions at run time, and (3) particular grayscale values for pixels in the periphery regions in the particular number of periphery regions at run time;

determining a current fovea region and a plurality of periphery regions by performing the eye tracking;

adjusting, at run time, the plurality of periphery regions, determined by performing the eye tracking, to the particular number of periphery regions defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

simultaneously adjusting at run time (1) the current fovea region, determined by performing the eye tracking, from a first size to a second size based on the particular size of the fovea region defined by the plurality of display-controlling parameters determined prior to performing the eye tracking and (2) grayscale values for the pixels in adjusted plurality of periphery regions based on the particular grayscale values defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

adjusting a rendered display frame based on the adjusted fovea region and the adjusted grayscale values for the pixels in the adjusted plurality of periphery regions; and

outputting the adjusted rendered display frame on a display screen of the device.

2. The method of claim 1 , wherein the display screen comprises a plurality of regions, wherein the plurality of regions comprises the adjusted fovea region and the adjusted plurality of periphery regions.

3. The method of claim 1 , wherein the plurality of display-controlling parameters are determined in accordance with a display peak power management policy applied to the display peak power budget and the display information, and wherein the plurality of display-controlling parameters further defines:

corresponding sizes of the particular number of periphery regions; and

grayscale values for pixels associated with the fovea region.

4. The method of claim 1 , wherein grayscale values for pixels associated with the adjusted fovea region are relatively higher than the grayscale values for the pixels associated with each of the periphery regions of the adjusted plurality of periphery regions.

5. The method of claim 1 , wherein the grayscale values for pixels associated with each periphery region of the adjusted plurality of periphery regions are different.

6. The method of claim 1 , wherein adjusting, at run time, the grayscale values for the pixels in the adjusted plurality of periphery regions comprises:

reducing color intensities of the pixels in the adjusted plurality of periphery regions.

7. The method of claim 1 , wherein performing the eye tracking comprises:

estimating a current gaze of the eye; and

determining the current fovea region and the plurality of periphery regions based on the current gaze of the eye.

8. The method of claim 1 , wherein adjusting, at run time, the current fovea region comprises:

reducing a size of the current fovea region from the first size to the second size.

9. The method of claim 1 , wherein:

the device is an augmented reality device.

10. The method of claim 1 , wherein determining the display peak power budget comprises:

analyzing a current state of battery charge, a status of battery wear out, and a temperature of the device;

estimating one or more of battery voltage or resistance based on the current state of battery charge, the status of battery wear out, and the temperature; and

allocating the display peak power budget to the display component of the device based on the estimated battery voltage or resistance.

11. The method of claim 1 , wherein the display telemetry comprises one or more of:

average grayscale values associated with content currently being displayed by the display component;

an estimated head pose of the user;

an estimated eye position of the user;

a fill rate of the display component; or

a frequency of head movement.

12. The method of claim 1 , further comprising:

receiving updated display peak power budget and updated display information; and

updating one or more display-controlling parameters of the plurality of display-controlling parameters based on the updated display peak power budget and the updated display information.

13. The method of claim 1 , wherein pixels in the adjusted fovea region are illuminated at a maximum luminance level allowed by the display peak power budget.

14. The method of claim 1 , wherein the adjusted fovea region substantially covers augmented reality content without comprise in image quality.

15. One or more computer-readable non-transitory storage media embodying software that is operable when executed, by one or more processors, to:

determine a display peak power budget allocated for a display component of a device;

determine display information comprising display workload and display telemetry associated with the display component;

prior to performing eye tracking, determine, based on the display peak power budget allocated for the display component of the device and the display information, a plurality of display-controlling parameters comprising instructions for the display component of the device on how to render a display at run time to meet the display peak power budget, wherein the plurality of display-controlling parameters, determined prior to performing the eye tracking, defines (1) particular size of a fovea region of an eye of a user at run time, (2) a particular number of periphery regions at run time, and (3) particular grayscale values for pixels in the periphery regions in the particular number of periphery regions at run time;

determine a current fovea region and a plurality of periphery regions by performing the eye tracking;

adjust, at run time, the plurality of periphery regions, determined by performing the eye tracking, to the particular number of periphery regions defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

simultaneously adjust at run time (1) the current fovea region, determined by performing the eye tracking, from a first size to a second size based on the particular size of the fovea region defined by the plurality of display-controlling parameters determined prior to performing the eye tracking and (2) grayscale values for the pixels in adjusted plurality of periphery regions based on the particular grayscale values defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

adjust a rendered display frame based on the adjusted fovea region and the adjusted grayscale values for the pixels in the adjusted plurality of periphery regions; and

output the adjusted rendered display frame on a display screen of the device.

16. The one or more computer-readable non-transitory storage media of claim 15 , wherein the display screen comprises a plurality of regions, wherein the plurality of regions comprises the adjusted fovea region and the adjusted plurality of periphery regions.

17. The one or more computer-readable non-transitory storage media of claim 15 , wherein the plurality of display-controlling parameters are determined in accordance with a display peak power management policy applied to the display peak power budget and the display information, and wherein the plurality of display-controlling parameters further defines:

corresponding sizes of the particular number of periphery regions; and

grayscale values for pixels associated with the fovea region.

18. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

determine a display peak power budget allocated for a display component of a device;

determine display information comprising display workload and display telemetry associated with the display component;

prior to performing eye tracking, determine, based on the display peak power budget allocated for the display component of the device and the display information, a plurality of display-controlling parameters comprising instructions for the display component of the device on how to render a display at run time to meet the display peak power budget, wherein the plurality of display-controlling parameters, determined prior to performing the eye tracking, defines (1) particular size of a fovea region of an eye of a user at run time, (2) a particular number of periphery regions at run time, and (3) particular grayscale values for pixels in the periphery regions in the particular number of periphery regions at run time;

determine a current fovea region and a plurality of periphery regions by performing the eye tracking;

adjust, at run time, the plurality of periphery regions, determined by performing the eye tracking, to the particular number of periphery regions defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

simultaneously adjust at run time (1) the current fovea region, determined by performing the eye tracking, from a first size to a second size based on the particular size of the fovea region defined by the plurality of display-controlling parameters determined prior to performing the eye tracking and (2) grayscale values for the pixels in adjusted plurality of periphery regions based on the particular grayscale values defined by the plurality of display-controlling parameters determined prior to performing the eye tracking;

adjust a rendered display frame based on the adjusted fovea region and the adjusted grayscale values for the pixels in the adjusted plurality of periphery regions; and

output the adjusted rendered display frame on a display screen of the device.

19. The system of claim 18 , wherein the display screen comprises a plurality of regions, wherein the plurality of regions comprises the adjusted fovea region and the adjusted plurality of periphery regions.

20. The system of claim 18 , wherein the plurality of display-controlling parameters are determined in accordance with a display peak power management policy applied to the display peak power budget and the display information, and wherein the plurality of display-controlling parameters further defines:

corresponding sizes of the particular number of periphery regions; and

grayscale values for pixels associated with the fovea region.

Assignments (2)
CHANGE OF NAME Recorded Jul 6, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060591/0848 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 2, 2021
From: GOSWAMI, NILANJAN; GORBATOV, EUGENE; CLOHSET, STEVE JOHN; YEE, MICHAEL
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 057999/0871 →