Managing augmented-reality graphical elements under thermal constraints

1 . A system comprising:

a display comprising a plurality of zones whose luminance is individually controllable;

at least one temperature sensor employed to sense at least one of: a temperature of the display, a temperature of an ambient environment surrounding the display; and

at least one processor configured to:

identify a plurality of graphical elements to be presented in a sequence of images;

for each graphical element, determine a corresponding importance level;

for each graphical element, determine a corresponding set of zones within the display that are to be used to present said graphical element;

for each zone, determine a minimum luminance level and a maximum luminance level, based on at least one graphical element to be presented by said zone, and an importance level of the at least one graphical element;

monitor a power usage of the display;

estimate a thermal load of the display based on the power usage and the at least one of: the temperature of the display, the temperature of the ambient environment;

determine whether the thermal load exceeds a current thermal budget; and

when it is determined that the thermal load exceeds the current thermal budget,

identify first zones that correspond to at least one graphical element having a lowest importance level; and

adjust luminance levels of the first zones to lie between respective minimum luminance levels and respective maximum luminance levels.

2 . The system of claim 1 , wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

determine whether a subsequent thermal load of the display still exceeds the current thermal budget, after adjusting the luminance levels of the first zones; and

when it is determined that the subsequent thermal load still exceeds the current thermal budget,

identify second zones that correspond to at least one graphical element whose importance level is higher than the lowest importance level and lower than a highest importance level; and

adjust luminance levels of the second zones to lie between respective minimum luminance levels and respective maximum luminance levels, until a further subsequent thermal load of the display does not exceed the current thermal budget.

3 . The system of claim 1 , wherein a size of a single zone is larger than a size of a single pixel of the display, wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

identify at least one graphical element from among the plurality of graphical elements that is not constrained to a fixed position in the sequence of images; and

relocate the at least one graphical element to a position proximate to at least one other graphical element from among the plurality of graphical elements, wherein at least one other zone is used to present both the at least one graphical element and the at least one other graphical element.

4 . The system of claim 3 , wherein when relocating, the at least one processor is configured to relocate a plurality of graphical elements to be presented within a predefined portion of a view frustum of the display.

5 . The system of claim 1 , further comprising a tracker and at least one real-world-facing camera, wherein the display is a heads-up display, and wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

determine relative positions of eyes of at least one user with respect to an image plane of the display, using the tracker;

capture, using the at least one real-world-facing camera, at least one real-world image of a real-world environment whose real-world light field is being augmented by the display;

identify at least one portion of the image plane for which a luminance level of a corresponding portion of the real-world light field as projected toward a given eye is lower than a predefined threshold luminance level, based on the at least one real-world image and a relative position of the given eye with respect to the image plane; and

relocate at least one graphical element to be presented via at least one other zone that corresponds to the at least one identified portion of the image plane.

6 . The system of claim 1 , wherein the display is a heads-up display comprising an active optical device arranged on an optical path of a real-world light field of a real-world environment, wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

activate at least one portion of the active optical device to decrease a transmission of a part of the real-world light field passing therethrough toward a given eye of at least one user; and

relocate at least one graphical element to be presented via at least one other zone that corresponds to the at least one portion of the active optical device.

7 . The system of claim 1 , wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget, perform at least one of:

reducing a size of at least one graphical element;

modifying at least one appearance parameter of at least one graphical element.

8 . The system of claim 1 , wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

determine when at least one first zone corresponds to both a first graphical element having a first importance level and a first portion of a second graphical element having a second importance level that is lower than the first importance level, and when the at least one second zone corresponds to a second portion of the second graphical element only;

when it is determined that the at least one first zone corresponds to both the first graphical element and the first portion of the second graphical element, and the at least one second zone corresponds to the second portion of the second graphical element only,

drive the at least one first zone at a first luminance level corresponding to the first importance level and the at least one second zone at a second luminance level corresponding to the second importance level; and

adjust a rendered intensity of the first portion of the second graphical element in the sequence of images, to match an estimated perceived luminance of the first portion of the second graphical element with an estimated perceived luminance of the second portion of the second graphical element.

9 . The system of claim 1 , further comprising a tracker, wherein the at least one processor is configured to:

determine gaze directions of eyes of at least one user, using the tracker;

identify a graphical element at which the at least one user is looking, based on the gaze directions; and

at least temporarily increase an importance level of the identified graphical element.

10 . The system of claim 1 , further comprising at least one real-world-facing camera, wherein the display is a heads-up display, and wherein the at least one processor is configured to:

capture, using the at least one real-world-facing camera, at least one real-world image of a real-world environment whose real-world light field is being augmented by the display;

identify a real-world object that lies on a predicted path of a vehicle in which the system is implemented; and

at least temporarily increase an importance level of a graphical element that corresponds to the identified real-world object.

11 . A method comprising:

sensing, using at least one temperature sensor, at least one of: a temperature of a display, a temperature of an ambient environment surrounding the display;

identifying a plurality of graphical elements to be presented in a sequence of images; for each graphical element, determining a corresponding importance level;

for each graphical element, determining a corresponding set of zones within the display that are to be used to present said graphical element, wherein the display comprises a plurality of zones whose luminance is individually controllable;

for each zone, determining a minimum luminance level and a maximum luminance level, based on at least one graphical element to be presented by said zone, and an importance level of the at least one graphical element;

monitoring a power usage of the display;

estimating a thermal load of the display based on the power usage and the at least one of: the temperature of the display, the temperature of the ambient environment;

determining whether the thermal load exceeds a current thermal budget; and

when it is determined that the thermal load exceeds the current thermal budget,

identifying first zones that correspond to at least one graphical element having a lowest importance level; and

adjusting luminance levels of the first zones to lie between respective minimum luminance levels and respective maximum luminance levels.

12 . The method of claim 11 , further comprising:

when it is determined that the thermal load exceeds the current thermal budget,

determining whether a subsequent thermal load of the display still exceeds the current thermal budget, after adjusting the luminance levels of the first zones; and

when it is determined that the subsequent thermal load still exceeds the current thermal budget,

identifying second zones that correspond to at least one graphical element whose importance level is higher than the lowest importance level and lower than a highest importance level; and

adjusting luminance levels of the second zones to lie between respective minimum luminance levels and respective maximum luminance levels, until a further subsequent thermal load of the display does not exceed the current thermal budget.

13 . The method of the claim 11 , wherein a size of a single zone is larger than a size of a single pixel of the display, the method further comprising:

when it is determined that the thermal load exceeds the current thermal budget,

identifying at least one graphical element from among the plurality of graphical elements that is not constrained to a fixed position in the sequence of images; and

relocating the at least one graphical element to a position proximate to at least one other graphical element from among the plurality of graphical elements, wherein at least one other zone is used to present both the at least one graphical element and the at least one other graphical element.

14 . The method of claim 11 , wherein the step of relocating comprises relocating a plurality of graphical elements to be presented within a predefined portion of a view frustum of the display.

15 . The method of claim 11 , further comprising:

when it is determined that the thermal load exceeds the current thermal budget,

determining when at least one first zone corresponds to both a first graphical element having a first importance level and a first portion of a second graphical element having a second importance level that is lower than the first importance level, and when the at least one second zone corresponds to a second portion of the second graphical element only;

when it is determined that the at least one first zone corresponds to both the first graphical element and the first portion of the second graphical element, and the at least one second zone corresponds to the second portion of the second graphical element only,

driving the at least one first zone at a first luminance level corresponding to the first importance level and the at least one second zone at a second luminance level corresponding to the second importance level; and

adjusting a rendered intensity of the first portion of the second graphical element in the sequence of images, to match an estimated perceived luminance of the first portion of the second graphical element with an estimated perceived luminance of the second portion of the second graphical element.

16 . A system comprising:

a display comprising a plurality of zones whose luminance is individually controllable;

at least one temperature sensor employed to sense at least one of: a temperature of the display, a temperature of an ambient environment surrounding the display; and

at least one processor configured to:

identify a plurality of graphical elements to be presented in a sequence of images; for each graphical element, determine a corresponding importance level;

for each graphical element, determine a corresponding set of zones within the display that are to be used to present said graphical element;

for each zone, determine a minimum luminance level and a maximum luminance level, based on at least one graphical element to be presented by said zone, and an importance level of the at least one graphical element;

monitor a power usage of the display;

estimate a thermal load of the display based on the power usage and the at least one of: the temperature of the display, the temperature of the ambient environment;

determine whether the thermal load exceeds a current thermal budget; and when it is determined that the thermal load exceeds the current thermal budget,

identify first zones that correspond to at least one graphical element having a lowest importance level; and

adjust luminance levels of the first zones to lie between respective minimum luminance levels and respective maximum luminance levels; and

when it is determined that the thermal load exceeds the current thermal budget, determine whether a subsequent thermal load of the display still exceeds the current thermal budget, after adjusting the luminance levels of the first zones; and

when it is determined that the subsequent thermal load still exceeds the current thermal budget,

identify second zones that correspond to at least one graphical element whose importance level is higher than the lowest importance level and lower than a highest importance level; and

adjust luminance levels of the second zones to lie between respective minimum luminance levels and respective maximum luminance levels, until a further subsequent thermal load of the display does not exceed the current thermal budget.

17 . The system of claim 16 , wherein a size of a single zone is larger than a size of a single pixel of the display, wherein the at least one processor is configured to:

when it is determined that the thermal load exceeds the current thermal budget,

identify at least one graphical element from among the plurality of graphical elements that is not constrained to a fixed position in the sequence of images; and

relocate the at least one graphical element to a position proximate to at least one other graphical element from among the plurality of graphical elements, wherein at least one other zone is used to present both the at least one graphical element and the at least one other graphical element.