IP Library Granted Patent US 11,494,960
Granted Patent B2
US 11,494,960 · App. 17/348,458 · Granted Nov 8, 2022

Display that uses a light sensor to generate environmentally matched artificial reality content

Inventors: Jiangtao Kuang (San Jose, CA); Edward Buckley (Redmond, WA); Honghong Peng (Mountain View, CA); Sapna Shroff (Burlingame, CA); Romain Bachy (Seattle, WA)
Assignee: Meta Platforms Technologies, LLC
G06T11/60G06T7/90G06T11/001G06T2207/10024G06T2207/10048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,494,960
App. No.
17/348,458
Granted
Nov 8, 2022
Kind
B2
Abstract

A display assembly generates environmentally matched virtual content for an electronic display. The display assembly includes a display controller and a display. The display controller is configured to estimate environmental matching information for a target area within a local area based in part on light information received from a light sensor. The target area is a region for placement of a virtual object. The light information describes light values. The display controller generates display instructions for the target area based in part on a human vision model, the estimated environmental matching information, and rendering information associated with the virtual object. The display is configured to present the virtual object as part of artificial reality content in accordance with the display instructions. The color and brightness of the virtual object is environmentally matched to the portion of the local area surrounding the target area.

Claims (44)

1. A headset comprising:

a light sensor configured to capture light information of a local area, the light information describing light values for a plurality of different color channels;

a controller configured to:

access an environmental matching model that includes environmental matching information estimated for a plurality of locations within the local area as a function of a location within the local area and based in part on the light information received from the light sensor, and

generate display instructions for presentation of artificial reality content based in part on the environmental matching model; and

a display configured to present the artificial reality content in accordance with the display instructions, the artificial reality content including a virtual object rendered in a target area within the local area and in a portion of the local area surrounding the target area, wherein color and brightness of the rendered virtual object is adjusted for each location within the target area and each location within the portion of the local area surrounding the target area based on the environmental matching model including the environmental matching information estimated as the function of the location within the local area.

2. The headset of claim 1 , wherein the display instructions include one or more pixel value adjustments for the target area, the pixel value adjustments controlling the color and the brightness of the virtual object.

3. The headset of claim 1 , wherein the light information includes a hue value and a saturation value for the target area, and the controller is further configured to estimate the environmental matching information for each location within the target area based in part on the hue value and the saturation value.

4. The headset of claim 1 , wherein the light information includes an infrared measurement value for the target area, and the controller is further configured to estimate the environmental matching information for each location within the target area based in part on the infrared measurement value.

5. The headset of claim 1 , wherein the controller is further configured to:

update the display instructions dynamically based in part on the light information received from the light sensor changing by a predefined threshold value.

6. The headset of claim 1 , wherein the display instructions are further based on a human vision model including standardized measurements of color that describe how to present a measured color to a user.

7. The headset of claim 1 , wherein the display instructions are further based on rendering information including a predetermined color and a predetermined brightness of the virtual object.

8. The headset of claim 1 , further comprising:

a camera configured to capture additional light information of the local area; and

wherein the environmental matching model is further based on the additional light information.

9. A method comprising:

receiving, from a light sensor, light information of a local area, the light information describing light values for a plurality of different color channels;

accessing an environmental matching model that includes environmental matching information estimated for a plurality of locations within the local area as a function of a location within the local area and based in part on the received light information;

generating display instructions for presentation of artificial reality content based in part on the environmental matching model; and

presenting, by a display, the artificial reality content in accordance with the display instructions, the artificial reality content including a virtual object rendered in a target area within the local area and in a portion of the local area surrounding the target area, wherein color and brightness of the rendered virtual object is adjusted for each location within the target area and each location within the portion of the local area surrounding the target area based on the environmental matching model including the environmental matching information estimated as the function of the location within the local area.

10. The method of claim 9 , wherein the display instructions include one or more pixel value adjustments for the target area, the pixel value adjustments controlling the color and the brightness of the virtual object.

11. The method of claim 9 , wherein the light information includes a hue value and a saturation value for the target area, the method further comprising:

estimating the environmental matching information for each location within the target area based in part on the hue value and the saturation value.

12. The method of claim 9 , wherein the light information includes an infrared measurement value for the target area, the method further comprising:

estimating the environmental matching information for each location within the target area based in part on the infrared measurement value.

13. The method of claim 9 , further comprising:

updating the display instructions dynamically based in part on the light information received from the light sensor changing by a predefined threshold value.

14. The method of claim 9 , wherein the display instructions are further based on a human vision model including standardized measurements of color that describe how to present a measured color to a user.

15. The method of claim 9 , wherein the display instructions are further based on rendering information including a predetermined color and a predetermined brightness of the virtual object.

16. The method of claim 9 , further comprising:

receiving, from a camera, additional light information of the local area; and

wherein the environmental matching model is further based on the received additional light information.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, from a light sensor, light information of a local area, the light information at least describes light values for a plurality of different color channels;

accessing an environmental matching model that includes environmental matching information estimated for a plurality of locations within the local area as a function of a location within the local area and based in part on the received light information;

generating display instructions for presentation of artificial reality content based in part on the environmental matching model; and

presenting, via a display, the artificial reality content in accordance with the display instructions, the artificial reality content including a virtual object rendered in a target area within the local area and in a portion of the local area surrounding the target area, wherein color and brightness of the rendered virtual object is adjusted for each location within the target area and each location within the portion of the local area surrounding the target area based on the environmental matching model including the environmental matching information estimated as the function of the location within the local area.

18. The computer-readable medium of claim 17 , wherein the display instructions include one or more pixel value adjustments for the target area, the pixel value adjustments controlling the color and the brightness of the virtual object.

19. The computer-readable medium of claim 17 , wherein the instructions further cause the one or more processors to perform operations comprising:

receiving, from the light sensor, light information including an infrared measurement value; and

estimating the environmental matching information for each location within the target area based in part on the infrared measurement value.

20. The computer-readable medium of claim 17 , wherein the instructions further cause the one or more processors to perform operations comprising:

updating the display instructions dynamically based in part on the light information received from the light sensor changing by a predefined threshold value.

Assignments (1)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060314/0965 →
Continuity (2)
Continuation 15931425 · May 13, 2020
Related Publication 20210358187A1 · Nov 18, 2021