IP Library Granted Patent US 11,361,474
Granted Patent B2
US 11,361,474 · App. 16/530,877 · Granted Jun 14, 2022

Method and system for subgrid calibration of a display device

Inventors: Kazunori Tanaka (Irvine, CA); Lei Zhang (Sunnyvale, CA); Etienne Gregoire Grossmann (Menlo Park, CA); Marshall Charles Capps (Fort Lauderdale, FL); Robert Blake Taylor (Porter Ranch, CA)
Assignee: Magic Leap, Inc.
G06T11/001G06F3/011G06T19/006G09G3/2003G09G5/026H04N9/69H04N17/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,474
App. No.
16/530,877
Granted
Jun 14, 2022
Kind
B2
Abstract

A method for calibrating a wearable device includes displaying an image with a plurality of pixels for each of three primary colors using the wearable device, and determining RGB and XYZ values for each of the plurality of pixels. The method includes selecting a subset of the plurality of pixels to form a group of grid points, and dividing the image into a group of tile regions, with each tile region including a grid point. Grid XYZ values are determined for each grid point, based on averaging XYZ values of all pixels in a corresponding tile region, and a grid RGB-to-XYZ conversion matrix is determined for each grid point. The method also includes determining a correction matrix for each grid point by multiplying an inverse of the grid RGB-to-XYZ conversion matrix for the grid with an sRGB-to-XYZ conversion matrix.

Claims (74)

1. A method for calibrating a wearable device, the method comprising:

displaying an image with a plurality of pixels for each of three primary colors using the wearable device;

determining RGB and XYZ values for each of the plurality of pixels;

selecting a subset of the plurality of pixels to form a group of grid points;

dividing the image into a group of tile regions, each tile region including a grid point;

determining grid XYZ values for each grid point, based on averaging XYZ values of all pixels in a corresponding tile region;

determining a grid RGB-to-XYZ conversion matrix for each grid point; and

determining a correction matrix for the plurality of pixels by:

for each pixel that is a grid point, multiplying an inverse of the grid RGB-to-XYZ conversion matrix for the grid point with an sRGB-to-XYZ conversion matrix to form grid point correction matrix elements; and

for each pixel that is not a grid point, forming pixel correction matrix elements by interpolating using the grid point correction matrix elements from adjacent grid points.

2. The method of claim 1

wherein, for each tile region, a corresponding grid point is positioned in a corner of each tile region.

3. The method of claim 1 , wherein the interpolation is performed using bilinear interpolation or barycentric interpolation.

4. The method of claim 1 , further comprising:

receiving color values for each pixel for an intended content as defined in a target color space;

applying the correction matrix for each pixel to received color values to generate corrected color values; and

sending the corrected color values to the wearable device for displaying the intended content.

5. The method of claim 4 , further comprising linearizing received color values.

6. The method of claim 4 , further comprising applying a gamma to corrected color values.

7. The method of claim 4 , further comprising scaling down corrected color values.

8. The method of claim 1 further comprising:

receiving color values for each pixel for an intended content as defined in a target color space;

applying the correction matrix for each grid point to received color values to generate new color values for the grid point;

generating new color values for each pixel that is not a grid point by interpolating from the new color values for adjacent grid points; and

sending the new color values to the wearable device for displaying the intended content.

9. The method of claim 1 , wherein determining RGB and XYZ values for each of the plurality of pixels comprises capturing RGB values using a digital camera and converting the RGB values to XYZ values.

10. The method of claim 1 , wherein the correction matrix is an sRGB-to-display-RGB correction matrix that is configured to provide corrected display-RGB color values to the wearable device.

11. A method for calibrating a wearable device, comprising:

displaying an image for each of three primary colors by a wearable display device;

capturing the image in M by N pixels using a digital color camera, where M and N are integers, the digital color camera providing RGB values for each pixel;

converting the RGB values for each of the pixels of the image to XYZ values using a camera conversion matrix that converts RGB values to corresponding XYZ values;

selecting K by L grid points from the M by N pixels, where K and L are integers smaller than M and N;

dividing the image into K by L tile regions, each tile region including a grid point;

determining grid XYZ values for each grid point, based on averaging XYZ values of all pixels in a corresponding tile region;

determining a grid RGB-to-XYZ conversion matrix for each grid point that converts the RGB values at the grid point to the XYZ values of the grid point;

determining a correction matrix for each pixel by:

for each pixel that is a grid point, multiplying an inverse of the grid RGB-to-XYZ conversion matrix for the grid point with an sRGB-to-XYZ conversion matrix to form grid point correction matrix elements; and

for each pixel that is not a grid point, forming pixel correction matrix elements by interpolating from grid point correction matrix elements from adjacent grid points;

wherein the correction matrix is an sRGB-to-display-RGB correction matrix that is configured to provide corrected display-RGB color values to the wearable device,

receiving color values for a pixel for an intended content as defined in a target color space;

applying the correction matrix for each pixel to received color values to generate new color values; and

sending the new color values to in the wearable device for displaying the intended content.

12. The method of claim 11 ,

wherein, for each tile region, a corresponding grid point is positioned in a corner of each tile region.

13. The method of claim 12 , wherein the interpolation is performed using bilinear interpolation or barycentric interpolation.

14. The method of claim 12 , further comprising linearizing received color values.

15. The method of claim 12 , further comprising applying a gamma to the new color values.

16. The method of claim 12 , further comprising scaling down the new color values.

17. A system for calibrating a wearable display device, the system comprising:

a digital color camera disposed to capture an image displayed by the wearable display device, the digital color camera configured to provide RGB values for each of a plurality of pixels of the image;

a first processor, configured for:

converting the RGB values provided by the digital color camera to XYZ values;

selecting a subset of the plurality of pixels to form a group of grid points;

dividing the image into a group of tile regions, each tile region including a grid point;

determining grid XYZ values for each grid point, based on averaging XYZ values of all pixels in a corresponding tile region;

determining a grid RGB-to-XYZ conversion matrix for each grid point; and

determining a correction matrix for the plurality of pixels by:

for each pixel that is a grid point multiplying an inverse of the grid RGB-to-XYZ conversion matrix for the pixel with an sRGB-to-XYZ conversion matrix to form grid point correction matrix elements; and

for each pixel that is not a grid point, forming pixel correction matrix elements by interpolating using the grid point correction matrix elements from adjacent grid points.

18. The system of claim 17 , wherein the correction matrix is an sRGB-to-display-RGB correction matrix that is configured to provide corrected display-RGB color values to the wearable display device.

19. The system of claim 17 , wherein the system is further configured for:

receiving color values for a pixel for an intended content as defined in a target color space;

applying the correction matrix to received color values to generate new color values; and

sending the new color values to the wearable display device for displaying the intended content.

20. A method for calibrating a wearable device, the method comprising:

displaying an image with a plurality of pixels for each of three primary colors using the wearable device;

determining RGB and XYZ values for each of the plurality of pixels;

determining a pixel RGB-to-XYZ conversion matrix for each pixel;

determining a correction matrix for each pixel by:

for each pixel that is a grid point, multiplying an inverse of the pixel RGB-to-XYZ conversion matrix for the grid point with an sRGB-to-XYZ conversion matrix to form grid point correction matrix elements; and

for each pixel that is not a grid point, forming pixel correction matrix elements by interpolating from grid point correction matrix elements from adjacent grid points;

receiving color values for each pixel for an intended content as defined in a target color space;

applying the correction matrix for each pixel to received color values to generate corrected color values; and

sending the corrected color values to the wearable device for displaying the intended content.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2022
From: TANAKA, KAZUNORI; ZHANG, LEI; GROSSMANN, ETIENNE GREGOIRE; CAPPS, MARSHALL CHARLES; TAYLOR, ROBERT BLAKE
To: MAGIC LEAP, INC.
Reel/Frame 060103/0061 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →