IP Library Granted Patent US 11,341,936
Granted Patent B2
US 11,341,936 · App. 17/313,295 · Granted May 24, 2022

Piecewise progressive and continuous calibration with coherent context

Inventor: Aydin Arpa (Plantation, FL)
Assignee: Magic Leap, Inc.
G09G5/37G02B27/0172G06V10/44G06V10/56G02B2027/014G09G2320/0693
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,341,936
App. No.
17/313,295
Granted
May 24, 2022
Kind
B2
Abstract

A piecewise progressive continuous calibration method with context coherence is utilized to improve display of virtual content. When a set of frames are rendered to depict a virtual image, the VAR system may identify a location of the virtual content in the frames. The system may convolve a test pattern at the location of the virtual content to generate a calibration frame. The calibration frame is inserted within the set of frames in a manner that is imperceptible to the user.

Claims (73)

1. A computer-implemented method comprising:

generating frames that include virtual content;

determining a location of the virtual content in the frames;

convolving, by a processor, a test pattern at the location of the virtual content;

generating a calibration frame based on the convolving; and

applying the calibration frame to the frames that include the virtual content.

2. The computer-implemented method of claim 1 , wherein determining the location of the virtual content in the frames comprises:

applying one or more of edge detection or chromatic pattern detection to identify a region in the frames that include the virtual content;

identifying a location of pixels and pixel values in the region; and

storing data indicative of the location of the pixels and pixel values in a memory.

3. The computer-implemented method of claim 1 , wherein convolving the test pattern at the location of the virtual content comprises:

selecting a representative virtual content frame; and

convolving the test pattern at the location of the virtual content in the representative virtual content frame.

4. The computer-implemented method of claim 1 , further comprising:

determining a refresh rate of a display of a virtual or augmented reality system,

wherein a number of the frames that include the virtual content is based on the refresh rate.

5. The computer-implemented method of claim 1 , wherein applying the calibration frame to the frames that include the virtual content comprises:

inserting the calibration frame between two of the frames that include the virtual content.

6. The computer-implemented method of claim 1 , wherein generating the calibration frame comprises:

determining an intensity level of the virtual content in the frames;

determining that the intensity level of the virtual content in the frames is less than a minimum threshold intensity level; and

generating the calibration frame having an intensity level matching the minimum threshold intensity level.

7. The computer-implemented method of claim 1 , comprising:

in response to applying the calibration frame to the frames, determining one or more calibration parameters; and

displaying the virtual content at a display using the calibration parameters.

8. The computer-implemented method of claim 1 , wherein generating the calibration frame comprises:

hiding a test image behind edges of the virtual content in the calibration frame; and

generating the calibration frame using a test pattern that has the same color as a color of the virtual content.

9. A virtual or augmented reality system, comprising:

a frame structure comprising a display device configured to display data; and

a processor coupled to the frame structure, the processor configured to:

generate frames that include virtual content;

determine a location of the virtual content in the frames;

convolve a test pattern at the location of the virtual content;

generate a calibration frame based on the convolving; and

transmit the calibration frame and the frames that include the virtual content to the display device,

wherein, in response to receiving the calibration frame and the frames that include the virtual content from the processor, the display device is configured to display the calibration frame and the frames that include the virtual content.

10. The virtual or augmented reality system of claim 9 , wherein the processor being configured to determine the location of the virtual content in the frames comprises the processor being configured to:

apply one or more of edge detection or chromatic pattern detection to identify a region in the frames that include the virtual content;

identify a location of pixels and pixel values in the region; and

store data indicative of the location of the pixels and pixel values in a memory.

11. The virtual or augmented reality system of claim 9 , wherein the processor being configured to convolve the test pattern at the location of the virtual content comprises the processor being configured to:

select a representative virtual content frame; and

convolve the test pattern at the location of the virtual content in the representative virtual content frame.

12. The virtual or augmented reality system of claim 9 , wherein:

the processor is configured to determine a refresh rate of a display of a virtual or augmented reality system; and

a number of the frames that include the virtual content is based on the refresh rate.

13. The virtual or augmented reality system of claim 9 , wherein the processor being configured to apply the calibration frame to the frames that include the virtual content comprises the processor being configured to:

insert the calibration frame between two of the frames that include the virtual content.

14. The virtual or augmented reality system of claim 9 , wherein:

the processor is configured to determine one or more calibration parameters in response to applying the calibration frame to the frames; and

the display device is configured to display the virtual content using the calibration parameters.

15. A non-transitory computer-readable storage medium comprising instructions, which, when executed by one or more computer processors, cause the one or more computer processors to perform operations comprising:

generating frames that include virtual content;

determining a location of the virtual content in the frames;

convolving, by a processor, a test pattern at the location of the virtual content;

generating a calibration frame based on the convolving; and

applying the calibration frame to the frames that include the virtual content.

16. The non-transitory computer-readable storage medium of claim 15 , wherein determining the location of the virtual content in the frames comprises:

applying one or more of edge detection or chromatic pattern detection to identify a region in the frames that include the virtual content;

identifying a location of pixels and pixel values in the region; and

storing data indicative of the location of the pixels and pixel values in a memory.

17. The non-transitory computer-readable storage medium of claim 15 , wherein convolving the test pattern at the location of the virtual content comprises:

selecting a representative virtual content frame; and

convolving the test pattern at the location of the virtual content in the representative virtual content frame.

18. The non-transitory computer-readable storage medium of claim 15 , further comprising:

determining a refresh rate of a display of a virtual or augmented reality system,

wherein a number of the frames that include the virtual content is based on the refresh rate.

19. The non-transitory computer-readable storage medium of claim 15 , wherein applying the calibration frame to the frames that include the virtual content comprises:

inserting the calibration frame between two of the frames that include the virtual content.

20. The non-transitory computer-readable storage medium of claim 15 , wherein the operations further comprise:

in response to applying the calibration frame to the frames, determining one or more calibration parameters; and

displaying the virtual content at a display using the calibration parameters.

Assignments (2)
SECURITY INTEREST Recorded Jul 1, 2022
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060560/0438 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2022
From: ARPA, AYDIN
To: MAGIC LEAP, INC.
Reel/Frame 059253/0934 →
Continuity (2)
Provisional Application 63027677 · May 20, 2020
Related Publication 20210366439A1 · Nov 25, 2021