IP Library Granted Patent US 11,410,269
Granted Patent B2
US 11,410,269 · App. 17/091,278 · Granted Aug 9, 2022

Mixed reality system with virtual content warping and method of generating virtual content using same

Inventors: Reza Nourai (Danville, CA); Robert Blake Taylor (Porter Ranch, CA)
G06T3/0093G06T15/205G06T15/40G06T17/10G06T19/006G06T2210/44
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,410,269
App. No.
17/091,278
Granted
Aug 9, 2022
Kind
B2
Abstract

A computer implemented method for warping virtual content includes generating warped virtual content by transforming source virtual content. The method also includes determining whether a memory location corresponding to an X, Y location of the warped virtual content in an output frame of reference is occupied by pre-existing virtual content. The method further includes storing the warped virtual content in the memory location if the memory location is not occupied. Moreover, the method includes comparing respective Z locations of the warped virtual content and the pre-existing virtual content to identify virtual content with a Z location closer to a viewing location if the memory location is occupied. The method also includes storing the warped virtual content in the memory location corresponding to the X, Y location if a Z location of warped virtual content is closer to the viewing location than a pre-existing Z location of pre-existing virtual content.

Claims (37)

1. A computer implemented method for warping virtual content:

generating warped virtual content;

comparing respective Z locations of the warped virtual content and pre-existing virtual content having a same X, Y location to identify virtual content with a Z location closer to a viewing location in an output frame of reference; and

storing the warped virtual content in a memory location corresponding to the same X, Y location if a Z location of the warped virtual content is closer to the viewing location than a pre-existing Z location of the pre-existing virtual content in the output frame of reference,

wherein generating the warped virtual content, comparing the respective Z locations of the warped virtual content and the pre-existing virtual content, and storing the warped virtual content in the memory location all occur in a single pass.

2. The method of claim 1 , further comprising discarding the warped virtual content if the pre-existing Z location of the pre-existing virtual content is closer to the viewing location than the Z location of the warped virtual content in the output frame of reference.

3. The method of claim 1 , wherein the warped virtual content is a quadrilateral.

4. The method of claim 1 , wherein the warped virtual content is a triangle.

5. The method of claim 1 , wherein the warped virtual content is a pixel.

6. The method of claim 1 , wherein the warped virtual content is generated from source virtual content, the method further comprising generating the source virtual content using a source frame of reference different from the output frame of reference.

7. The method of claim 6 , wherein the source virtual content is transformed from the source frame of reference to the output frame of reference.

8. The method of claim 7 , wherein the respective Z locations of the warped virtual content and the pre-existing virtual content in the output frame of reference are different from corresponding Z locations of the warped virtual content and the pre-existing virtual content in the source frame of reference.

9. The method of claim 6 , wherein the source virtual content comprises image information and a source X, Y location in the source frame of reference.

10. The method of claim 9 , wherein the image information comprises a brightness.

11. The method of claim 9 , wherein the image information comprises a color.

12. The method of claim 9 , wherein the image information comprises a Z location in the source frame of reference.

13. The method of claim 9 , wherein the warped virtual content comprises an output X, Y location in the output frame of reference.

14. A system for warping virtual content, comprising:

a graphics processing unit (GPU) core to generate warped virtual content;

a raster operations processor (ROP) to rasterize the warped virtual content for display; and

a buffer memory to temporarily store the warped virtual content,

wherein the ROP compares respective Z locations of pre-existing virtual content and the warped virtual content and writes the warped virtual content in the buffer memory when a Z location of the warped virtual content is closer to a viewing location than a pre-existing Z location of the pre-existing virtual content, and

wherein the following occur in a single pass:

the GPU generates the warped virtual content,

the ROP compares the respective Z locations of the pre-existing virtual content and the warped virtual content, and

the ROP writes the warped virtual content in the buffer memory when the Z location of the warped virtual content is closer to the viewing location than the pre-existing Z location of the pre-existing virtual content.

15. The system of claim 14 , wherein the GPU generates the warped virtual content from source virtual content, and

wherein the source virtual content comprises a source X, Y value in a source frame of reference.

16. The system of claim 15 , wherein the warped virtual content comprises an output X, Y value in an output frame of reference.

17. The system of claim 16 , wherein the respective Z locations of the warped virtual content and the pre-existing virtual content in the output frame of reference are different from corresponding Z locations of the warped virtual content and the pre-existing virtual content in the source frame of reference.

18. The system of claim 15 , wherein the source virtual content comprises image information and a source X, Y location in the source frame of reference.

19. The system of claim 18 , wherein the image information comprises, a brightness, a color, or a Z location in the source frame of reference.

20. A computer program product embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for warping virtual content, the method comprising:

generating warped virtual content;

comparing respective Z locations of the warped virtual content and pre-existing virtual content to identify virtual content with a Z location closer to a viewing location in an output frame of reference; and

storing the warped virtual content in a memory location corresponding to an X, Y location if a Z location of the warped virtual content is closer to the viewing location than a pre-existing Z location of the pre-existing virtual content in the output frame of reference,

wherein generating the warped virtual content, comparing the respective Z locations of the warped virtual content and the pre-existing virtual content, and storing the warped virtual content in the memory location all occur in a single pass.

Assignments (2)
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 7, 2020
From: NOURAI, REZA; TAYLOR, ROBERT BLAKE
To: MAGIC LEAP, INC.
Reel/Frame 054305/0981 →