IP Library Granted Patent US 11,818,325
Granted Patent B2
US 11,818,325 · App. 17/368,096 · Granted Nov 14, 2023

Blended mode three dimensional display systems and methods

Inventor: Robert Blake Taylor (Porter Ranch, CA)
Assignee: Magic Leap, Inc.
H04N13/156H04N13/128H04N13/398H04N2213/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,818,325
App. No.
17/368,096
Granted
Nov 14, 2023
Kind
B2
Abstract

A method for displaying a three dimensional (“3D”) image includes rendering a frame of 3D image data. The method also includes analyzing the frame of 3D image data to generate depth data. The method further includes using the depth data to segment the 3D image data into i) at least one near frame of two dimensional (“2D”) image data corresponding to a near depth, and ii) at least one far frame of 2D image data corresponding to a far depth that is farther than the near depth from a point of view. Moreover, the method includes displaying the near and far frames at the near and far depths respectively. The near and far frames are displayed simultaneously.

Claims (62)

1. A method for displaying a three dimensional (“3D”) image in a blended mode, comprising:

rendering a frame of 3D image data;

analyzing the frame of 3D image data to generate depth data;

using the depth data to segment the 3D image data into: i) at least one near frame of two dimensional (“2D”) image data corresponding to a near depth, and ii) at least one far frame of 2D image data corresponding to a far depth that is farther than the near depth from a point of view;

using the depth data to determine scalar values for pixels of a sub-image in the frame of 3D image data; and

displaying the near and far frames at the near and far depths respectively, wherein the sub-image is displayed in at least one of the near frame and far frame at a level of illumination based at least in part on the scalar values of the pixels of the sub-image, and wherein the near and far frames are displayed simultaneously.

2. The method of claim 1 , wherein the frame of 3D image data comprises:

depth segmentation data;

stereo color pair data; and

real world mesh data.

3. The method of claim 1 , wherein analyzing the frame of 3D image data to generate the depth data comprises:

generating a disparity map from the frame of 3D image data; and

re-projecting the frame of 3D image data.

4. The method of claim 1 , wherein using the depth data to segment the 3D image data into the near and far of frames of 2D image data comprises:

identifying a near only set of virtual objects/pixels having respective depths in a near depth range;

identifying a far only set of virtual objects/pixels having respective depths in a far depth range; and

identifying a near and far overlap set of virtual objects/pixels having respective depths in a middle depth range.

5. The method of claim 4 , wherein using the depth data to segment the 3D image data into the near and far of frames of 2D image data further comprises:

adding the near only set of virtual objects/pixels to the near frame of 2D image data;

adding the far only set of virtual objects/pixels to the far frame of 2D image data;

performing a blending analysis on the near and far overlap set of virtual objects/pixels to identify a near overlap set of virtual objects/pixels and a far overlap set of virtual objects/pixels;

adding the near overlap set of virtual objects/pixels to the near frame of 2D image data; and

adding the far overlap set of virtual objects/pixels to the far frame of 2D image data.

6. The method of claim 5 , wherein the blending analysis involves a linear interpolation, a non-linear interpolation, and/or a plurality of linear interpolations.

7. A method for displaying a three dimensional (“3D”) image in a blended mode, comprising:

rendering a frame of 3D image data;

analyzing the frame of 3D image data to generate depth data;

using the depth data to segment the 3D image data into a plurality of frames of two dimensional (“2D”) image data;

using the depth data to determine scalar values for pixels of a sub-image in the frame of 3D image data;

displaying the plurality of frames, wherein the plurality of frames comprises:

a left near frame of 2D image data corresponding to a near depth;

a left far frame of 2D image data corresponding to a far depth that is farther than the near depth from a point of view;

a right near frame of 2D image data corresponding to the near depth; and

a right far frame of 2D image data corresponding to the far depth,

wherein the left near frame and the left far frame are displayed simultaneously, and

wherein the right near frame and the right far frame are displayed simultaneously; and

wherein the sub-image is displayed in at least one of the frames at a level of illumination based at least in part on the scalar values of the pixels of the sub-image.

8. The method of claim 7 , wherein the left near frame and the left far frame are displayed to a left eye of a user.

9. The method of claim 7 , wherein the right near frame and the right far frame are displayed to a right eye of a user.

10. The method of claim 7 , wherein the left near frame and the right near frame are displayed to a user at a first depth from the user.

11. The method of claim 10 , wherein the left far frame and the right far frame are displayed to the user at a second depth from the user, the second depth being greater than the first depth.

12. The method of claim 11 , wherein the first and second depths correspond to about 1.96 and about 0.67 respectively.

13. The method of claim 7 , wherein when the left near frame, the left far frame, the right near frame, and the right far frame are displayed to a user, the user perceives a 3D image.

14. The method of claim 13 , wherein the 3D image corresponds to the frame of 3D image data.

15. The method of claim 1 , wherein the near depth corresponds to about 1.96 diopters, and wherein the far depth corresponds to about 0.67 diopters.

16. The method of claim 7 , wherein the near depth corresponds to about 1.96 diopters, and wherein the far depth corresponds to about 0.67 diopters.

17. A method for displaying a three dimensional (“3D”) image in a blended mode, comprising:

rendering a frame of 3D image data;

analyzing the frame of 3D image data to generate depth data;

using the depth data to segment the 3D image data into: i) at least one near frame of two dimensional (“2D”) image data corresponding to a near depth, and ii) at least one far frame of 2D image data corresponding to a far depth that is farther than the near depth from a point of view; and

displaying the near and far frames at the near and far depths respectively, wherein the near depth corresponds to about 1.96 diopters and the far depth corresponds to about 0.67 diopters, and wherein the near and far frames are displayed simultaneously,

wherein using the depth data to segment the 3D image data into the near and far of frames of 2D image data comprises:

identifying a near only set of virtual objects/pixels having respective depths in a near depth range;

identifying a far only set of virtual objects/pixels having respective depths in a far depth range; and

identifying a near and far overlap set of virtual objects/pixels having respective depths in a middle depth range.

18. The method of claim 17 , wherein using the depth data to segment the 3D image data into the near and far of frames of 2D image data further comprises:

adding the near only set of virtual objects/pixels to the near frame of 2D image data;

adding the far only set of virtual objects/pixels to the far frame of 2D image data;

performing a blending analysis on the near and far overlap set of virtual objects/pixels to identify a near overlap set of virtual objects/pixels and a far overlap set of virtual objects/pixels;

adding the near overlap set of virtual objects/pixels to the near frame of 2D image data; and

adding the far overlap set of virtual objects/pixels to the far frame of 2D image data.

19. The method of claim 18 , wherein the blending analysis involves a linear interpolation, a non-linear interpolation, and/or a plurality of linear interpolations.

Assignments (4)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0238 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 7, 2021
From: TAYLOR, ROBERT BLAKE
To: MAGIC LEAP, INC.
Reel/Frame 056781/0861 →