IP Library Granted Patent US 11,189,252
Granted Patent B2
US 11,189,252 · App. 16/928,313 · Granted Nov 30, 2021

Image correction due to deformation of components of a viewing device

Inventor: Adam Neustein (San Francisco, CA)
Assignee: Magic Leap, Inc.
G09G5/38G02B27/0179G06T7/73G09G5/02G09G5/373G02B2027/0187G06T2207/30201G09G2340/045G09G2340/0464G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,189,252
App. No.
16/928,313
Granted
Nov 30, 2021
Kind
B2
Abstract

A display assembly displays a virtual object in a select location wherein an eye viewing the virtual object has an expected gaze direction. Deformation of the display assembly is detected. The deformation causes the virtual object to be viewable in an altered location wherein the eye has altered gaze direction. The virtual object may be displayed in a corrected location wherein the eye viewing the virtual object in the correct location has a corrected gaze direction that is moved closer to the expected gaze direction than the altered gaze direction.

Claims (28)

1. A viewing device for displaying rendered content comprising:

a display assembly that includes an optical display configured to display a virtual object in a select location on the display assembly, wherein an eye viewing the virtual object has an expected gaze direction;

a deformation detection system connected to the display assembly and including an eye tracking camera configured to capture light reflected from the eye; and

a gaze angle calculation module, being a software module, connected to the eye tracking camera, configured to calculate a measured gaze direction of the eye viewing the virtual object on the display assembly based on the light captured by the eye tracking camera, the measured gaze direction being an altered gaze direction due to deformation of the eye tracking camera and due to deformation of the optical display, and calculate a deformation of the eye tracking camera of the display assembly based on the measured gaze direction, wherein the difference between the expected gaze direction and the altered gaze direction is in a first direction and the change in the measured gaze direction due to the deformation of the eye tracking camera is in a second direction with an angle relative to one another; and

a correction system, being a software module, connected to the deformation detection system to display, with the display assembly, the virtual object in a corrected location wherein the eye viewing the virtual object in the corrected location has a corrected gaze direction that is moved toward the expected gaze direction from the altered gaze direction.

2. The system of claim 1 , wherein the deformation includes a deformation of the optical display.

3. The system of claim 2 , wherein the optical display is deformed in a display deformation direction and the altered gaze direction is moved relative to the expected gaze direction in the display deformation direction.

4. The system of claim 3 , wherein the optical display is transparent and the virtual object is viewable by the eye at a location on a side of the optical display opposing the eye.

5. The system of claim 1 , wherein:

the difference between the expected gaze direction and the altered gaze direction is in a first direction and the change in the measured gaze direction due to the deformation of the eye tracking camera is in a second direction and the first and second directions are less than 90 degrees relative to one another.

6. The system of claim 1 , wherein:

the change from the expected gaze direction to the altered gaze direction is in a first direction and the change in the measured gaze direction due to the deformation of the eye tracking camera is in a second direction and the first and second directions are separated by more than 90 degrees relative to one another.

7. The system of claim 1 , further comprising:

a reference system, being a software module, connected to the eye tracking camera, to detect a change in the measured gaze direction due to the deformation of the eye tracking camera; and

a correction calculation unit, being a software module, connected to the eye tracking camera, to determine the change in the measured gaze direction due to the deformation of the display assembly by deducting the change in the measured gaze direction due to the deformation of the eye tracking camera from the change in the gaze angle that is detected by the deformation detection system.

8. The system of claim 1 , wherein the deformation detection system includes:

a statistical system, being a software module, configured to receive and analyze multiple measured gaze direction measurements;

a gaze angle calculation unit, being a software module, that calculates multiple gaze directions, each gaze direction for a respective measurement; and

a correction calculation unit, being a software module, that determines the altered gaze direction based on the multiple actual gaze direction measurements.

9. The system of claim 1 , wherein the deformation detection system includes an attention generation unit, being a software module, configured to alter the display of the virtual object and attract a user's attention to the virtual object before detecting the deformation.

10. The system of claim 9 , wherein the attention generation unit is configured to attract the user's attention by changing a color of the virtual object.

11. The system of claim 9 , wherein the attention generation unit is configured to attract the user's attention to the virtual object by reducing a size of the virtual object.

12. A method of displaying rendered content comprising:

displaying, with an optical display of a display assembly, a virtual object in a select location on the display assembly wherein an eye viewing the virtual object has an expected gaze direction;

detecting, with an eye tracking camera configured to capture light reflected from the eye, a deformation of the display assembly, wherein the deformation causes the virtual object to be viewable in an altered location wherein the eye has an altered gaze direction;

calculating, with a gaze angle calculation module, a measured gaze direction of the eye viewing the virtual object on the display assembly based on the light captured by the eye tracking camera, the measured gaze direction being the altered gaze direction due to deformation of at least the eye tracking camera;

calculating a deformation of the eye tracking camera of display assembly based on the measured gaze direction being the altered gaze direction that is different from the expected gaze direction, wherein the difference between the expected gaze direction and the altered gaze direction is in a first direction and the change in the measured gaze direction due to the deformation of the eye tracking camera is in a second direction with an angle relative to one another; and

displaying, with a display assembly, the virtual object in a corrected location wherein the eye viewing the virtual object in the corrected location has a corrected gaze direction that is moved toward the expected gaze direction from the altered gaze direction.

Assignments (3)
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2021
From: NEUSTEIN, ADAM
To: MAGIC LEAP, INC.
Reel/Frame 058353/0337 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 27, 2020
From: NEUSTEIN, ADAM
To: MAGIC LEAP, INC.
Reel/Frame 053316/0130 →