IP Library Granted Patent US 10,813,619
Granted Patent B2
US 10,813,619 · App. 15/590,922 · Granted Oct 27, 2020

Augmented reality systems and methods for user health analysis

Inventors: Nicole Elizabeth Samec (Fort Lauderdale, FL); Mark Baerenrodt (Millbrae, CA); Nastasja U. Robaina (Coconut Grove, FL); Charlotte Dorothea Wilhelmina Vinkers (Fort Lauderdale, FL); Christopher M. Harrises (Nashua, NH); Nicholas Atkinson Kramer (Fort Lauderdale, FL)
Assignee: Magic Leap, Inc.
A61B7/04A61B3/113A61B5/0077A61B5/01A61B5/021A61B5/024A61B5/14532A61B5/6898G02B27/017G02B27/0172G06F3/013A61B5/0066A61B5/0095A61B5/0402A61B5/0476A61B5/0488A61B5/082A61B5/0816A61B5/1116A61B5/1123A61B5/1128A61B5/441A61B5/744A61B2562/0219G02B6/009G02B6/0076G02B2027/014G02B2027/0112G02B2027/0125G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,813,619
App. No.
15/590,922
Granted
Oct 27, 2020
Kind
B2
Abstract

Augmented reality systems and methods for user health analysis. Methods for user health analysis may include collecting data for an initial prediction model and continuing to collect additional data based on one or more data criteria. The methods may further include updating the initial prediction model based on the additional data to produce a revised prediction model or causing an intervention to occur based on the additional data. The data may be collected by a display system including one or more sensors configured to collect user-specific data and a display device configured to present virtual content to a user. The display device may be configured to output light with variable wavefront divergence.

Claims (59)

1. A display system comprising:

a head-mountable frame;

a display device disposed on the head-mountable frame and configured to present virtual content to a user, wherein the display device comprises a waveguide stack including a plurality of waveguides, wherein one or more waveguides of the plurality of waveguides are configured to output light with a different amount of wavefront divergence than one or more other waveguides of the plurality of waveguides;

one or more sensors attached to the head-mountable frame, the one or more sensors comprising a first sensor configured to collect user-specific data and a second sensor configured to collect environmental data, the user-specific data comprising behavioral information characterizing behavior of the user and the second sensor comprising an outwardly-facing camera;

one or more processors; and

one or more computer storage media storing instructions that, when executed by the system, cause the system to perform operations comprising:

collecting the user-specific data and the environmental data from the first and the second sensors;

correlating the environmental data with the behavioral information;

applying the behavioral information and the environmental data to an initial prediction model;

continuing to collect additional behavioral information and additional environmental data from the one or more sensors; and

updating the initial prediction model based on the additional behavioral information and the additional environmental data to produce a revised prediction model.

2. The display system of claim 1 , wherein the display device is configured to output images with an accommodation-vergence mismatch less than about 0.25 diopters.

3. The display system of claim 1 , wherein applying the data comprises determining an outcome of the prediction model.

4. The display system of claim 1 , wherein applying the data comprises determining that a criterion or set of criteria for the data are satisfied before determining an outcome of the prediction model.

5. The display system of claim 1 , wherein the display device is configured to continue to collect additional behavioral information from the first sensor over a period of at least one month.

6. The display system of claim 5 , wherein the display device is configured to continue to collect additional behavioral information from the first sensor for durations of about 3 hours or more a day over the period.

7. The display system of claim 1 , wherein an amount of the additional behavioral information to be collected is determined based on one or more statistical power analyses.

8. The display system of claim 1 , wherein updating the initial prediction model comprises modifying at least one of a measured variable within the behavioral information, a number of behavioral information points to be collected, a time period for collecting the additional behavioral information, or a weighting of one or more measured variables.

9. The display system of claim 1 , wherein updating the initial prediction model comprises correlating data from the first sensor and the second sensor.

10. The display system of claim 1 , further comprising

a remote data repository comprising the one or more computer storage media storing the instructions; and

a remote processing module comprising the one or more processors, wherein the one or more processors are configured to perform the operations.

11. A method of conducting a user health analysis, the method comprising:

collecting user-specific data and environmental data from one or more sensors attached to a head-mountable frame of an augmented reality display system having an augmented reality display device comprising a waveguide stack including a plurality of waveguides, wherein one or more waveguides of the plurality of waveguides are configured to output light with a different amount of wavefront divergence than one or more other waveguides of the plurality of waveguides, the one or more sensors comprising a first sensor configured to collect the user-specific data and a second sensor configured to collect the environmental data, the user-specific data comprising behavioral information characterizing behavior of the user and the second sensor comprising an outwardly-facing camera;

correlating the environmental data with the behavioral information;

applying the behavioral information and the environmental data to an initial prediction model;

continuing to collect additional behavioral information and additional environmental data from the one or more sensors; and

updating the initial prediction model based on the additional behavioral information and the additional environmental data to produce a revised prediction model.

12. The method of claim 11 , further comprising displaying virtual content on the display device with an accommodation-vergence mismatch of less than about 0.25 diopters.

13. The method of claim 11 , further comprising displaying virtual content on the display device with an accommodation-vergence mismatch of less than about 0.33 diopters.

14. The method of claim 11 , wherein continuing to collect additional behavioral information comprises collecting additional behavioral information from the first sensor over a period of at least one month.

15. The method of claim 14 , wherein continuing to collect additional behavioral information comprises collecting additional behavioral information from the first sensor for durations of about 3 hours or more a day over the period.

16. The method of claim 11 , wherein an amount of the additional behavioral information to be collected is determined based on one or more statistical power analyses.

17. The method of claim 11 , wherein updating the initial prediction model comprises modifying at least one of a measured variable within the behavioral information, a number of behavioral information points to be collected, a time period for collecting the additional behavioral information, or a weighting of one or more measured variables.

18. The method of claim 11 , wherein updating the initial prediction model comprises correlating behavioral information from the first sensor and environmental data from the second sensor.

19. A display system comprising:

a display device disposed on a head-mountable frame and configured to present virtual content to a user, wherein the display device comprises a waveguide stack including a plurality of waveguides, wherein one or more waveguides of the plurality of waveguides are configured to output light with a different amount of wavefront divergence than one or more other waveguides of the plurality of waveguides;

one or more sensors attached to the head-mountable frame, the one or more sensors comprising a first sensor configured to collect user-specific data and a second sensor configured to collect environmental data, the user-specific data comprising behavioral information characterizing behavior of the user and the second sensor comprising an outwardly-facing camera;

one or more processors; and

one or more computer storage media storing instructions that, when executed by the system, cause the system to perform operations comprising:

collecting the user-specific data and the environmental data from the first and the second sensors;

correlating the environmental data with the behavioral information;

applying the behavioral information and the environmental data to an initial prediction model;

continuing to collect additional behavioral information and additional environmental data from the one or more sensors; and

causing the display device to initiate an intervention by presenting virtual content to the user based on the additional behavioral information and the additional environmental data.

20. The display system of claim 19 , wherein the display device is configured to output images with an accommodation-vergence mismatch less than about 0.25 diopters.

21. The display system of claim 19 , wherein the display device is configured to continue to collect additional behavioral information from the first sensor over a period of at least one month.

22. The display system of claim 21 , wherein the display device is configured to continue to collect additional behavioral information from the first sensor for durations of about 3 hours or more a day over the period.

23. The display system of claim 19 , wherein the intervention is a treatment for a condition indicated by an outcome of the prediction model.

24. A method of conducting a user health analysis, the method comprising:

collecting user-specific data and environmental data from one or more sensors disposed on a head-mountable frame of an augmented reality display system having an augmented reality display device comprising a waveguide stack including a plurality of waveguides, wherein one or more waveguides of the plurality of waveguides are configured to output light with a different amount of wavefront divergence than one or more other waveguides of the plurality of waveguides, the one or more sensors comprising a first sensor configured to collect the user-specific data and a second sensor configured to collect the environmental data, the user-specific data comprising behavioral information characterizing behavior of the user and the second sensor comprising an outwardly-facing camera;

correlating the environmental data with the behavioral information;

applying the behavioral information and the environmental data to an initial prediction model;

continuing to collect additional behavioral information and additional environmental data from the one or more sensors; and

causing the augmented reality display system to initiate an intervention by presenting virtual content to a wearer of the augmented reality display system based on the additional behavioral information and the additional environmental data.

25. The method of claim 24 , further comprising displaying virtual content on the display device with an accommodation-vergence mismatch of less than about 0.25 diopters.

26. The method of claim 24 , wherein continuing to collect additional behavioral information comprises collecting additional behavioral information from the first sensor over a period of at least one month.

27. The method of claim 26 , wherein continuing to collect additional behavioral information comprises collecting additional behavioral information from the first sensor for durations of about 3 hours or more a day over the period.

28. The method of claim 24 , wherein the intervention is a treatment for a condition indicated by an outcome of the prediction model.

Assignments (11)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2020
From: SAMEC, NICOLE ELIZABETH; BAERENRODT, MARK; ROBAINA, NASTASJA U.; KRAMER, NICHOLAS ATKINSON; VINKERS, CHARLOTTE DOROTHEA WILHELMINA
To: MAGIC LEAP, INC.
Reel/Frame 053907/0679 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2020
From: HARRISES, CHRISTOPHER M.; TECHNICAL SOLUTIONS, INC.
To: MAGIC LEAP, INC.
Reel/Frame 053907/0627 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: SAMEC, NICOLE ELIZABETH; BAERENRODT, MARK; KRAMER, NICHOLAS
To: MAGIC LEAP, INC.
Reel/Frame 047060/0445 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: SAMEC, NICOLE ELIZABETH; BAERENRODT, MARK; KRAMER, NICHOLAS; ROBAINA, NASTASJA U.
To: MAGIC LEAP, INC.
Reel/Frame 047060/0474 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: HARRISES, CHRISTOPHER M.; TECHNICAL SOLUTIONS, INC.
To: MAGIC LEAP, INC.
Reel/Frame 047060/0421 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: SAMEC, NICOLE ELIZABETH; BAERENRODT, MARK; KRAMER, NICHOLAS
To: MAGIC LEAP, INC.
Reel/Frame 047060/0338 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: HARRISES, CHRISTOPHER M.; TECHNICAL SOLUTIONS, INC.
To: MAGIC LEAP, INC.
Reel/Frame 047060/0311 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 3, 2018
From: HARRISES, CHRISTOPHER M.; TECHNICAL SOLUTIONS, INC.
To: MAGIC LEAP, INC.
Reel/Frame 047060/0495 →
Continuity (4)
Provisional Application 62333734 · May 9, 2016
Provisional Application 62366576 · Jul 25, 2016
Provisional Application 62440348 · Dec 29, 2016
Related Publication 20170323485A1 · Nov 9, 2017
Cited By (2)
US 12,293,834 US 12,303,203