IP Library Granted Patent US 11,227,294
Granted Patent B2
US 11,227,294 · App. 14/244,417 · Granted Jan 18, 2022

Sight information collection in head worn computing

Inventors: Edward H. Nortrup (Stoneham, MA); John N. Border (Eaton, NH)
Assignee: Mentor Acquisition One, LLC
G06Q30/0201G06F3/013
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,227,294
App. No.
14/244,417
Granted
Jan 18, 2022
Kind
B2
Abstract

Aspects of the present invention relate to methods and systems for collecting an using eye heading and sight heading information in head worn computing.

Claims (46)

1. A method, comprising:

determining a first geo-spatial location corresponding to a first location in proximity to a physical object;

determining a first viewing information for a first user of a plurality of wearable head device users, and a second viewing information for a second user of the plurality of wearable head device users, wherein:

the first viewing information is determined using an initial GPS location of the first user's wearable head device at a first time, an orientation of the first user's wearable head device, and an image of a first eye of the first user, the image captured by the first user's wearable head device, and

the second viewing information is determined using an initial GPS location of the second user's wearable head device at a second time, an orientation of the second user's wearable head device, and an image of a first eye of the second user, the image captured by the second user's wearable head device;

determining an alignment of the first user with the first geo-spatial location based on the initial GPS location of the first user's wearable head device;

determining, based on the first viewing information and the alignment of the first user with the first geo-spatial location, whether the first user is viewing the physical object;

in accordance with a determination that the first user is viewing the physical object, determining, at a third time, a subsequent GPS location of the first user's wearable head device;

generating a first personal profile for the first user, the first personal profile identifying a relationship between the physical object and a movement of the first user, the movement determined using the initial GPS location and the subsequent GPS location of the first user's wearable head device;

determining an alignment of the second user with the first geo-spatial location based on the initial GPS location of the second user's wearable head device;

determining, based on the second viewing information and the alignment of the second user with the first geo-spatial location, whether the second user is viewing the physical object;

in accordance with a determination that the second user is viewing the physical object, determining, at a fourth time, a subsequent GPS location of the second user's wearable head device;

generating a second personal profile for the second user, the second personal profile identifying a relationship between the physical object and a movement of the second user, the movement determined using the initial GPS location and the final GPS location of the second user's wearable head device; and

developing a crowd profile associated with the physical object based on the first personal profile and the second personal profile,

wherein the crowd profile identifies a viewing preference of the plurality of wearable head device users.

2. The method of claim 1 , wherein the crowd profile further identifies a relationship between the physical object and a characteristic of the plurality of users.

3. The method of claim 2 , wherein the crowd profile further identifies a relationship between the physical object and an average length of time for which the plurality of users view the physical object.

4. The method of claim 1 wherein the image of the first eye of the first user includes a reflection of a content being displayed via the first user's wearable head device, and wherein the image of the first eye of the second user includes a reflection of a content being displayed via the second user's wearable head device.

5. The method of claim 4 , wherein the content being displayed via the first user's wearable head device includes the object, and wherein the content being displayed via the second user's wearable head device includes the object.

6. The method of claim 1 , further comprising:

capturing an image of a second eye of the first user, wherein the image of the first eye of the first user and the image of the second eye of the first user are collected via a first camera disposed in line with an optical path internal to content display optics of the first user's wearable head device; and

capturing an image of a second eye of the second user, wherein the image of the first eye of the second user and the image of the second eye of the second user are collected via a second camera disposed in line with an optical path internal to content display optics of the second user's wearable head device.

7. The method of claim 6 , wherein the image of the first eye of the first user and the image of the second eye of the first user are used to determine an amount of convergence of the first eye of the first user and the second eye of the first user and to determine a focal plane of the first user, and wherein the image of the first eye of the second user and the image of the second eye of the second user are used to determine an amount of convergence of the first eye of the second user and the second eye of the second user and to determine a focal plane of the second user.

8. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors cause the one or more processors to perform a method comprising:

determining a first geo-spatial location corresponding to a first location in proximity to a physical object;

determining a first viewing information for a first user of a plurality of wearable head device users, and a second viewing information for a second user of the plurality of wearable head device users, wherein:

the first viewing information is determined using an initial GPS location of the first user's wearable head device at a first time, an orientation of the first user's wearable head device, and an image of a first eye of the first user, the image captured by the first user's wearable head device, and

the second viewing information is determined using an initial GPS location of the second user's wearable head device at a second time, an orientation of the second user's wearable head device, and an image of a first eye of the second user, the image captured by the second user's wearable head device;

determining an alignment of the first user with the first geo-spatial location based on the initial GPS location of the first user's wearable head device;

determining, based on the first viewing information and the alignment of the first user with the first geo-spatial location, whether the first user is viewing the physical object;

in accordance with a determination that the first user is viewing the physical object, determining, at a third time, a subsequent GPS location of the first user's wearable head device;

generating a first personal profile for the first user, the first personal profile identifying a relationship between the physical object and a movement of the first user, the movement determined using the initial GPS location and the subsequent GPS location of the first user's wearable head device;

determining an alignment of the second user with the first geo-spatial location based on the initial GPS location of the second user's wearable head device;

determining, based on the first viewing information and the alignment of the second user with the first geo-spatial location, whether the second user is viewing the physical object;

in accordance with a determination that the second user is viewing the physical object, determining, at a fourth time, a subsequent GPS location of the second user's wearable head device;

generating a second personal profile for the second user, the second personal profile identifying a relationship between the physical object and a movement of the second user, the movement determined using the initial GPS location and the final GPS location of the second user's wearable head device; and

developing a crowd profile associated with the physical object based on the first personal profile and the second personal profile,

wherein the crowd profile identifies a viewing preference of the plurality of wearable head device users.

9. The non-transitory computer-readable medium of claim 8 , wherein the crowd profile further identifies a relationship between the physical object and a characteristic of the plurality of users.

10. The non-transitory computer-readable medium of claim 9 , wherein the crowd profile further identifies a relationship between the physical object and an average length of time for which the plurality of users view the physical object.

11. The non-transitory computer-readable medium of claim 8 wherein the image of the first eye of the first user includes a reflection of a content being displayed via the first user's wearable head device, and wherein the image of the first eye of the second user includes a reflection of a content being displayed via the second user's wearable head device.

12. The non-transitory computer-readable medium of claim 11 , wherein the content being displayed via the first user's wearable head device includes the object, and wherein the content being displayed via the second user's wearable head device includes the object.

13. The non-transitory computer-readable medium of claim 8 , further comprising:

capturing an image of a second eye of the first user, wherein the image of the first eye of the first user and the image of the second eye of the first user are collected via a first camera disposed in line with an optical path internal to content display optics of the first user's wearable head device; and

capturing an image of a second eye of the second user, wherein the image of the first eye of the second user and the image of the second eye of the second user are collected via a second camera disposed in line with an optical path internal to content display optics of the second user's wearable head device.

14. The non-transitory computer-readable medium of claim 13 , wherein the image of the first eye of the first user and the image of the second eye of the first user are used to determine an amount of convergence of the first eye of the first user and the second eye of the first user and to determine a focal plane of the first user, and wherein the image of the first eye of the second user and the image of the second eye of the second user are used to determine an amount of convergence of the first eye of the second user and the second eye of the second user and to determine a focal plane of the second user.

Assignments (9)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF SECURITY INTEREST IN PATENTS Recorded Nov 7, 2019
From: JPMORGAN CHASE BANK, N.A.
To: CITIBANK, N.A.
Reel/Frame 050967/0138 →
PATENT SECURITY AGREEMENT Recorded Aug 22, 2019
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: JP MORGAN CHASE BANK, N.A.
Reel/Frame 050138/0287 →
ASSIGNMENT OF PATENTS AND PATENT APPLICATIONS Recorded Apr 18, 2019
From: JGB COLLATERAL, LLC
To: MENTOR ACQUISITION ONE, LLC
Reel/Frame 048945/0939 →
SECURED PARTY BILL OF SALE Recorded Apr 18, 2019
From: OSTERHOUT GROUP, INC.
To: JGB COLLATERAL, LLC
Reel/Frame 051018/0883 →
SECURITY INTEREST Recorded Mar 15, 2018
From: OSTERHOUT GROUP, INC.
To: JGB COLLATERAL, LLC
Reel/Frame 045606/0295 →
SECURITY INTEREST Recorded Oct 3, 2017
From: OSTERHOUT GROUP, INC.
To: O-FILM GLOBAL (HK) TRADING LIMITED
Reel/Frame 044127/0501 →
SECURITY INTEREST Recorded Sep 28, 2017
From: OSTERHOUT GROUP, INC.
To: 21ST CENTURY FOX AMERICA, INC.
Reel/Frame 044052/0367 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 28, 2015
From: NORTRUP, EDWARD H.; BORDER, JOHN N.
To: OSTERHOUT GROUP, INC.
Reel/Frame 034829/0539 →
Continuity (1)
Related Publication 20150287048A1 · Oct 8, 2015
Cited By (3)
US 12,282,171 US 12,493,891 US 12,613,414