IP Library Granted Patent US 11,782,529
Granted Patent B2
US 11,782,529 · App. 17/961,958 · Granted Oct 10, 2023

External user interface for head worn computing

Inventor: Ralph F. Osterhout (San Francisco, CA)
Assignee: Mentor Acquisition One, LLC
G06F3/03545B43K29/00B43K29/004B43K29/08G02B27/017G06F3/012G06F3/0304G06F3/0317G06F3/042G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,782,529
App. No.
17/961,958
Granted
Oct 10, 2023
Kind
B2
Abstract

Aspects of the present invention relate to external user interfaces used in connection with head worn computers (HWC).

Claims (34)

1. A method, comprising:

presenting, via a see-through display of a wearable device, a virtual line as a guide for alignment of one or more input strokes written by a wearer of the wearable head device;

tracking a motion of a pen held by the wearer as the pen moves along a surface relative to the virtual line, wherein the tracking comprises:

receiving, via an IMU located on the pen, a signal indicative of movement of the pen; and

obtaining images of the pen as the pen moves relative to the virtual line;

determining, based on the signal, a first relative motion between a tip of the pen and the virtual line;

determining, based on the images, a second relative motion between the tip of the pen and the virtual line; and

determining an input sequence corresponding to the one or more input strokes, wherein the input sequence is determined based on a comparison of the first relative motion and the second relative motion.

2. The method of claim 1 , wherein the virtual line is presented as a portion of a note.

3. The method of claim 1 , wherein the virtual line is presented as a portion of a message template.

4. The method of claim 1 , wherein the virtual line is presented as a portion of an email template.

5. The method of claim 1 , wherein the virtual line is presented as a portion of a writing teaching platform.

6. The method of claim 1 , wherein the virtual line is presented as a portion of a painting teaching platform.

7. The method of claim 1 wherein the virtual line is presented as a portion of a drawing teaching platform.

8. The method of claim 1 , wherein the input sequence is presented via the see-through display relative to the virtual line.

9. The method of claim 8 , wherein the presentation is in real time.

10. The method of claim 1 , wherein determining the input sequence of the pen includes determining a relative position between the tip of the pen and the virtual line.

11. The method of claim 1 , wherein tracking the motion of the pen held by the wearer as the pen moves along the surface relative to the virtual line further comprises receiving surface images captured by a pen tip camera located on the pen.

12. The method of claim 11 , wherein determining the input sequence of the pen further comprises determining, based on the surface images, a third relative motion between the tip of the pen and the virtual line and in accordance with a discrepancy between the first relative motion and the second relative motion, determining the input sequence of the pen further based on the third relative motion.

13. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of one or more electronic devices, cause the electronic devices to:

present, via a see-through display of a wearable device, a virtual line as a guide for alignment of input strokes written by a wearer of the wearable head device;

track a relative motion of a pen held by the wearer as the pen moves along a surface relative to the virtual line, wherein the tracking comprises:

receiving, via an IMU located on the pen, a signal indicative of movement of the pen; and

obtaining images of the pen as the pen moves relative to the virtual line;

determine, based on the signal, a first relative motion between a tip of the pen and the virtual line;

determine, based on the images, a second relative motion between the tip of the pen and the virtual line; and

determine an input sequence corresponding to the one or more input strokes, wherein the input sequence is determined based on a comparison of the first relative motion and the second relative motion.

14. The non-transitory computer-readable storage medium of claim 13 , wherein determining the input sequence of the pen includes determining a relative position between the tip of the pen and the virtual line.

15. The non-transitory computer-readable storage medium of claim 13 , wherein tracking the relative motion of the pen held by the wearer as the pen moves along the surface relative to the virtual line further comprises receiving surface images captured by a pen tip camera located on the pen.

16. The non-transitory computer-readable storage medium of claim 15 , wherein determining the input sequence of the pen further comprises determining, based on the surface images, a third relative motion between the tip of the pen and the virtual line and in accordance with a discrepancy between the first relative motion and the second relative motion, determining the input sequence of the pen further based on the third relative motion.

17. The non-transitory computer-readable storage medium of claim 13 , wherein the input sequence is presented via the see-through display relative to the virtual line.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the presentation is in real time.

19. The non-transitory computer-readable storage medium of claim 13 , wherein the virtual line is presented as a portion of at least one of a note, a message template, and an email.

20. The non-transitory computer-readable storage medium of claim 13 , wherein the virtual line is presented as a portion of at least one of a writing teaching platform, a painting teaching platform, and a drawing teaching platform.

Assignments (3)
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073031/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 9, 2025
From: OSTERHOUT, RALPH F.
To: OSTERHOUT GROUP, INC.
Reel/Frame 072529/0446 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
Continuity (5)
Continuation 17494746 · Oct 5, 2021
Continuation 16378191 · Apr 8, 2019
Continuation 14280752 · May 19, 2014
Continuation In Part 14158198 · Jan 17, 2014
Related Publication 20230030938A1 · Feb 2, 2023
Cited By (1)
US 12,443,294