Granted Patent B2

US 12,443,294 · App. 18/752,188 · Granted Oct 14, 2025

Inventor: Ralph F. Osterhout (San Francisco, CA)

Assignee: Mentor Acquisition One, LLC

G06F3/03545B43K29/00B43K29/004B43K29/08G02B27/017G06F3/012G06F3/0304G06F3/0317G06F3/042G02B2027/0187

Abstract

Claims (40)

1. A method, comprising:

displaying a line in a mixed reality environment, the mixed reality environment visible to a user of a wearable device via a display of the wearable device;

tracking a motion of a finger of the user relative to the line, wherein said tracking comprises:

receiving, via a sensor, a signal indicative of the motion of the finger of the user relative to the line; and

obtaining images of the motion of the finger;

determining, based on the signal, a first relative motion between the finger and the line;

determining, based on the images, a second relative motion between the finger and the line; and

determining an input sequence corresponding to one or more input strokes of the user, wherein:

the input sequence is determined based on a comparison of the first relative motion and the second relative motion,

the comparison indicates a movement of the line relative to the motion of the finger, and

the determining the input sequence based on the comparison comprises compensating for the movement of the line relative to the motion of the finger.

2. The method of claim 1 , further comprising displaying a message in the mixed reality environment, wherein the message comprises the line.

3. The method of claim 1 , further comprising displaying an email in the mixed reality environment, wherein the email comprises the line.

4. The method of claim 1 , further comprising displaying a user interface in the mixed reality environment, wherein the user interface comprises the line.

5. The method of claim 4 , wherein the user interface is associated with a writing platform.

6. The method of claim 4 , wherein the user interface is associated with a painting platform.

7. The method of claim 4 , wherein the user interface is associated with a drawing platform.

8. The method of claim 1 , further comprising presenting the input sequence via the display relative to the line.

9. The method of claim 8 , wherein the input sequence is presented concurrently with the one or more input strokes of the user.

10. The method of claim 1 , wherein said determining the input sequence comprises determining a position of a tip of the finger relative to the line.

11. The method of claim 1 , wherein the motion of the finger comprises motion on a surface in the mixed reality environment and wherein the images comprise images of the surface.

12. The method of claim 1 , further comprising displaying an object in the mixed reality environment, wherein the object is associated with the input sequence.

13. The method of claim 1 , wherein the line comprises a virtual line.

14. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

displaying a line in a mixed reality environment, the mixed reality environment visible to a user of a wearable device via a display of the wearable device;

tracking a motion of a finger of the user relative to the line, wherein said tracking comprises:

receiving, via a sensor, a signal indicative of the motion of the finger of the user relative to the line; and

obtaining images of the motion of the finger;

determining, based on the signal, a first relative motion between the finger and the line;

determining, based on the images, a second relative motion between the finger and the line; and

determining an input sequence corresponding to one or more input strokes of the user, wherein:

the input sequence is determined based on a comparison of the first relative motion and the second relative motion,

the comparison indicates a movement of the line relative to the motion of the finger, and

the determining the input sequence based on the comparison comprises compensating for the movement of the line relative to the motion of the finger.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the method further comprises displaying a message in the mixed reality environment, wherein the message comprises the line.

16. The non-transitory computer-readable storage medium of claim 14 , wherein the method further comprises presenting the input sequence via the display relative to the line.

17. The non-transitory computer-readable storage medium of claim 14 , wherein said determining the input sequence comprises determining a position of a tip of the finger relative to the line.

18. The non-transitory computer-readable storage medium of claim 14 , wherein the motion of the finger comprises motion on a surface in the mixed reality environment and wherein the images comprise images of the surface.

19. The non-transitory computer-readable storage medium of claim 14 , wherein the method further comprises displaying an object in the mixed reality environment, wherein the object is associated with the input sequence.

20. The non-transitory computer-readable storage medium of claim 14 , wherein the line comprises a virtual line.

Assignments (4)

Continuity (7)

References Cited (400)