IP Library › Granted Patent US 10,437,323
Granted Patent B2
US 10,437,323 · App. 14/816,012 · Granted Oct 8, 2019

Controlling a head mounted device

Inventors: Shay Solomon (Kfar-Saba, IL); Evyatar Meiron (Kfar-Saba, IL); Alex Rapoport (Tel-Aviv, IL)
Assignee: FIELDBIT LTD.
G06F3/012G02B27/0093G02B27/017G06F3/04842G02B2027/0138G02B2027/0141G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,437,323
App. No.
14/816,012
Filed
Aug 2, 2015
Granted
Oct 8, 2019
Kind
B2
Art Unit
2693
USPC
345/8
Abstract

A method for controlling a display of a head mounted display (HMD) device, the method may include projecting on a display of the HMD device a given user interface element; sensing, by a sensor, a head movement of a user of the HMD device; unchanging a position of the projection of the given user interface element when a speed of the head movement is below a first speed threshold; and changing the position of the given user interface element when the speed of the head movement exceeds the first speed threshold.

Claims (72)

1. A controller to perform a method for controlling a display of a head mounted display (HMD) device, the method comprising the steps of:

(a) projecting a given user interface element and a selectable user interface element on the display;

(b) sensing, by a sensor, a user head movement of the HMD device;

(c) unchanging a given projection position of a projection of said given user interface element when a speed of said user head movement is below a first speed threshold;

(d) changing said given projection when said speed exceeds said first speed threshold;

(e) returning said given user interface element and said selectable user interface element to an original position upon sensing said user head movement is paused for a predefined reset period and without overlap between said given user interface element and said selectable user interface element, wherein said given user interface element is projected on a center of the display when in said original position;

(f) associating multiple frames of user selectable interface elements with multiple segments of a three-dimensional space having an axis system;

(g) changing an axis position, by tracking said user head movement using said axis system, when said speed is below said first speed threshold or is above a second speed threshold that exceeds said first speed threshold;

(h) unchanging said axis position when said speed is between said first speed threshold and said second speed threshold; and

(i) displaying at least one of said multiple frames based on a relationship between a field of view of the HMD device and said axis system.

2. The controller of claim 1 , the method further comprising the steps of:

(j) projecting said given user interface element on a fixed location of the display; and

(k) determining a selection of said selectable user interface element when said given user interface element and said selectable user interface element overlap for a predefined overlap period.

3. The controller of claim 2 , wherein said step of determining includes determining said selection upon detecting no user head movement during said predefined overlap period.

4. The controller of claim 3 , wherein said predefined overlap period exceeds one second.

5. The controller of claim 1 , wherein said predefined reset period exceeds a predefined overlap period, the method further comprising the step of:

(j) determining a selection of said selectable user interface element when said given user interface element and said selectable user interface element overlap for said predefined overlap period.

6. The controller of claim 5 , wherein said predefined reset period is 1.2 seconds long, and wherein said predefined overlap period is one second long.

7. The controller of claim 1 , wherein said step of changing of said given projection position includes tracking said user head movement.

8. The controller of claim 1 , the method further comprising the step of changing said first speed threshold based on a history of user head movements.

9. The controller of claim 1 , the method further comprising the steps of:

(j) associating a first frame of user interface elements with an area of the display; and

(k) unchanging a first-frame position of a projection of said first frame when said speed is below said first speed threshold;

wherein said first frame includes said given user interface element.

10. The controller of claim 1 , wherein the HMD device is a see-through binocular head mounted display (STBHMD) device, the method further comprising the steps of:

(j) performing a calibration process based on:

(i) at least a first image of an objects; and

(ii) user feedback of said STBHMD device;

(k) receiving an input augmented image of said object, wherein said input augmented image includes a certain object image and input digital content referring to a certain object element;

(l) calculating, in response to an outcome of said calibration process, output digital content referring to a visual representation of said certain object element in secondary images of said object; and

(m) displaying said output digital content on the display of said STBHMD device, thereby forming output augmented images of said object.

11. The controller of claim 1 , wherein said given user interface element overlaps with multiple selectable user interface elements, the method further comprising the step of:

(j) selecting a selectable user interface element A, having the largest overlap area with said given user interface element, out of said multiple selectable user interface elements.

12. A head mounted display (HMD) device comprising:

(a) a camera;

(b) a display;

(c) a sensor configured to sense a user head movement of the HMD device;

(d) a processor configured for:

(i) associating multiple frames of user selectable interface elements with multiple segments of a three-dimensional space having an axis system;

(ii) changing an axis position, by tracking said user head movement using said axis system, when a speed of said user head movement is below a first speed threshold or is above a second speed threshold that exceeds said first speed threshold; and

(iii) unchanging said axis position when said speed is between said first speed threshold and said second speed threshold; and

(e) a projector configured to project on said display of the HMD device a given user interface element and a selectable user interface element;

wherein the projector is configured, under a control of said processor, for:

(i) unchanging a given projection position of a projection of said given user interface element when said speed is below said first speed threshold;

(ii) changing said given projection position when said speed exceeds said first speed threshold;

(iii) unchanging a selectable projection position of a projection of said selectable user interface element when said speed is below said first speed threshold;

(iv) changing said selectable projection position when said speed exceeds said first speed threshold;

(v) returning said given user interface element and said selectable user interface element to an original position upon sensing said user head movement is paused for a predefined reset period without overlap between said given user interface element and said selectable user interface element, wherein said given user interface element is projected on a center of said display when in said original position; and

(vi) displaying at least one of said multiple frames based on a relationship between a field of view of the HMD device and said axis system.

13. The HMD device of claim 12 , wherein said processor is further configured for:

(iv) determining a selection of said selectable user interface element when said given user interface element and said selectable user interface element overlap for a predefined overlap period; and

wherein said projector is further configured for:

(vii) projecting said given user interface element on a fixed location of said display.

14. The HMD device of claim 13 , wherein said processor is further configured for:

(v) determining said selection upon detecting no user head movement during said predefined overlap period.

15. The HMD device of claim 14 , wherein said predefined overlap period exceeds one second.

16. The HMD device of claim 12 , wherein said changing said selectable projection is responsive to a tracking of said user head movement.

17. The HMD device of claim 12 , wherein said processor is further configured for:

(iv) changing said first speed threshold based on a history of user head movements.

18. The HMD device of claim 12 , wherein said processor is further configured for:

(iv) associating a first frame of user interface elements with an area of said display; and

(v) unchanging a first-frame position of a projection of said first frame when said speed is below said first speed threshold;

wherein said first frame includes said given user interface element.

19. A non-transitory computer readable medium, having computer-readable code embodied on the non-transitory computer-readable medium for controlling a display of a head mounted display (HMD) device, the computer-readable code comprising:

(a) program code for projecting a given user interface element and a selectable user interface element on the display;

(b) program code for sensing, by a sensor, a user head movement of the HMD device;

(c) program code for unchanging a given projection position of a projection of said given user interface element when a speed of said user head movement is below a first speed threshold;

(d) program code for changing said given projection position when said speed exceeds said first speed threshold;

(e) program code for associating multiple frames of user selectable interface elements with multiple segments of a three-dimensional space having an axis system;

(f) program code for changing an axis position, by tracking said user head movement using said axis system, when said speed is below said first speed threshold or is above a second speed threshold that exceeds said first speed threshold;

(g) program code for unchanging said axis position when said speed is between said first speed threshold and said second speed threshold; and

(h) program code for displaying at least one of said multiple frames based on a relationship between a field of view of the HMD device and said axis system.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2015
From: SOLOMON, SHAY; MEIRON, EVYATAR; RAPOPORT, ALEX
To: FIELDBIT LTD.
Reel/Frame 036454/0119 →
Continuity (2)
Continuation In Part 14726542 · May 31, 2015
Related Publication 20160349838A1 · Dec 1, 2016