Reactive user interface for head-mounted display
Embodiments of the present invention are directed toward allowing a see-through HMD to intelligently react to sensor and other input to adjust how visual elements showing video are displayed. The HMD can react to different sensor input in different ways, enabling the HMD to provide graduated responses based on the degree and type of sensor input received. These responses may further be based on a determination of the context of the user to help ensure an intelligent, appropriate response.
1. A method of providing a reactive video playback interface for a head-mounted display, the method comprising:
causing a display of the head-mounted display to show at least one visual element including a video, wherein the display is at least semi-transparent from a view of a user of the head-mounted display when no content is displayed;
detecting a triggering event using a first set of sensor data from one or more sensors;
changing a visibility of the video based on the triggering event while the video continues to be shown; and
causing the display to show at least one new visual element based on the triggering event while the video continues to be shown, the at least one new visual element indicative of an environmental condition.
2. The method of claim 1 , further comprising determining a context of the user, based on a second set of sensor data, wherein changing the visibility of the video is further based on the determined context.
3. The method of claim 1 , wherein the triggering event comprises at least one of:
a certain movement of the user,
a transition of the user from one state of movement to another,
a proximity of an object to the user,
a proximity of the user to a specific person,
entry of the user into a vehicle,
exit of the user from a vehicle,
a pattern of events,
a loud noise, or
arrival of the user at a certain location.
4. The method of claim 3 , wherein the certain movement of the user comprises at least one of:
walking, running, or lying down,
a turning of the user's head,
the user standing up,
the user sitting down, or
the user falling down.
5. The method of claim 1 , wherein the one or more sensors comprise at least one of:
a motion sensor,
an inertial sensor,
a light sensor,
a microphone, or
an antenna.
6. The method of claim 5 , wherein the one or more sensors are attached to the head-mounted display.
7. The method of claim 1 , wherein changing the visibility of the video includes increasing a transparency of the video.
8. The method of claim 1 , wherein changing the visibility of the video includes reducing a size of the video.
9. The method of claim 1 , further including pausing playback of the video.
10. The method of claim 1 , wherein:
detecting the triggering event comprises detecting a movement; and
changing the visibility of the video includes changing the visibility of the video in proportion to at least one of a speed or an amount of the detected movement.
11. The method of claim 1 , further including:
causing the head-mounted display to playback audio related to the video; and
reducing a volume of the audio based on the triggering event.
12. A head-mounted display system comprising:
display means for displaying one or more visual elements to a user such that the user is able to at least selectively view the user's surroundings;
sensing means configured to gather sensor data;
means for causing the display means to show a visual element including a video;
means for detecting a triggering event using a first set of sensor data from the sensing means;
means for changing a visibility of the video shown with the display means, based on the triggering event, while the video continues to be shown; and
means for causing the display means to show at least one new visual element based on the triggering event while the video continues to be shown, the at least one new visual element indicative of an environmental condition.
13. The head-mounted display system of claim 12 , further comprising
means for determining a context of the user, based on a second set of sensor data;
wherein the visibility change of the video is based on the determined context.
14. The head-mounted display system of claim 12 , wherein the triggering event comprises at least one of:
a certain movement of the user,
a transition of the user from one state of movement to another,
a proximity of an object to the user,
a proximity of the user to a specific person,
entry of the user into a vehicle,
exit of the user from a vehicle,
a pattern of events,
a loud noise, or
arrival of the user at a certain location.
15. The head-mounted display system of claim 14 , wherein the certain movement comprises at least one of:
walking, running, or lying down,
a turning of the user's head,
the user standing up,
the user sitting down, or
the user falling down.
16. The head-mounted display system of claim 12 , wherein the sensing means comprises at least one of:
a motion sensor,
an inertial sensor,
a light sensor,
a microphone, or
an antenna.
17. The head-mounted display system of claim 12 , wherein the means for changing comprise means for changing the visibility of the video by increasing a transparency of the video.
18. The head-mounted display system of claim 12 , wherein the means for changing comprise means for changing the visibility of the video by reducing a size of the video.
19. The head-mounted display system of claim 12 , further comprising means for pausing playback of the video.
20. The head-mounted display system of claim 12 , further comprising:
means for detecting a movement comprising the triggering event; and
means for changing the visibility of the video in proportion to at least one of a speed or an amount of the detected movement.
21. The head-mounted display system of claim 12 , further including:
audio means configured to playback audio related to the video;
wherein means for changing reduce a volume of the audio based on the triggering event.
22. A head-mounted display device comprising:
a body configured to be worn on a user's head;
a display, coupled with the body and configured to allow the user to view:
the user's surroundings, and
a visual element shown with the display;
at least one sensor configured to provide sensor data; and
a processing unit communicatively coupled to the display and the at least one sensor and configured to:
cause the display to show the visual element, the visual element including a video;
detect a triggering event using a first set of sensor data from the at least one sensor;
cause the display to change a visibility of the video shown with the display, based on the triggering event, while the video continues to be shown; and
cause the display to show at least one new visual element based on the triggering event while the video continues to be shown, the at least one new visual element indicative of an environmental condition.
23. The head-mounted display device of claim 22 , wherein the processing unit is further configured to:
determine a context of the user, based on a second set of sensor data; and
cause the display to change the visibility of the video further based on the determined context.
24. The head-mounted display device of claim 22 , wherein the processing unit is further configured to determine the triggering event by determining an event comprising at least one of:
a certain movement of the user,
a transition of the user from one state of movement to another,
a proximity of an object to the user,
a proximity of the user to a specific person,
entry of the user into a vehicle,
exit of the user from a vehicle,
a pattern of events,
a loud noise, or
arrival of the user at a certain location.
25. The head-mounted display device of claim 24 , wherein the processing unit is further configured to determine the certain movement by determining an event comprising at least one of:
walking, running, or lying down,
a turning of the user's head,
the user standing up,
the user sitting down, or
the user falling down.
26. The head-mounted display device of claim 22 , wherein the at least one sensor comprises at least one of:
a motion sensor,
an inertial sensor,
a light sensor,
a microphone, or
an antenna.
27. The head-mounted display device of claim 22 , wherein the processing unit is configured to cause the display to change the visibility of the video by increasing a transparency of the video.
28. The head-mounted display device of claim 22 , wherein the processing unit is configured to cause the display to change the visibility of the video by reducing a size of the video.
29. The head-mounted display device of claim 22 , wherein the processing unit is further configured to pause playback of the video.
30. The head-mounted display device of claim 22 , wherein the processing unit is further configured to:
detect the triggering event by detecting a movement; and
change the visibility of the video in proportion to at least one of a speed or an amount of the detected movement.
31. The head-mounted display device of claim 22 , wherein the processing unit is further configured to:
cause audio related to the video to be played back; and
reduce a volume of the audio based on the triggering event.
32. A non-transitory computer-readable medium having instructions embedded thereon for providing a reactive video playback interface for a head-mounted display, the instructions including computer-executable code for:
causing a display of the head-mounted display to show at least one visual element including a video, wherein the display is at least semi-transparent from a view of a user of the head-mounted display when no content is displayed;
detecting a triggering event using a set of sensor data from one or more sensors;
changing a visibility of the video based on the triggering event while the video continues to be shown; and
causing the display to show at least one new visual element based on the triggering event while the video continues to be shown, the at least one new visual element indicative of an environmental condition.