Camera based safety mechanisms for users of head mounted displays
Methods and systems for warning a user of a head mounted display (HMD) during game play of a video game. A game is executed causing interactive scenes of the game to be transmitted for rendering on a display portion of the HMD. Coordinates of the HMD are determined in a three-dimensional space of a real-world environment in which the user wearing the HMD is present, to identify a current position of the user. A movement of the HMD is determined during execution of the game by identifying a change in one or more coordinates of the HMD in the three-dimensional space. When it is determined that the user is approaching a boundary of an interaction space in a real-world environment, a warning signal is conveyed to the HMD to indicate proximity of the user to the boundary of the interaction space.
1. A method for warning a user of a head mounted display (HMD), the method comprising:
capturing images of a physical environment in which the user wearing the HMD is present, using one or more sensors disposed on the HMD;
processing the images to generate a three-dimensional map of the physical environment, the three-dimensional map identifying depth information of real-world objects identified in an area of the physical environment;
determining a location of the HMD by tracking movement of the user wearing the HMD in the physical environment as the user is interacting with content presented on a display portion of the HMD, wherein the tracking includes computing relative distance between the HMD worn by the user and one or more of the real-world objects in the physical environment using depth information captured in the three-dimensional map and from inputs obtained from inertial sensors disposed in the HMD; and
generating a warning to the user of the HMD upon detecting the user approaching an edge of the area of the physical environment, based on the tracking of the movement of the user,
wherein operations of the method are performed by a processor of the HMD.
2. The method of claim 1 , wherein the one or more sensors are automatically activated upon detecting the user wearing the HMD or at any time a change in data related to the HMD is detected during use of the HMD, or at any time a change is detected in the physical environment during use of the HMD by the user, the activation causing the one or more sensors to capture the images of the physical environment in which the user wearing the HMD is present.
3. The method of claim 1 , wherein tracking movement of the user is done for a time the user is wearing the HMD.
4. The method of claim 1 , wherein the one or more sensors include one or more of a depth sensor, or an image sensor, or a stereo three-dimensional camera, or an infrared camera, or an ultraviolet camera, or a video camera, or a radar device, or a laser radar device, or a depth camera, or an inertial sensor, or combinations of any two or more thereof.
5. The method of claim 1 , wherein the depth map is automatically calibrated to capture the depth of the one or more real-world objects disposed in the physical environment, the calibration based on amount of space available in the area of the physical environment captured in the images.
6. The method of claim 1 , wherein processing the images includes combining the images obtained from the one or more sensors and using the combined images to generate the three-dimensional map.
7. The method of claim 6 , wherein the images are processed to distinctly identify the real-world objects in the physical environment, wherein the real-world objects identified include attributes of a room, a building, a person, a pet, a vehicle, a scenery, or any combinations thereof.
8. The method of claim 1 , wherein tracking the movement of the user includes,
generating a virtual skeleton of the user within the area of the physical environment;
tracking coordinates of different body parts represented in the virtual skeleton of the user, of the HMD, and of the real-world objects disposed within the area; and
computing distance of the different body parts of the user, the HMD from the real-world objects disposed in the area and from the edge of the physical environment, using the corresponding coordinates.
9. The method of claim 1 , wherein the warning is provided as an image highlighting the edge of the area of the physical environment.
10. The method of claim 9 , wherein the image highlighting the edge of the area is blended with the content rendered on the display portion of the HMD.
11. The method of claim 1 , wherein the one or more sensors are distributed on a front face of the HMD and are facing outward.
12. The method of claim 1 , wherein the warning includes one of an audio message, or a visible message, or activating light emitting diodes of the HMD, or a gradual fading of content displayed on the HMD, or blending images from the physical world with the content, or vibrating of the HMD, or disengaging the HMD, or disengaging one or more parts of the HMD used to render the content, or moving the display portion of the HMD away from at least one eye of the user, or any combinations thereof.
13. A method for warning a user of a head mounted display (HMD), the method comprising:
generating a signal to activate one or more sensors disposed on the HMD, the activation causing the one or more sensors to capture images of a physical environment in which the user wearing the HMD is present;
tracking a location of the HMD within the physical environment, based at least in part on the images as the user moves around the physical environment, the location determined by generating a three-dimensional map of the physical environment that includes depth information of real-world objects present in the physical environment and computing a relative distance between the HMD worn by the user and one or more of the real-world objects; and
sending data for rendering on a display portion of the HMD, the data providing information regarding proximity of the user to the one or more real-world objects in the physical environment, based on the tracking of the movement of the user,
wherein operations of the method are performed by a processor of the HMD.
14. The method of claim 13 , wherein tracking the location of the HMD includes,
computing relative distance between the HMD worn by the user and the one or more of the real-world objects in the physical environment, using depth information captured in the three-dimensional map and from inputs obtained from inertial sensors disposed in the HMD.
15. The method of claim 13 , wherein the data includes descriptive context of a safety condition detected in the physical environment in the vicinity of the user wearing the HMD.
16. The method of claim 15 , wherein the data includes instructions to guide the user away from the one or more real-world objects in the physical environment.
17. The method of claim 15 , wherein the data includes a warning identifying a severity level of the safety condition in the vicinity of the user within the physical environment.
18. The method of claim 17 , wherein when the severity level is low, the warning is presented alongside any content rendered on the display portion of the HMD.
19. The method of claim 17 , wherein when the severity level is high, the data includes a signal to render the warning on the display portion of the HMD, the data including a signal to pause any content that is currently rendering on the display portion.