Method of rendering using a display device
One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.
1. A user display device, comprising:
a lens disposed adjacent an eye of a user, the lens configured to transition from a transparent setting to an opaque setting to block visible light from an outside environment from passing through the lens as part of entering a virtual reality mode from an augmented reality mode, wherein the lens is configured to transition from the transparent setting to the opaque setting, and wherein the lens includes at least one transparent mirror;
a first pair of cameras to track movement of the eyes of the user and estimate a depth of focus based at least in part on the movement of the eyes of the user;
a projection module having a light generating mechanism adapted to generate, based at least in part on the estimated depth of focus, a projected light associated with a display object; and
a processor configured to generate a digital rendering of at least a portion of an environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering is a rendered physical body part of the user as it appears in the environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering of the physical body part is generated in real time and in response to a determination that the physical body part is in a field-of-view of the user, and
wherein the light generating mechanism of the projection module modifies, based at least in part on the estimated depth of focus, the projected light associated with the display object such that the display object appears to be in focus.
2. The user display device of claim 1 , wherein the transparent mirror is positioned on the user display device so as to be in front of the user's eyes to bounce the projected light into the user's eyes and configured to selectively allow a transmission of light from the outside environment.
3. The user display device of claim 1 , further comprising a second pair of cameras to capture a field-of-view image of the eye of the user.
4. The user display device of claim 3 , wherein the processor is configured to calculate a head pose of the user based on the field-of-view of the user.
5. The user display device of claim 1 , wherein the first pair of cameras includes an inward infrared camera and an infrared light source to track movement of the eye.
6. The user display device of claim 1 , further comprising a sensor assembly comprising a sensor to sense a movement of the user, a location of the user, a direction of the user, or an orientation of the user.
7. The user display device of claim 6 , wherein the sensor is an accelerometer, a compass, or a gyroscope.
8. The user display device of claim 6 , wherein the processor is configured to estimate a head pose of the user based on the at least one of the movements of the user, the location of the user, the direction of the user, and the orientation of the user.
9. A user display device, comprising:
a lens disposed adjacent an eye of a user, the lens configured to transition from a transparent setting to an opaque setting to block visible light from an outside environment from passing through the lens as part of entering a virtual reality mode from an augmented reality mode, wherein the lens is configured to transition from the transparent setting to the opaque setting in response to a selection of the virtual reality mode corresponding to blocking light from the outside environment; and
a processor configured to generate a digital rendering of at least a portion of an environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering is a rendered physical body part of the user as it appears in the environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering of the physical body part is generated in real time and in response to a determination that the physical body part is in a field-of-view of the user, the processor generating the rendered physical body part using an image-based 3D reconstruction software to digitally reconstruct the physical body part as the digital rendering, and further comprising a GPS system, wherein the processor is configured to estimate a head pose of the user based on course and/or fast pose estimates provided by the GPS system.
10. The user display device of claim 1 , wherein the processor is communicatively coupled to a computer network to transmit at least a portion of a virtual world data, and receive another portion of the virtual world data.
11. The user display device of claim 1 , further comprising an audio speaker module to output sounds and a microphone to capture sounds local to the use, and wherein the processor is configured to render frames of the display object at a rate of at least 60 frames per second.
12. The user display device of claim 1 , further comprising a haptic interface device to provide tactile feedback.
13. The user display device of claim 1 , further comprising an environment sensing system to digitally reconstruct the environment of the user.
14. The user display device of claim 1 , wherein the lens is configured to transition from the transparent setting to the opaque setting by employing a cover to block the lens.
15. The user display device of claim 1 , wherein the lens is configured to transition from the transparent setting to the opaque setting based at least in part on a selected mode of the user display device.
16. The user display device of claim 15 , wherein the selected mode is a virtual reality mode, and wherein the lens is in the opaque setting.
17. A user display device, comprising:
a lens disposed adjacent an eye of a user, the lens configured to transition from a transparent setting to an opaque setting to block visible light from an outside environment from passing through the lens as part of entering a virtual reality mode from an augmented reality mode, wherein the lens is configured to transition from the transparent setting to the opaque setting;
an environment-sensing system comprising one or more sensors for obtaining data from the physical environment around a user, the one or more sensors including a generally outward-facing camera or a scanner for interpreting scene information through continuously and/or intermittently projected infrared structured light, the environment-sensing system further comprising a processor configured to generate a digital rendering of at least a portion of an environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering is a rendered physical body part of the user as it appears in the environment of the user when the user display device is in a virtual reality mode,
wherein the digital rendering of the physical body part is generated in real time and in response to a determination that the physical body part is in a field-of-view of the user, the processor generating the rendered physical body part using an image-based 3D reconstruction software to digitally reconstruct the physical body part as the digital rendering.
18. The user display device of claim 17 , wherein the environment-sensing system is adapted to provide one or more of:
motion capture data including gesture recognition;
depth sensing;
facial recognition;
object recognition;
unique object feature recognition;
voice/audio recognition and processing;
acoustic source localization;
noise reduction; and
infrared or similar laser projection,
and wherein the environment-sensing system comprises at least one optical-enhancing sensor.
19. The user display device of claim 17 , wherein the environment-sensing system comprises a microphone for receiving audio from the environment.
20. The user display device of claim 17 , wherein the environment-sensing system comprises at least one of: one or more infrared camera sensors; one or more visible spectrum camera sensors; structure light emitters and/or sensors; infrared light emitters; coherent light emitters and/or sensors; gyros; accelerometers; magnetometers; proximity sensors; GPS sensors; ultrasonic emitters and detectors; and, haptic interfaces.