Systems and methods for a peripheral-centric augmented/virtual reality environment
In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.
1. A method of operating a peripheral device, the method comprising:
receiving polling data from one or more sensors, the polling data corresponding to physical characteristics of a physical environment around the peripheral device;
determining an area to orient a projected virtual display relative to the peripheral device within the physical environment based on the physical characteristics;
determining a second area to orient a virtual interactive display relative to the peripheral device, wherein the interactive display is configured to facilitate an augmentation of functional capabilities of the peripheral device;
determining a spatial relationship between the peripheral device and the projected virtual display; and
generating control data configured to cause an AR/VR-based head-mounted display (HMD) to project the virtual display in the determined area and the determined second area at a maintained spatial relationship between the peripheral device and the projected virtual display as the peripheral device is moved within the physical environment.
2. The method of claim 1 further comprising:
detecting that the peripheral device is placed on a surface or interfaced by a user,
wherein receiving the polling data from the one or more sensors occurs in response to detecting that the peripheral device is placed on the surface or interfaced by the user.
3. The method of claim 1 wherein the control data causes the spatial relationship between the peripheral device and the virtual display to be maintained such that:
a movement of the peripheral device that is within a threshold distance from an initial location of the peripheral device does not cause the virtual display to move; and
a movement of the peripheral device that is greater than the threshold distance from the initial location of the peripheral device causes the spatial relationship between the peripheral device and the projected interactive display to be fixed,
the spatial relationship between the peripheral device and the projected interactive display is fixed as the peripheral device is moved in the physical environment.
4. The method of claim 1 further comprising:
determining a third area on the peripheral device to orient a virtual overlay, wherein the virtual overlay is configured to further facilitate the augmentation of the functional capabilities of the peripheral device; and
determining a spatial relationship between the peripheral device and the projected virtual overlay,
wherein the control data is further configured to cause the HIVID to project the virtual overlay in the determined third area and at a maintained spatial relationship between the peripheral device and the projected interactive display as the peripheral device is moved in the physical environment.
5. A system for operating a virtual workstation, the system comprising:
one or more processors;
one or more non-transitory computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including:
receiving polling data from one or more sensors, the polling data corresponding to physical characteristics of a physical environment around a peripheral device;
determining an area to orient a projected virtual display relative to the peripheral device within the physical environment based on the physical characteristics;
determining a second area to orient a virtual interactive display relative to the peripheral device, wherein the virtual interactive display is configured to facilitate an augmentation of functional capabilities of the peripheral device;
determining a spatial relationship between the peripheral device and the projected virtual display; and
generating control data configured to cause an AR/VR-based head-mounted display (HMD) to project the virtual display in the determined area and the determined second area at a maintained spatial relationship between the peripheral device and the projected virtual display as the peripheral device is moved within the physical environment.
6. The system of claim 5 wherein the one or more non-transitory computer-readable storage mediums further contain instructions comprising:
detecting that the peripheral device is placed on a surface or interfaced by a user,
wherein receiving the polling data from the one or more sensors occurs in response to detecting that the peripheral device is placed on the surface or interfaced by the user.
7. The system of claim 5 wherein the control data causes the spatial relationship between the peripheral device and the virtual display to be maintained such that:
a movement of the peripheral device that is within a threshold distance from an initial location of the peripheral device does not cause the virtual display to move; and
a movement of the peripheral device that is greater than the threshold distance from the initial location of the peripheral device causes the spatial relationship between the peripheral device and the projected interactive display to be fixed,
the spatial relationship between the peripheral device and the projected interactive display is fixed as the peripheral device is moved in the physical environment.
8. The system of claim 5 wherein the one or more non-transitory computer-readable storage mediums further contain instructions comprising:
determining a third area on the peripheral device to orient a virtual overlay, wherein the virtual overlay is configured to further facilitate the augmentation of the functional capabilities of the peripheral device; and
determining a spatial relationship between the peripheral device and the projected virtual overlay,
wherein the control data is further configured to cause the HMD to project the virtual overlay in the determined third area and at a maintained spatial relationship between the peripheral device and the projected interactive display as the peripheral device is moved in the physical environment.