Peripheral device for head-mounted display
A head-mounted display (HMD) system includes an HMD device wearable upon a head of a user, and a peripheral device that is dockable with the HMD device via a pair of electronic connectors. The HMD device and the peripheral device may communicate with each other via wireless communications in a docked configuration, and may communicate with each other via wired communications over the pair of electronic connectors in the docked configuration. The HMD system supports a variety of different modes of operation that may be varied responsive to whether the peripheral device is in the docked configuration or the undocked configuration with the HMD device.
1. A head-mounted display (HMD) system, comprising:
a HMD device configured to display augmented reality (AR) content; and
a peripheral device dockable to the HMD device, the peripheral device comprising:
a user input interface configured to receive user input while in an undocked configuration of the peripheral device, the user input identifying a portion of the AR content, and
a peripheral electronic control system programmed with peripheral instructions executable by the peripheral electronic control system configured to transmit a wireless communication to the HMD device from the peripheral device based on the peripheral device being in the undocked configuration, the wireless communication indicating the user input that identifies the portion of the AR content, wherein the AR content visually appears as a plurality of AR objects radially distributed along a perimeter of the face of the user input interface and forming a plurality of regions, each region providing a different corresponding visual indication of where to select and activate a different corresponding function implemented by the HMD device.
2. The HMD system of claim 1 , wherein the HMD device comprises:
an HMD electronic control system,
a see-through graphical display operatively coupled to the HMD electronic control system to present graphical content,
a first electronic connector operatively coupled to the HMD electronic control system, and
a first wireless interface operatively coupled to the HMD electronic control system to at least receive wireless communications,
wherein the peripheral device further comprises:
a second electronic connector operatively coupled to the peripheral electronic control system to establish a set of physical electronic connections with the first electronic connector of the HMD device in a docked configuration,
a second wireless interface operatively coupled to the peripheral electronic control system to at least transmit the wireless communication;
wherein the user input interface is operatively coupled to the peripheral electronic control system,
wherein the peripheral electronic control system is configured to transmit the wireless communication via the second wireless interface indicating the user input received via the user input interface for reception by the HMD device via the first wireless interface in the undocked configuration in which the set of physical electronic connections are not established between the first electronic connector and the second electronic connector.
3. The HMD system of claim 2 , wherein the HMD electronic control system is programmed with HMD instructions executable by the HMD electronic control system to:
receive, via first wireless interface, the wireless communication transmitted by the peripheral device;
store the user input indicated by the wireless communication received from the peripheral device; and
vary operation of the HMD device responsive to the user input.
4. The HMD system of claim 2 , wherein the HMD electronic control system is programmed with HMD instructions executable by the HMD electronic control system to:
monitor a connection state of the set of physical electronic connections at the first and second electronic connector; and
determine whether the peripheral device is in the docked configuration or the undocked configuration based on the connection state of the set of physical electronic connections.
5. The HMD system of claim 4 , wherein the HMD instructions are further executable by the HMD electronic control system to:
activate a dwell mode in the docked configuration, the dwell mode directing the HMD device to identify a sighting axis of a reticle of the HMD device or a gaze vector of an eye of the user wearing the HMD device dwelling upon an AR object, presented by the HMD device via the see-through graphical display, for at least a threshold period of time as indicating a user input directed at the AR object; and
deactivate the dwell mode in an undocked configuration in which the HMD does not identify the sighting axis of the reticle or the gaze vector dwelling upon the AR object for at least the threshold period of time as indicating the user input directed at the AR object.
6. The HMD system of claim 4 , wherein the HMD instructions are further executable by the HMD electronic control system to:
select a current user state for the user wearing the HMD device from among a plurality of predefined user states, the current user state being selected responsive to user input or programmatically selected based on an operating condition detected by the HMD device;
activate an optical signaling protocol for signaling the user state in the docked configuration, the optical signaling protocol directing the HMD device and the peripheral device to output light from one or more light source elements visible along an exterior of the HMD device and/or peripheral device according to an optically detectable pattern defined by the optical signaling protocol for the current user state; and
deactivate the optical signaling protocol for signaling the user state in the undocked configuration.
7. The HMD system of claim 4 , wherein the peripheral device includes one or more sensors; and
wherein the peripheral instructions are further executable by the peripheral electronic control system to:
in the undocked configuration, store measurement data obtained via the one or more sensors of the peripheral device in a data storage device of the peripheral device; and
otherwise, in the docked configuration, transmit the measurement data stored in the data storage device to the HMD device via the second electronic connector of the peripheral device over at least some of the set of physical electronic connections.
8. The HMD system of claim 7 , wherein the peripheral device includes one or more sensors; and
wherein the peripheral instructions are further executable by the peripheral electronic control system to:
transmit a further wireless communication including the measurement data via the second wireless interface for reception by the HMD device in the undocked configuration.
9. The HMD system of claim 8 , wherein the one or more sensors includes a camera; and
wherein the measurement data includes one or more images captured by the camera.
10. The HMD system of claim 9 , wherein the HMD instructions are further executable by the HMD electronic control system to:
receive, via the first wireless interface, the further wireless communication including the measurement data transmitted by the peripheral device; and
present the one or more images captured by the camera of the peripheral device via the see-through graphical display of the HMD device.
11. The HMD system of claim 4 , wherein the peripheral device includes an electric battery; and
wherein the peripheral instructions are further executable by the peripheral electronic control system to:
in the docked configuration, activate charging of the electric battery with electrical power received from the HMD device via the second electronic connector over at least some of the set of physical electronic connections.
12. The HMD system of claim 1 , wherein the user input interface includes a touch-sensitive user interface, and wherein the peripheral electronic control system is configured to detect the user touching the user input interface within one of the plurality of regions that appears to be overlaid by one of the plurality of AR objects to initiate selection of the corresponding function of a region among the plurality of regions, the region corresponding to the overlaid AR object.
13. The HMD system of claim 1 , wherein the user input interface includes a plurality of physical buttons corresponding to the plurality of regions, wherein the peripheral electronic control system is configured to detect the user depressing one of the physical buttons that is overlaid by one of the plurality of AR objects to initiate selection of the corresponding function of a region among the plurality of regions, the region corresponding to the overlaid AR object.
14. The HMD system of claim 1 , wherein the AR content visually appears as an AR window that extends outwards from the peripheral device, the AR window being aligned with an AR object selected by the user from the plurality of AR objects, the AR window including information related to the selected AR object.
15. The HMD system of claim 14 , wherein the HMD is configured to modify the plurality of AR objects based on a type of information being displayed in the AR window, the modified plurality of AR objects being configured to provide functions corresponding to the type of information being displayed in the AR window.
16. The HMD system of claim 1 , wherein the AR content includes a three-dimensional model of a physical object, a motion of the three-dimensional model being manipulated based on the user input interface.
17. The HMD system of claim 1 , wherein the peripheral electronic control system is configured to access movement data from an Inertial Measurement Unit (IMU) sensor in the peripheral device and to transmit the movement data to a control system of the physical object associated with the three-dimensional model.
18. A method performed by a head-mounted display (HMD) device wearable upon a head of a user, the method comprising:
presenting augmented reality (AR) content in a display of the HMD;
detecting a user input at a user input interface of a peripheral device in an undocked configuration, the peripheral device dockable to the HMD device; and
receiving wireless communications at the HMD device from the peripheral device based on the peripheral device being in the undocked configuration, the wireless communications indicating the user input and identifies a portion of the augmented reality content, wherein the AR content visually appears as a plurality of AR objects radially distributed along a perimeter of the face of the user input interface and forming a plurality of regions, each region providing a different corresponding visual indication of where to select and activate a different corresponding function implemented by the HMD device.
19. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
presenting augmented reality (AR) content in a display of the HMD;
detecting a user input at a user input interface of a peripheral device in an undocked configuration, the peripheral device dockable to the HMD device; and
receiving wireless communications at the HMD device from the peripheral device based on the peripheral device being in the undocked configuration, the wireless communications indicating the user input and identifies a portion of the augmented reality content, wherein the AR content visually appears as a plurality of AR objects radially distributed along a perimeter of the face of the user input interface and forming a plurality of regions, each region providing a different corresponding visual indication of where to select and activate a different corresponding function implemented by the HMD device.