Periocular test for mixed reality calibration
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, and/or whether an adjustment to a rendering location of a virtual object can be made to compensate for a deviation of the wearable device from its normal resting position relative to the user's face.
1 . A head-mounted device (HMD) comprising:
one or more eye cameras arranged to image at least a portion of a head of a user wearing the HMD; and
a hardware processor programmed to:
receive a set of sequentially captured images captured by the one or more eye cameras, the set of sequentially captured images comprising a first image and a second image;
determine whether at least one periocular feature of a user of the HMD is in at least one of the set of sequentially captured images;
in response to a determination that the at least one periocular feature of the user is in the first image and not in the second image, provide an indication that the user has taken off the HMD;
in response to a determination that the at least one periocular feature of the user is in the second image and not in the first image, determine that the user has put on the HMD;
dynamically lower resolution of the one or more eye cameras responsive to determining that the user has put on the HMD;
analyze the set of sequentially captured images with lowered resolution to determine a measurement of a relative position between the HMD and the head of the user;
access the measurement of the relative position between the HMD and the head of the user; and
calculate an adjustment to a normal rendering position of a virtual object based at least partly on the relative position between the HMD and the head of the user.
2 . The HMD of claim 1 , wherein the hardware processor is programmed to:
determine a rendering location of the virtual object based on the adjustment to the normal rendering position; and
render the virtual object at the rendering location.
3 . The HMD of claim 1 , wherein the measurement of the relative position between the HMD and the head of the user is calculated by performing at least one of: tracking the at least one periocular feature using visual keypoints, or matching a region of the face with a dense map encoding at least a portion of the head of the user.
4 . The HMD of claim 3 , wherein the visual keypoints are computed using at least one of: scale-invariant feature transform, speeded up robust features, oriented FAST and rotated BRIEF, binary robust invariant scalable keypoints, or fast retina keypoint; or wherein the dense map is calculated using iterative closest point algorithm.
5 . The HMD of claim 1 , wherein the hardware processor is further programmed to determine a normal resting position of the user's eye associated with a rendering viewpoint of the HMD, wherein the normal rendering position of the virtual object corresponds to the normal resting position of the user's eye.
6 . The HMD of claim 5 , wherein to calculate the adjustment to the normal rendering position of the virtual object, the hardware processor is programmed to:
determine a shift with respect to the normal resting position based at least partly on the relative position between the HMD and the head of the user; and
shift a coordinate associated with the rendering viewpoint of the HMD based at least partly on the shift with respect to the normal resting position.
7 . The HMD of claim 5 , wherein the relative position between the HMD and the head of the user comprises one or more of: a horizontal shift, a vertical shift, a depth shift, a tilt to a side, or a forward tilt with respect to the normal resting position.
8 . The HMD of claim 1 , wherein the relative position between the HMD and the head of the user comprises a first relative position between the HMD for a first eye of the user and a second relative position for a second eye of the user.
9 . A head-mounted device (HMD) comprising:
one or more eye cameras arranged to image at least a portion of a head of a user wearing the HMD; and
a hardware processor programmed to:
receive a set of sequentially captured images captured by the one or more eye cameras;
dynamically lower resolution of the one or more eye cameras responsive to determining that the user has put on the HMD;
analyze the set of sequentially captured images with lowered resolution to determine a measurement of a relative position between the HMD and the head of the user;
determine the relative position between the HMD and the head of the user based on an appearance of a periocular region of the user in at least of one of the set of sequentially captured images;
calculate an adjustment to a normal rendering position of a virtual object based at least partly on the relative position between the HMD and the head of the user;
determine a rendering location of the virtual object based on the adjustment to the normal rendering position; and
instruct the HMD to render the virtual object at the rendering location.
10 . The HMD of claim 9 , wherein the relative position between the HMD and the head of the user is determined by performing at least one of: tracking the periocular region using visual keypoints, or matching a region of the face with a dense map encoding at least a portion of the head of the user.
11 . The HMD of claim 10 , wherein the visual keypoints are computed using at least one of: scale-invariant feature transform, speeded up robust features, oriented FAST and rotated BRIEF, binary robust invariant scalable keypoints, or fast retina keypoint; or wherein the dense map is calculated using iterative closest point algorithm.
12 . The HMD of claim 9 , wherein the hardware processor is further programmed to determine a normal resting position of the user's eye associated with a rendering viewpoint of the HMD, wherein the normal rendering position of the virtual object corresponds to the normal resting position of the user's eye.
13 . The HMD of claim 12 , wherein to calculate the adjustment to the normal rendering position of the virtual object, the hardware processor is programmed to:
determine a shift with respect to the normal resting position based at least partly on the relative position between the HMD and the head of the user; and
shift a coordinate associated with the rendering viewpoint of the HMD based at least partly on the shift with respect to the normal resting position.
14 . The HMD of claim 12 , the relative position between the HMD and the head of the user comprises one or more of: a horizontal shift, a vertical shift, a depth shift, a tilt to a side, or a forward tilt with respect to the normal resting position.
15 . The HMD of claim 9 , wherein the relative position between the HMD and the head of the user comprises a first relative position between the HMD for a first eye of the user and a second relative position for a second eye of the user.