Periocular test for mixed reality calibration
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.
1. A head-mounted device (HMD) comprising:
a hardware processor programmed to:
receive a set of sequentially captured images comprising a first image and a second image;
determine whether at least one periocular feature of a user of the HMD is in at least one of the set of sequentially captured images; and
in response to a determination that at least one periocular feature of the user is in the first image and not in the second image:
determine, based on the at least one periocular feature of the user, a likelihood that the HMD has been removed from the head of the user; and
in response to determining that the likelihood that the HMD has been removed from the head of the user exceeds a threshold likelihood, provide an indication that the HMD has been removed from the head of the user; or
in response to determining that the likelihood that the HMD has been removed from the head of the user does not exceed the threshold likelihood, provide an indication that the HMD is not properly positioned on the head of the user.
2. The HMD of claim 1 , wherein the hardware processor is programmed to:
access a measurement of a relative position between the HMD and a head of the user; and
calculate an adjustment to a normal rendering position of a virtual object based at least partly on the relative position between the HMD and the head of the user.
3. The HMD of claim 2 , wherein the hardware processor is programmed to:
determine a rendering location of the virtual object based on the adjustment to the normal rendering position; and
render the virtual object at the rendering location.
4. The HMD of claim 2 , wherein the measurement of the relative position between the HMD and the head of the user is calculated by performing at least one of:
tracking the at least one periocular feature using visual keypoints, or matching a region of the face with a dense map encoding at least a portion of the head of the user.
5. The HMD of claim 4 , wherein the visual keypoints are computed using at least one of: scale-invariant feature transform, speeded up robust features, oriented FAST and rotated BRIEF, binary robust invariant scalable keypoints, or fast retina keypoint; or wherein the dense map is calculated using iterative closest point algorithm.
6. The HMD of claim 2 , wherein the hardware processor is further programmed to determine a normal resting position of the user's eye associated with a rendering viewpoint of the HMD, and wherein the normal rendering position of the virtual object corresponds to the normal resting position of the user's eye.
7. The HMD of claim 6 , wherein to calculate the adjustment to the normal rendering position of the virtual object, the hardware processor is programmed to:
determine a shift with respect to the normal resting position of the user's eye based at least partly on the relative position between the HMD and the head of the user; and
shift a coordinate associated with the rendering viewpoint of the HMD based at least partly on the determined shift with respect to the normal resting position.
8. The HMD of claim 6 , wherein the relative position between the HMD and the head of the user comprises one or more of: a horizontal shift, a vertical shift, a depth shift, a tilt to a side, or a forward tilt with respect to the normal resting position.
9. The HMD of claim 2 , wherein the relative position between the HMD and the head of the user comprises a first relative position between the HMD and the head of the user for a first eye of the user and a second relative position between the HMD and the head of the user for a second eye of the user.
10. The HMD of claim 1 , wherein the hardware processor is programmed to:
in response to a determination that at least one periocular feature of the user is in the second image and not in the first image, provide an indication that the HMD is properly positioned on the head of the user.