Periocular test for mixed reality calibration
A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.
1. A head-mounted device (HMD) comprising:
a hardware processor programmed to:
receive a set of sequentially captured images comprising a first image and a second image;
determine whether at least one periocular feature of a user of the HMD is in at least one of the set of sequentially captured images;
in response to a determination that at least one periocular feature of the user is in the first image and not in the second image, providing an indication that the HIMD is not properly positioned on the head of the user; and
in response to a determination that at least one periocular feature of the user is in the second image and not in the first image, providing an indication that the HIMD is properly positioned on the head of the user.
2. The HMD of claim 1 , wherein the hardware processor is programmed to:
access a measurement of a relative position between the HMD and a head of the user; and
calculate an adjustment to a normal rendering position of a virtual object based at least partly on the relative position between the HMID and the head of the user.
3. The HMD of claim 2 , wherein the hardware processor is programmed to:
determine a rendering location of the virtual object based on the adjustment to the normal rendering position; and
render the virtual object at the rendering location.
4. The HMD of claim 2 , wherein the measurement of the relative position between the HMID and the head of the user is calculated by performing at least one of: tracking the periocular features using visual keypoints, or matching a region of the face with a dense map encoding at least a portion of the head of the user.
5. The HMD of claim 4 , wherein the visual keypoints are computed using at least one of: scale-invariant feature transform, speeded up robust features, oriented FAST and rotated BRIEF, binary robust invariant scalable keypoints, or fast retina keypoint; or wherein the dense map is calculated using iterative closest point algorithm.
6. The HMD of claim 2 , wherein the hardware processor is further programmed to determine a normal resting position of the user's eye associated with a rendering viewpoint of the HMD, wherein the normal rendering position of the virtual object which corresponds to the normal resting position of the user's eye.
7. The HIMD of claim 6 , wherein to calculate the adjustment to the normal rendering position of the virtual object, the hardware processor is programmed to:
determine a shift with respect to the normal resting position based at least partly on the relative position between the HMD and the head of the user; and
shift a coordinate associated with the rendering viewpoint of the HMD based at least partly on the shift with respect to the normal resting position.
8. The HMD of claim 2 , wherein the relative position between the HMD and the head of the user comprises one or more of: a horizontal shift, a vertical shift, a depth shift, a tilt to a side, or a forward tilt with respect to the normal resting position.
9. The HMD of claim 2 , wherein the relative position between the HMD and the head of the user comprises a first relative position between the HMD for a first eye of the user and a second relative position for a second eye of the user.
10. A head-mounted device (HMD) comprising:
a hardware processor programmed to:
calculate a measurement of a relative position between the HMD and a head of a user;
receive a set of sequentially captured images;
determine the relative position between the HMD and the head of the user based on an appearance of a periocular region of the user in at least one of the set of sequentially captured images;
calculate an adjustment to a normal rendering position of a virtual object based at least partly on the relative position between the HIMD and the head of the user;
determine a rendering location of the virtual object based on the adjustment to the normal rendering position; and
instruct the HMD to render the virtual object at the rendering location.
11. The HMD of claim 10 , wherein the relative position between the HMD and the head of the user is determined by performing at least one of: tracking the periocular features using visual keypoints, or matching a region of the face with a dense map encoding at least a portion of the head of the user.
12. The HMD of claim 11 , wherein the visual keypoints are computed using at least one of: scale-invariant feature transform, speeded up robust features, oriented FAST and rotated BRIEF, binary robust invariant scalable keypoints, or fast retina keypoint; or wherein the dense map is calculated using iterative closest point algorithm.
13. The HMD of claim 10 , wherein the hardware processor is further programmed to determine a normal resting position of the user's eye associated with a rendering viewpoint of the HMD, wherein the normal rendering position of the virtual object which corresponds to the normal resting position of the user's eye.
14. The HIMD of claim 13 , wherein to calculate the adjustment to the normal rendering position of the virtual object, the hardware processor is programmed to:
determine a shift with respect to the normal resting position based at least partly on the relative position between the HMD and the head of the user; and
shift a coordinate associated with the rendering viewpoint of the HMD based at least partly on the shift with respect to the normal resting position.
15. The HMD of claim 10 , the relative position between the HMD and the head of the user comprises one or more of: a horizontal shift, a vertical shift, a depth shift, a tilt to a side, or a forward tilt with respect to the normal resting position.
16. The HMD of claim 10 , wherein the relative position between the HMD and the head of the user comprises a first relative position between the HMD for a first eye of the user and a second relative position for a second eye of the user.