Pose augmentation of head worn display video for latency reduction
A system comprising a head worn display and an image rendering computing device is disclosed. The head worn display comprises one or more image sensors, a head position sensor, and a head position computing device configured to determine a first head pose. The image rendering computing device is configured to render an enhanced image and embed the first head pose in the enhanced image. The head position computing platform is further configured to determine a second head pose, determine a difference between the second head pose and the first head pose, and warp the enhanced image based on the difference between the second head pose and the first head pose to correct for head motion of the user.
1. A system configured for low-latency split-rendering with embedded pose data comprising:
a) a head worn display communicatively coupled to an image rendering computing device, the head worn display comprising:
one or more image sensors configured to capture an image of an environment surrounding a user of the head worn display;
a head position sensor configured to measure the user's head position; and
a head position computing device configured to determine a first head pose based on head position measurements generated by the head position sensor at a first time; and
b) the image rendering computing device configured to:
render an enhanced image based on the image of the environment surrounding the user of the head worn display; and
embed the first head pose in the enhanced image;
wherein the head position computing device of the head worn display is further configured to:
determine a second head pose based on second head position measurements generated by the head position sensor at a second time which is different than the first time, wherein the second head pose is a projected future head pose;
determine a difference between the second head pose and the first head pose wherein the second head pose is a projected future head pose;
warp the enhanced image based on the difference between the second head pose and the first head pose wherein the second head pose is a projected future head pose, to correct for head motion of the user, wherein warping the enhanced image comprises:
constructing a planar homography based on the difference between the second head pose and the first head pose; and
present the enhanced image to the user of the head worn display,
wherein the system is at least one of an augmented reality (AR) display system, a virtual reality (VR) display system, or a mixed reality (MR) display system.
2. The system of claim 1 , wherein the head worn display is at least one of a transparent display, a semi-transparent display, or an opaque display.
3. The system of claim 1 , wherein the head position computing device comprises a field programmable gate array (FPGA), the FPGA configured to determine the first head pose.
4. The system of claim 1 , wherein the head position computing device comprises a field programmable gate array (FPGA), the FPGA configured to perform the warp of the enhanced image.