Apparatus, system, and method for accelerating positional tracking of head-mounted displays

1. A special-purpose hardware device comprising:

a camera interface that receives, from a camera device of a head-mounted-display system, image frames of a physical environment surrounding the head-mounted-display system;

a positional tracking component communicatively coupled to the camera interface, wherein the positional tracking component:

processes at least a portion of the image frames in a hardware-accelerator that:

detects one or more features of the physical environment within an incoming image frame as the incoming image frame is received; and

passes the features to a main memory of the special-purpose hardware device to be incorporated into an image map that identifies physical locations of the features of the physical environment;

tracks, based at least in part on the image map, a change in a position of the head-mounted-display system within the physical environment; and

stores the change in the position of the head-mounted-display system for use in generating at least one augmented-reality frame to be presented to a user wearing the head-mounted-display system; and

a frame-output interface communicatively coupled to the positional tracking component, wherein the frame-output interface feeds the augmented-reality frame to a display device of the head-mounted-display system for presentation to the user.

2. The special-purpose hardware device of claim 1 , wherein the features of the physical environment represent at least portions of physical objects located within the physical environment.

3. The special-purpose hardware device of claim 2 , wherein the positional tracking component identifies a set of key features within the physical environment to be used in tracking changes in the position of the head-mounted-display system.

4. The special-purpose hardware device of claim 3 , wherein the positional tracking component tracks the change in the position of the head-mounted-display system by:

identifying an initial location of a representation of at least one key feature of the physical environment within at least one of:

an initial image frame captured by the camera device; or

the image map;

identifying a subsequent location of a representation of the key feature of the physical environment within a subsequent image frame captured by the camera device; and

comparing the initial location of the representation of the key feature with the subsequent location of the representation of the key feature.

5. The special-purpose hardware device of claim 4 , wherein the positional tracking component identifies the subsequent location of the representation of the key feature within the subsequent image frame by:

determining an expected region of the subsequent image frame in which the key feature of the physical environment is likely to be represented based on at least one of:

the initial location of the representation of the key feature within the initial image frame; or

the image map; and

identifying a specific location of the representation of the key feature within the subsequent image frame by searching the expected region of the subsequent image frame.

6. The special-purpose hardware device of claim 5 , wherein the positional tracking component:

generates one or more image patches that represent physical appearances of the key feature of the physical environment from distinct perspectives; and

stores the image patches within the main memory of the special-purpose hardware device.

7. The special-purpose hardware device of claim 6 , wherein the positional tracking component comprises an image searching unit that identifies the specific location of the representation of the key feature within the subsequent image frame by:

warping at least one image patch such that the image patch resembles a current physical appearance of the key feature from a perspective at which the camera device captured the subsequent image frame; and

identifying a location within the subsequent image frame at which pixel values of the warped image patch most closely match corresponding pixel values of the subsequent image frame.

8. The special-purpose hardware device of claim 7 , wherein the image searching unit identifies the specific location of the representation of the key feature within the subsequent image frame by:

performing a sub-pixel interpolation analysis of pixel values within the expected region of the subsequent image frame; and

determining, based on the sub-pixel interpolation analysis, a sub-pixel location at which the pixel values of the warped image patch most closely match the corresponding pixel values of the subsequent image frame.

9. The special-purpose hardware device of claim 8 , wherein the image searching unit:

determines that an additional key feature of the physical environment is expected to be represented within the expected region of the subsequent image frame;

loads the expected region of the subsequent image frame into a cache of the special-purpose hardware device; and

identifies both the specific location of the representation of the key feature and a specific location of a representation of the additional key feature within the subsequent image frame by searching the expected region of the subsequent image frame within the cache of the special-purpose hardware device.

10. The special-purpose hardware device of claim 5 , wherein the hardware-accelerator of the special-purpose hardware device comprises a hardware-accelerated searching component that is dedicated to both:

identifying the specific location of the representation of the key feature within the subsequent image frame; and

identifying a physical location of the key feature within the physical environment.

11. The special-purpose hardware device of claim 10 , wherein the hardware-accelerated searching component identifies the physical location of the key feature within the physical environment by:

identifying specific locations of representations of the key feature within previous image frames captured by the camera device; and

performing an epipolar search for a three-dimensional location of the key feature within the physical environment based on the specific locations of the representations of the key feature within the previous image frames.

12. The special-purpose hardware device of claim 1 , wherein the hardware-accelerator of the special-purpose hardware device comprises a hardware-accelerated image processing component that:

creates at least one low-resolution version of the incoming image frame as the incoming image frame is received at the hardware-accelerated image processing component; and

passes the low-resolution version of the incoming image frame to the main memory of the special-purpose hardware device to facilitate tracking the change in the position of the head-mounted-display system.

13. The special-purpose hardware device of claim 1 , further comprising a frame-generation unit that:

retrieves the change in the position of the head-mounted-display system; and

while generating the augmented-reality frame, adjusts at least one aspect of the augmented-reality frame to account for the change in the position of the head-mounted-display system.

14. The special-purpose hardware device of claim 1 , wherein the positional tracking component processes the portion of the image frames in the hardware-accelerator of the special-purpose hardware device as the image frames are received from the camera interface and before the image frames are stored in a main memory of the special-purpose hardware device.

15. The special-purpose hardware device of claim 1 , wherein the positional tracking component further:

stores at least a portion of the image frames directly in the main memory of the special-purpose hardware device before the image frames are processed by the hardware-accelerator of the special-purpose hardware device; and

tracks the change in the position of the head-mounted-display system based at least in part on the portion of the image frames stored directly in the main memory of the special-purpose hardware device.

16. The special-purpose hardware device of claim 1 , further comprising an image signal processor that:

receives at least a portion of the image frames from the camera interface;

performs at least one image processing operation on the received image frames; and

forwards the processed image frames to the positional tracking component.

17. A head-mounted-display system comprising:

at least one camera device that captures image frames of a physical environment surrounding the head-mounted-display system;

a special-purpose hardware device communicatively coupled to the camera device, wherein the special-purpose hardware device:

processes at least a portion of the image frames in a hardware-accelerator that:

detects one or more features of the physical environment within an incoming image frame as the incoming image frame is received; and

passes the features to a main memory of the special-purpose hardware device to be incorporated into an image map that identifies physical locations of the features of the physical environment;

tracks, based at least in part on the image map, a change in a position of the head-mounted-display system within the physical environment; and

stores the change in the position of the head-mounted-display system for use in generating at least one augmented-reality frame to be presented to a user wearing the head-mounted-display system; and

a display device communicatively coupled to the special-purpose hardware device, wherein the display device:

receives the augmented-reality frame from the special-purpose hardware device; and

presents the augmented-reality frame to the user wearing the head-mounted-display system.

18. The head-mounted-display system of claim 17 , further comprising at least one communication interface communicatively coupled to an independent host device that feeds virtual image content from the independent host device to the head-mounted-display system.

19. A method comprising:

receiving, by a special-purpose hardware device from a camera device of a head-mounted-display system, image frames of a physical environment surrounding the head-mounted-display system;

processing, by the special-purpose hardware device, at least a portion of the image frames in a hardware-accelerator that:

detects one or more features of the physical environment within an incoming image frame as the incoming image frame is received; and

passes the features to a main memory of the special-purpose hardware device to be incorporated into an image map that identifies physical locations of the features of the physical environment;

tracking, by the special-purpose hardware device based at least in part on the image map, a change in a position of the head-mounted-display system within the physical environment;

storing, by the special-purpose hardware device, the change in the position of the head-mounted-display system for use in generating at least one augmented-reality frame to be presented to a user wearing the head-mounted-display system; and

feeding, by the special-purpose hardware device, the augmented-reality frame to a display device of the head-mounted-display system such that the display device presents the augmented-reality frame to a user wearing the head-mounted-display system.