Head-mounted device with active optical foveation
A pass-through camera in a head-mounted device may capture image data for displaying on a display in the head-mounted device. However, only low-resolution image data may be needed to display low-resolution images in the periphery of the user's field of view on the display. Therefore, the pass-through camera may only capture high-resolution images that correspond to the portion of the user's field-of-view that is being directly viewed and may capture lower resolution image data that corresponds to the real-world objects in the user's peripheral vision. To enable the camera module to selectively capture high-resolution images, the pass-through camera may include an image sensor with two or more pixel densities, a distortion lens, and/or one or more planar or curved mirrors. Any of the components in the camera module may be adjusted to change which portion of a scene is captured with high-resolution image data.
1. A head-mounted device, comprising:
a head-mounted support structure;
a pass-through camera that is supported by the head-mounted support structure and that is configured to capture an image of a real-world scene, wherein the image has a first-resolution portion and a second-resolution portion that has a higher resolution than the first-resolution portion;
a display that is supported by the head-mounted support structure and that is configured to display the image;
a gaze-tracking system that is configured to obtain point of gaze information; and
control circuitry that is configured to adjust a lens in the pass-through camera based on the point of gaze information, wherein the lens outputs light with varying angular resolution and wherein adjusting the lens adjusts which portion of the real-world scene corresponds to the second-resolution portion of the image.
2. The head-mounted device defined in claim 1 , wherein the pass-through camera comprises an image sensor that is configured to capture the image.
3. The head-mounted device defined in claim 2 , wherein the lens is configured spread light incident in a first portion of the lens across a greater corresponding area of the image sensor than light incident in a second portion of the lens.
4. The head-mounted device defined in claim 2 , wherein the lens is configured spread light incident in a center of the lens across a greater corresponding area of the image sensor than light incident in a periphery of the lens.
5. The head-mounted device defined in claim 1 , wherein the lens increases an angular resolution of a portion of incident light from the real-world scene that corresponds to the second-resolution portion of the image.
6. A head-mounted device, comprising:
a head-mounted support structure;
a display that is supported by the head-mounted support structure and that is configured to display an image;
a gaze-tracking system that is configured to obtain point of gaze information; and
a camera module comprising:
an image sensor that is configured to capture the image based on incident light from real-world objects; and
a lens that magnifies some but not all of the incident light onto the image sensor, wherein the lens is configured to be adjusted based on the point of gaze information.
7. The head-mounted device defined in claim 6 , wherein light in a first area of the lens is spread onto a second area of the image sensor and wherein the second area is greater than the first area.
8. The head-mounted device defined in claim 6 , wherein the image has a first-resolution portion and a second-resolution portion having a higher resolution than the first-resolution portion and wherein the incident light magnified by the lens corresponds to the second-resolution portion of the image.
9. A head-mounted device, comprising:
a head-mounted support structure;
an outward-facing camera that is supported by the head-mounted support structure and that is configured to use an image sensor to capture an image of a real-world scene, wherein the image has a first-resolution portion and a second-resolution portion that has a higher resolution than the first-resolution portion;
a gaze-tracking system that is supported by the head-mounted support structure and that includes an inward-facing camera, wherein the gaze-tracking system is configured to obtain point of gaze information based on images captured by the inward-facing camera;
a display that is supported by the head-mounted support structure and that is configured to display the image of the real-world scene; and
control circuitry that is configured to adjust which portion of the real-world scene corresponds to the second-resolution portion of the image based on the point of gaze information.
10. The head-mounted device defined in claim 9 , wherein the control circuitry is configured to physically move a component in the outward-facing camera to adjust which portion of the real-world scene corresponds to the second-resolution portion of the image.
11. The head-mounted device defined in claim 10 , wherein the component comprises a component selected from the group consisting of: an image sensor, a mirror, a lens, and a camera module housing.
12. The head-mounted device defined in claim 10 , wherein the component is a distortion lens that magnifies incident light from a first portion of the real-world scene that corresponds to the first-resolution portion of the image less than incident light from a second portion of the real-world scene that corresponds to the second-resolution portion of the image.