GAZE-GUIDED IMAGE CAPTURE
A gaze direction of an eye or eyes of a user of a head mounted device is determined. One or more images is captured with a camera of a head mounted device. Gaze-guided images are generated from the one or more images based on the gaze direction of the user.
1 . A head mounted device comprising:
an eye-tracking system including one or more sensors configured to determine a gaze direction of an eye in an eyebox region of the head mounted device;
a first image sensor configured to capture first images of an external environment of the head mounted device;
a second image sensor configured to capture second images of the external environment of the head mounted device, wherein the first image sensor has a first field of view (FOV) that is different from a second FOV of the second image sensor; and
processing logic configured to:
receive the gaze direction from the eye-tracking system; and
select a selected image sensor between the first image sensor and the second image sensor to capture one or more gaze-guided images, wherein the first image sensor or the second image sensor is selected to capture the one or more gaze-guided images based on the gaze direction with respect to the first FOV and the second FOV.
2 . The head mounted device of claim 1 , wherein the selected image sensor is selected based in part on a gaze vector representative of the gaze direction being closest to a middle of a selected FOV of the selected image sensor.
3 . The head mounted device of claim 1 , wherein the processing logic is further configured to:
receive a subsequent-gaze direction from the eye-tracking system; and
select a subsequent-selected image sensor that is different from the selected image sensor when a subsequent-gaze vector representative of the subsequent-gaze direction becomes closer to a subsequent-selected FOV of the subsequent-selected image sensor that is different from the selected image sensor.
4 . The head mounted device of claim 3 , wherein the first image sensor is the selected image sensor and the second image sensor is the subsequent-selected image sensor.
5 . The head mounted device of claim 1 further comprising:
a memory, wherein the gaze-guided images are saved to the memory as a gaze-guided video file.
6 . The head mounted device of claim 1 , wherein the first FOV of the first image sensor does not overlap with the second FOV of the second image sensor.
7 . The head mounted device of claim 1 , wherein the first FOV of the first image sensor overlaps the second FOV of the second image sensor.
8 . A head mounted device comprising:
an eye-tracking system including one or more sensors configured to determine a gaze direction of an eye in an eyebox region of the head mounted device;
at least one camera configured to capture images of an external environment of the head mounted device; and
processing logic configured to:
receive the gaze direction from the eye-tracking system; and
generate one or more gaze-guided images from the images based on the gaze direction.
9 . The head mounted device of claim 8 , wherein generating the one or more gaze-guided images includes:
cropping the one or more images to generate the gaze-guided images, wherein the one or more images are cropped in response to the gaze direction with respect to a field of view (FOV) of the at least one camera.
10 . The head mounted device of claim 8 , wherein the at least one camera includes a lens assembly configured to focus image light onto an image sensor of the camera, and wherein generating the one or more gaze-guided images includes:
driving an optical zoom of the lens assembly in response to the gaze direction.
11 . The head mounted device of claim 8 , wherein the at least one camera includes a lens assembly configured to focus image light onto an image sensor of the camera, and wherein generating the one or more gaze-guided images includes:
adjusting an auto-focus of the lens assembly in response to the gaze direction.
12 . The head mounted device of claim 11 , wherein adjusting the auto-focus of the lens assembly in response to the gaze direction includes:
identifying a subject in the images that corresponds to the gaze direction;
determining an approximate focus distance to the subject in the images; and
adjusting the auto-focus of the lens assembly to the focus distance to image the subject.
13 . The head mounted device of claim 8 , wherein the at least one camera includes a lens assembly configured to focus image light onto an image sensor of the camera, and wherein generating the one or more gaze-guided images includes:
rotating the image sensor and the lens assembly in response to the gaze direction.
14 . A method of operating a head mounted device, the method comprising:
determining a gaze direction of an eye of a user of the head mounted device, wherein an eye-tracking system of the head mounted device determines the gaze direction of the user;
capturing one or more images with at least one camera of the head mounted device, wherein the at least one camera is configured to image an external environment of the head mounted device; and
generating one or more gaze-guided images from the one or more images based on the gaze direction of the user.
15 . The method of claim 14 , wherein the at least one camera is included in a plurality of cameras of the head mounted device, and wherein generating the one or more gaze-guided images includes:
selecting a selected camera among the plurality of cameras of the head mounted device, wherein the selected camera is selected to capture the one or more gaze-guided images based on the gaze direction.
16 . The method of claim 14 , wherein generating the one or more gaze-guided images includes:
cropping the one or more images to generate the gaze-guided images, wherein the one or more images are cropped in response to the gaze direction with respect to a field of view (FOV) of the at least one camera.
17 . The method of claim 14 , wherein generating the one or more gaze-guided images includes:
transmitting the gaze direction from the head mounted device to a mobile device;
transmitting the one or more images from the head mounted device to the mobile device, wherein processing logic of the mobile device generates the gaze-guided images from the one or more images based on the gaze direction of the user.
18 . The method of claim 14 , wherein the at least one camera includes a lens assembly configured to focus image light onto an image sensor of the camera, and wherein generating the one or more gaze-guided images includes:
adjusting an auto-focus of the lens assembly in response to the gaze direction.
19 . The method of claim 18 , wherein adjusting the auto-focus of the lens assembly in response to the gaze direction includes:
identifying a subject in the images that corresponds to the gaze direction;
determining an approximate focus distance to the subject in the images; and
adjusting the auto-focus of the lens assembly to the focus distance to image the subject.
20 . The method of claim 14 , wherein generating the one or more gaze-guided images includes:
identifying a focus distance that corresponds with the gaze direction of the user; and
applying blur effects to the one or more images to blur at least one of a foreground or a background in the one or more images, wherein the background has a background depth that is greater than the focus distance and the foreground has a foreground depth that is less than the focus distance.