Foveated image capture for power efficient video see-through
Generating an image stream may include obtaining image data from a camera, identifying a first subset of the image data including a region of interest, identifying a second subset of the image data different than the first subset of the image data, processing the first subset of image data by a first processing pipeline to obtain a first processed set of image data, processing the second subset of image data by a second processing pipeline to obtain a second processed set of image data, wherein the second processing pipeline processes at a lower quality than the first processing pipeline, and combining the first processed set of image data and the second processed set of image data to obtain a processed image frame.
1. A method comprising:
obtaining image data from a camera corresponding to an image frame;
determining a region of interest in the image data;
identifying a first subset of the image data corresponding to the region of interest and a second subset of the image data different than the first subset of the image data;
processing, by a first processing pipeline of a plurality of processing pipelines, the first subset of the image data to obtain a first processed set of the image data, wherein the first processing pipeline is configured to apply a first processing technique;
processing the second subset of the image data by a second processing pipeline of the plurality of processing pipelines to obtain a second processed set of the image data, wherein the second processing pipeline is configured to perform a different processing method than the first processing pipeline, wherein the second processing pipeline is configured to apply a second processing technique; and
combining the first processed set of the image data and the second processed set of the image data to obtain a processed image frame.
2. The method of claim 1 , wherein the region of interest is identified based on a view direction of a user.
3. The method of claim 2 , wherein the view direction of the user is determined using an eye tracking sensor.
4. The method of claim 1 , further comprising:
providing the processed image frame for presentation on a display device,
wherein the region of interest is determined in accordance with one or more display properties of the display device.
5. The method of claim 1 , wherein the first processing pipeline is less power efficient than the second processing pipeline.
6. The method of claim 1 , wherein the first processing pipeline produces higher resolution image data than the second processing pipeline.
7. The method of claim 1 , wherein the second subset of the image data comprises at least a portion of the first subset of the image data.
8. A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
obtain image data from a camera corresponding to an image frame;
determine a region of interest in the image data;
identify a first subset of the image data corresponding to the region of interest and a second subset of the image data different than the first subset of the image data;
process, by a first processing pipeline of a plurality of processing pipelines, the first subset of the image data to obtain a first processed set of the image data, wherein the first processing pipeline is configured to apply a first processing technique;
process the second subset of the image data by a second processing pipeline of the plurality of processing pipelines to obtain a second processed set of the image data, wherein the second processing pipeline is configured to perform a different processing method than the first processing pipeline, wherein the second processing pipeline is configured to apply a second processing technique; and
combine the first processed set of the image data and the second processed set of the image data to obtain a processed image frame.
9. The non-transitory computer readable medium of claim 8 , wherein the region of interest is identified based on a view direction of a user.
10. The non-transitory computer readable medium of claim 9 , wherein the view direction of the user is determined using an eye tracking sensor.
11. The non-transitory computer readable medium of claim 8 , further comprising computer readable code to:
provide the processed image frame for presentation on a display device,
wherein the region of interest is determined in accordance with one or more display properties of the display device.
12. The non-transitory computer readable medium of claim 8 , wherein the first processing pipeline is less power efficient than the second processing pipeline.
13. The non-transitory computer readable medium of claim 8 , wherein the first processing pipeline produces higher resolution image data than the second processing pipeline.
14. The non-transitory computer readable medium of claim 8 , wherein the second subset of the image data comprises at least a portion of the first subset of the image data.
15. A system comprising:
one or more processors; and
one or more non-transitory computer readable media comprising computer readable code executable by the one or more processors to:
obtain image data from a camera corresponding to an image frame;
determine a region of interest in the image data;
identify a first subset of the image data corresponding to the region of interest and a second subset of the image data different than the first subset of the image data;
process, by a first processing pipeline of a plurality of processing pipelines, the first subset of the image data to obtain a first processed set of the image data, wherein the first processing pipeline is configured to apply a first processing technique;
process the second subset of the image data by a second processing pipeline of the plurality of processing pipelines to obtain a second processed set of the image data, wherein the second processing pipeline is configured to perform a different processing method than the first processing pipeline, wherein the second processing pipeline is configured to apply a second processing technique; and
combine the first processed set of the image data and the second processed set of the image data to obtain a processed image frame.
16. The system of claim 15 , wherein the region of interest is identified based on a view direction of a user.
17. The system of claim 16 , wherein the view direction of the user is determined using an eye tracking sensor.
18. The system of claim 15 , further comprising computer readable code to:
provide the processed image frame for presentation on a display device,
wherein the region of interest is determined in accordance with one or more display properties of the display device.
19. The system of claim 15 , wherein the first processing pipeline is less power efficient than the second processing pipeline.
20. The system of claim 15 , wherein the first processing pipeline produces higher resolution image data than the second processing pipeline.