Multi-stream foveal display transport
Systems, apparatuses, and methods for using a multi-stream foveal display transport layer are disclosed. A virtual reality (VR) system includes a transmitter sending a plurality of streams over a display transport layer to a receiver coupled to a display. Each stream corresponds to a different image to be blended together by the receiver. The images include at least a foveal region image corresponding to a gaze direction of the eye and a background image which is a lower-resolution image with a wider field of view than the foveal region image. The phase timing of the foveal region stream being sent over the transport layer is adjusted with respect to the background stream to correspond to the location of the foveal region within the overall image. This helps to reduce the amount of buffering needed at the receiver for blending the images together to create a final image to be driven to the display.
1. A system comprising:
a rendering unit, comprising circuitry, configured to render a plurality of regions of a frame, wherein each region of the plurality of regions is rendered at a different resolution from other regions; and
a transmitter, comprising circuitry, configured to:
transmit each region of the plurality of regions as a separate image in a separate stream to a receiver; and
transmit a first stream by shifting a phase of the first stream with respect to a second stream by changing a time at which the first stream is transmitted, wherein the phase is shifted by an amount which corresponds to a location of a foveal region within the frame, wherein the first stream contains the foveal region, and wherein the second stream contains a background image.
2. The system as recited in claim 1 , wherein:
the foveal region is updated at a first refresh rate;
the background image is updated at a second refresh rate; and
the second refresh rate is different from the first refresh rate.
3. The system as recited in claim 1 , wherein the rendering unit is further configured to increase a number of regions being rendered at different resolutions responsive to receiving an indication that an eye saccade has been detected.
4. The system as recited in claim 3 , wherein the rendering unit is further configured to:
render two separate regions for a first frame prior to receiving the indication; and
render more than two separate regions for a second frame subsequent to receiving the indication.
5. The system as recited in claim 3 , wherein the rendering unit is further configured to adjust a refresh rate of the foveal region responsive to receiving the indication.
6. The system as recited in claim 1 , wherein the transmitter is further configured to convey a blend mask to the receiver to specify how to combine the plurality of regions to create the composite image.
7. The system as recited in claim 1 , wherein the transmitter is further configured to convey a background image scaling attribute to the receiver to specify how to scale the background image.
8. A method comprising:
rendering, by a rendering unit comprising circuitry, a plurality of regions of a frame, wherein each region of the plurality of regions is rendered at a different resolution from other regions;
transmitting, by a transmitter comprising circuitry, each region as a separate image in a separate stream to a receiver; and
transmitting a first stream by shifting a phase of the first stream with respect to a second stream by changing a time at which the first stream is transmitted, wherein the phase is shifted by an amount which corresponds to a location of a foveal region within the frame, wherein the first stream contains the foveal region, and wherein the second stream contains a background image.
9. The method as recited in claim 8 , wherein:
the foveal region is updated at a first refresh rate;
the background image is updated at a second refresh rate; and
the second refresh rate is different from the first refresh rate.
10. The method as recited in claim 8 , further comprising increasing a number of regions being rendered at different resolutions responsive to receiving an indication that an eye saccade has been detected.
11. The method as recited in claim 10 , further comprising:
rendering two separate regions for a first frame prior to receiving the indication; and
rendering more than two separate regions for a second frame subsequent to receiving the indication.
12. The method as recited in claim 10 , further comprising adjusting a refresh rate of the foveal region responsive to receiving the indication.
13. The method as recited in claim 8 , further comprising conveying a blend mask to the receiver to specify how to combine the plurality of regions to create the composite image.
14. The method as recited in claim 8 , further comprising conveying a background image scaling attribute to the receiver to specify how to scale the background image.
15. A system comprising:
a transmitter, comprising circuitry, configured to:
render a plurality of regions of a frame, wherein each region of the plurality of regions is rendered at a different resolution from other regions;
transmit each region as a separate image in a separate stream to a receiver; and
transmit a first stream by shifting a phase of the first stream with respect to a second stream by changing a time at which the first stream is transmitted, wherein the phase is shifted by an amount which corresponds to a location of a foveal region within the frame, wherein the first stream contains the foveal region, and wherein the second stream contains a background image; and
the receiver configured to:
combine the plurality of regions from a plurality of streams to create a composite image; and
drive the composite image to a display.
16. The system as recited in claim 15 , wherein:
the foveal region is updated at a first refresh rate;
the background image is updated at a second refresh rate; and
the second refresh rate is different from the first refresh rate.
17. The system as recited in claim 15 , wherein the transmitter is further configured to increase a number of regions being rendered at different resolutions responsive to receiving an indication that an eye saccade has been detected.
18. The system as recited in claim 17 , wherein the transmitter is further configured to:
render two separate regions for a first frame prior to receiving the indication; and
render more than two separate regions for a second frame subsequent to receiving the indication.
19. The system as recited in claim 17 , wherein the transmitter is further configured to adjust a refresh rate of the foveal region responsive to receiving the indication.
20. The system as recited in claim 15 , wherein the transmitter is further configured to convey a blend mask to the receiver to specify how to combine the plurality of regions to create the composite image.