Artificial reality system using superframes to communicate surface data
This disclosure describes efficient communication of surface texture data between system on a chip (SOC) integrated circuits. An example system includes a first integrated circuit and a second integrated circuit communicatively coupled to the first integrated circuit by a video communication interface. The first integrated generates a superframe in a video frame of the video communication interface for transmission to the second integrated circuit. The superframe includes multiple subframe payloads that carry surface texture data to be updated in the frame and corresponding subframe headers that include parameters of the subframe payloads. The second integrated circuit includes a direct access memory (DMA) controller. The DMA upon receipt of the superframe, writes the surface texture data within each of the subframe payloads directly to an allocated location in memory based on the parameters included in the corresponding one of the subframe headers.
1. An artificial reality system, comprising:
a first integrated circuit; and
a second integrated circuit communicatively coupled to the first integrated circuit by a data communication interface and a video communication interface;
wherein the first integrated circuit comprises at least one processor configured to:
identify surfaces of a frame to be displayed in a next display cycle, wherein each of the surfaces has a specified width and height within the frame and comprises surface texture data;
categorize a portion of the surfaces of the frame as dynamic surfaces to be updated in the frame, wherein each of the dynamic surfaces comprises one of the surfaces for which the surface texture data changes in the frame;
generate a setup frame for transmission to the second integrated circuit using the data communication interface, wherein the setup frame defines the dynamic surfaces to be updated in the frame; and
generate a superframe as a video frame of a video data transfer protocol defined to carry pixel data for transmission to the second integrated circuit using the video communication interface, wherein the superframe includes multiple subframe payloads that carry the surface texture data of the dynamic surfaces to be updated in the frame and a superframe header that includes subframe headers corresponding to the subframe payloads, and wherein each of the subframe headers includes parameters of a corresponding one of the subframe payloads;
wherein the second integrated circuit comprises a direct access memory (DMA) controller configured to:
upon receipt of the superframe, write the surface texture data within each of the subframe payloads directly to an allocated location in memory based on the parameters included in the corresponding one of the subframe headers; and
transmit the surface texture data from the memory to a display device to be rendered for display.
2. The artificial reality system of claim 1 , wherein the superframe header including the subframe headers is positioned at a beginning of the superframe.
3. The artificial reality system of claim 1 , wherein the superframe header includes multiple validity fields used by the second integrated circuit to determine whether the superframe is valid.
4. The artificial reality system of claim 1 , wherein the superframe header specifies a location within the superframe of a first subframe payload of the multiple subframe payloads.
5. The artificial reality system of claim 1 , wherein each of the subframe headers specifies an input pixel format of the surface texture data within the corresponding one of the subframe payloads and an output pixel format for the surface texture data within the corresponding one of the subframe payloads.
6. The artificial reality system of claim 5 , wherein the second integrated circuit is configured to, when the input pixel format and the output pixel format do not match, convert the surface texture data within the corresponding one of the subframe payloads from the first pixel format to the second pixel format before storing the surface texture data into the memory.
7. The artificial reality system of claim 1 , wherein each of the subframe headers within the superframe header specifies a location within the superframe of the corresponding one of the subframe payloads.
8. The artificial reality system of claim 1 , wherein the first integrated circuit is configured to transmit the superframe to the second integrated circuit as the video frame of the video data transfer protocol using a mode of the video communication interface that does not alter the surface texture data within the subframe payloads of the superframe during transmission.
9. The artificial reality system of claim 1 , wherein the DMA controller of the second integrated circuit is configured to, upon receipt of the superframe, write the subframe headers within the superframe header of the superframe into status and control registers.
10. A method comprising:
identifying, by a first integrated circuit of an artificial reality (AR) system, surfaces of a frame to be displayed in a next display cycle, wherein each of the surfaces has a specified width and height within the frame and comprises surface texture data;
categorizing, by the first integrated circuit, a portion of the surfaces of the frame as dynamic surfaces to be updated in the frame, wherein each of the dynamic surfaces comprises one of the surfaces for which the surface texture data changes in the frame;
generating, by the first integrated circuit, a setup frame for transmission to a second integrated circuit of the AR system using a data communication interface, wherein the setup frame defines the dynamic surfaces to be updated in the frame;
generating, by the first integrated circuit, a superframe as a video frame of a video data transfer protocol defined to carry pixel data for transmission to the second integrated circuit using a video communication interface, wherein the superframe includes multiple subframe payloads that carry the surface texture data of the dynamic surfaces to be updated in the frame and a superframe header that includes subframe headers corresponding to the subframe payloads, and wherein each of the subframe headers includes parameters of the corresponding one of the subframe payloads;
upon receipt of the superframe, writing, by a direct access memory (DMA) controller of the second integrated circuit, the surface texture data within each of the subframe payloads directly to an allocated location in memory based on the parameters included in the corresponding one of the subframe headers; and
transmitting, by the second integrated circuit, the surface texture data from the memory to a display device to be rendered for display.
11. The method of claim 10 , wherein generating the superframe comprises generating the superframe header positioned at a beginning of the superframe, the superframe header including the subframe headers.
12. The method of claim 10 , wherein the superframe header includes multiple validity fields used by the second integrated circuit to determine whether the superframe is valid.
13. The method of claim 10 , wherein the superframe header specifies a location within the superframe of a first subframe payload of the multiple subframe payloads.
14. The method of claim 10 , wherein each of the subframe headers specifies an input pixel format of the surface texture data within the corresponding one of the subframe payloads and an output pixel format for the surface texture data within the corresponding one of the subframe payloads.
15. The method of claim 14 , further comprising, when the input pixel format and the output pixel format do not match, converting, by the second integrated circuit, the surface texture data within the corresponding one of the subframe payloads from the first pixel format to the second pixel format before storing the surface texture data into the memory.
16. The method of claim 10 , wherein each of the subframe headers within the superframe header specifies a location within the superframe of the corresponding one of the subframe payloads.
17. The method of claim 10 , further comprising, upon receipt of the superframe, writing, by the DMA controller of the second integrated circuit, the subframe headers within the superframe header of the superframe into status and control registers.
18. A non-transitory computer-readable storage medium comprising instructions that, when executed, configure processing circuitry of an artificial reality (AR) system to:
identify, by a first integrated circuit of the AR system, surfaces of a frame to be displayed in a next display cycle, wherein each of the surfaces has a specified width and height within the frame and comprises surface texture data;
categorize, by the first integrated circuit, a portion of the surfaces of the frame as dynamic surfaces to be updated in the frame, wherein each of the dynamic surfaces comprises one of the surfaces for which the surface texture data changes in the frame;
generate, by the first integrated circuit, a setup frame for transmission to a second integrated circuit of the AR system using a data communication interface, wherein the setup frame defines the dynamic surfaces to be updated in the frame;
generate, by the first integrated circuit, a superframe as a video frame of a video data transfer protocol defined to carry pixel data for transmission to the second integrated circuit using a video communication interface, wherein the superframe includes multiple subframe payloads that carry the surface texture data of the dynamic surfaces to be updated in the frame and a superframe header that includes subframe headers corresponding to the subframe payloads, and wherein each of the subframe headers includes parameters of the corresponding one of the subframe payloads;
upon receipt of the superframe, write, by a direct access memory (DMA) controller of the second integrated circuit, the surface texture data within each of the subframe payloads directly to an allocated location in memory based on the parameters included in the corresponding one of the subframe headers; and
transmit, by the second integrated circuit, the surface texture data from the memory to a display device to be rendered for display.