Multi-viewer extended reality reprojection
A multi-viewer extended reality (XR) streaming method of streaming XR images between a plurality of XR application instances and a plurality of XR devices is described. The multi-viewer XR streaming method comprises the steps of streaming, by means of a first XR application instance of the plurality of XR application instances, first XR images to at least one XR device, wherein the first XR images correspond to virtual content viewed from a first spectator position; determining a position of the at least one XR device relative to the first spectator position; and reprojecting, by means of the at least one XR device, the first XR images based on the determined position of the at least one XR device. Further, an XR streaming system is described.
1 . A method of streaming extended reality (XR) images between a plurality of XR application instances and a plurality of XR devices, wherein the method comprises:
streaming, by a first XR application instance of the plurality of XR application instances, first XR images to a plurality of XR devices in a first subset of the plurality of XR devices, wherein the first XR images correspond to virtual content viewed from a first spectator position, wherein the first spectator position is determined based on an average of world positions of the plurality of XR devices in the first subset that are subscribed to the first XR application instance, wherein the first subset of the plurality of XR devices comprises at least two XR devices;
determining a position for each XR device of the first subset relative to the first spectator position; and
reprojecting, with XR devices of the first subset, the first XR images based on the corresponding positions of the XR devices of the first subset relative to the first spectator position to generate reprojected XR images.
2 . The method of claim 1 , further comprising:
streaming, by a second XR application instance of the plurality of XR application instances, second XR images to a second subset of XR devices of the plurality of XR devices, wherein the second XR images correspond to virtual content viewed from a second spectator position, wherein the second spectator position is different from the first spectator position;
determining a second position of the second subset of XR devices relative to the second spectator position; and
reprojecting, with the second subset of XR devices, the second XR images based on the position of XR devices of the second subset.
3 . The method of claim 1 , wherein the plurality of XR application instances executes a same XR application.
4 . The method of claim 1 , wherein the plurality of XR application instances is synchronized.
5 . The method of claim 1 , wherein each of the plurality of XR application instances are associated with a single spectator position.
6 . The method of claim 5 , wherein an additional XR device receives a stream of images for reprojection from an XR application instance hosting a closest spectator position to the additional XR device.
7 . The method of claim 1 , wherein a plurality of spectator positions including the first spectator position is arranged in a predefined pattern.
8 . The method of claim 1 , wherein at least one of the plurality of XR devices receives XR images from at least two XR application instances.
9 . The method of claim 1 , wherein the first XR images further comprise image data associated with the first XR images, wherein the image data comprises information of at least one of a view matrix and a projection matrix.
10 . The method of claim 9 , wherein at least one of the view matrix and the projection matrix are associated with a corresponding spectator position.
11 . A system, comprising:
a server hosting a plurality of XR application instances connected to a plurality of XR devices, wherein the server is configured to stream, by a first XR application instance of the plurality of XR application instances, first XR images to a first subset of the plurality of XR devices, wherein the first XR images are rendered to display virtual content viewed from a first spectator position, wherein the first spectator position is determined based on an average of world positions of a plurality of XR devices in the first subset that are subscribed to the first XR application instance, wherein the first subset of the plurality of XR devices comprises at least two XR devices; and
a first XR device of the plurality of XR devices in the first subset that is configured to, determine a position of the first XR device relative to the first spectator position; and
reproject the first XR images based on the corresponding position of the first XR device relative to the first spectator position to generate reprojected XR images.
12 . The system of claim 11 , further comprising a plurality of servers, wherein one or more XR application instances of the plurality of XR application instances are implemented on each of the plurality of servers, wherein the plurality of servers includes the server.
13 . The system of claim 11 , wherein the server comprises a user management circuit that is configured to allocate at least one of the plurality of XR application instances to each of the plurality of XR devices.
14 . A non-transitory machine-readable medium having program code stored thereon, the program code comprising instructions to:
stream, by a first XR application instance of a plurality of XR application instances, first XR images to a plurality of XR devices in a first subset of a plurality of XR devices, wherein the first XR images correspond to virtual content viewed from a first spectator position, wherein the first spectator position is determined based on an average of world positions of the plurality of XR devices in the first subset that are subscribed to the first XR application instance, wherein the first subset of the plurality of XR devices comprises at least two XR devices;
determine a position for each XR device of the first subset relative to the first spectator position; and
reproject, with XR devices of the first subset, the first XR images based on the corresponding position of each XR device of the first subset relative to the first spectator position to generate reprojected XR images.
15 . The method of claim 1 , further comprising determining a plurality of spectator positions including the first spectator position so that an average distance of the XR devices subscribed to respective XR application instances of the plurality of XR application instances is minimized.
16 . The system of claim 11 , wherein the plurality of XR application instances hosted on the server execute a same XR application.
17 . The system of claim 11 , wherein the plurality of XR application instances hosted on the server are each associated with a single spectator position.
18 . The system of claim 11 , wherein the server is configured to stream to each XR device of the plurality of XR devices a stream of images for reprojection from an XR application instance corresponding to a closest spectator position to the XR device.
19 . The system of claim 11 , wherein at least one XR device of the plurality of XR devices receives XR images from at least two XR application instances.
20 . The system of claim 11 , wherein the server is configured to stream XR image data associated with the first XR images to the plurality of XR devices, wherein the XR image data comprises information for at least one of a view matrix and a projection matrix.