Inter-eye prediction models for XR
Method and apparatus for inter-eye prediction models for XR. The apparatus transforms at least part of a first frame associated with a first perspective of a first camera to at least part of a second frame associated with a second perspective of a second camera based on at least one prediction model. The apparatus transmits, to a second wireless device, a prediction indication of the at least one prediction model used to transform the at least part of the first frame to the at least part of the second frame. The apparatus transmits, to the second wireless device, an encoded signal comprising a combined frame comprising a combination of at least the first frame and the second frame.
1 . An apparatus for wireless communication at a first wireless device, comprising:
at least one memory; and
at least one processor coupled to the at least one memory and, based at least in part on information stored in the at least one memory, the at least one processor, individually or in any combination, is configured to cause the apparatus to:
transform at least part of a first frame associated with a first perspective of a first camera to at least part of a second frame associated with a second perspective of a second camera based on at least one prediction model;
transmit, to a second wireless device, a prediction indication of the at least one prediction model used to transform the at least part of the first frame to the at least part of the second frame; and
transmit, to the second wireless device, an encoded signal comprising a combined frame comprising a combination of at least the first frame and the second frame.
2 . The apparatus of claim 1 , further comprising a transceiver coupled to the at least one processor, the transceiver being configured to:
transmit, to the second wireless device, the prediction indication of the at least one prediction model used to transform the at least part of the first frame to the at least part of the second frame; and
transmit, to the second wireless device, the encoded signal comprising the combined frame comprising the combination of the at least the first frame and the second frame.
3 . The apparatus of claim 1 , wherein the at least one prediction model predicts the at least part of the second frame based on the at least part of the first frame.
4 . The apparatus of claim 1 , wherein the at least one processor is configured to:
select the at least one prediction model based on a prediction quality, wherein the at least one prediction model is selected from a set of prediction models.
5 . The apparatus of claim 1 , wherein the prediction indication comprises one or more parameters associated with the at least one prediction model.
6 . The apparatus of claim 1 , wherein the prediction indication is transmitted when a connection between the first wireless device and the second wireless device is established.
7 . The apparatus of claim 1 , wherein the prediction indication is transmitted within at least one of a downlink control indicator (DCI), radio resource control (RRC) signaling, or media access control (MAC) control element (CE) (MAC-CE).
8 . The apparatus of claim 1 , wherein the prediction indication is transmitted in a periodic configuration or an aperiodic configuration.
9 . The apparatus of claim 1 , wherein the at least one processor is configured to:
provide, to the second camera, a transformed portion of the at least part of the first frame, wherein the transformed portion comprises an overlap region between the first perspective and the second perspective.
10 . The apparatus of claim 9 , wherein the transformed portion is transformed based on the at least one prediction model, wherein the second perspective comprises the transformed portion and at least a second region that extends beyond the first perspective.
11 . The apparatus of claim 10 , wherein the first perspective comprises a first region that extends beyond the overlap region and the second perspective.
12 . A method of wireless communication at a first wireless device, comprising:
transforming at least part of a first frame associated with a first perspective of a first camera to at least part of a second frame associated with a second perspective of a second camera based on at least one prediction model;
transmitting, to a second wireless device, a prediction indication of the at least one prediction model used to transform the at least part of the first frame to the at least part of the second frame; and
transmitting, to the second wireless device, an encoded signal comprising a combined frame comprising a combination of at least the first frame and the second frame.
13 . The method of claim 12 , wherein the at least one prediction model predicts the at least part of the second frame based on the at least part of the first frame.
14 . The method of claim 12 , further comprising:
selecting the at least one prediction model based on a prediction quality, wherein the at least one prediction model is selected from a set of prediction models.
15 . The method of claim 12 , further comprising:
providing, to the second camera, a transformed portion of the at least part of the first frame, wherein the transformed portion comprises an overlap region between the first perspective and the second perspective.
16 . The method of claim 15 , wherein the transformed portion is transformed based on the at least one prediction model, wherein the second perspective comprises the transformed portion and at least a second region that extends beyond the first perspective, wherein the first perspective comprises a first region that extends beyond the overlap region and the second perspective.
17 . An apparatus for wireless communication at a second wireless device, comprising:
at least one memory; and
at least one processor coupled to the at least one memory and, based at least in part on information stored in the at least one memory, the at least one processor, individually or in any combination, is configured to cause the apparatus to:
receive, from a first wireless device, a prediction indication indicating at least one prediction model used to transform at least part of a first frame associated with a first perspective of a first camera to at least part of a second frame associated with a second perspective of a second camera;
receive, from the first wireless device, a compressed signal based on an encoder scheme, wherein an encoded signal comprising a combined frame comprising a combination of at least the first frame and the second frame; and
decode the combined frame based on the at least one prediction model.
18 . The apparatus of claim 17 , further comprising a transceiver coupled to the at least one processor, the transceiver being configured to:
receive, from the first wireless device, the prediction indication indicating the at least one prediction model used to transform the at least part of the first frame associated with the first perspective of the first camera to the at least part of the second frame associated with the second perspective of the second camera; and
receive, from the first wireless device, the compressed signal based on the encoder scheme, wherein the encoded signal comprising the combined frame comprising the combination of the at least the first frame and the second frame.
19 . The apparatus of claim 17 , wherein the at least one prediction model predicts the at least part of the second frame based on the at least part of the first frame.
20 . The apparatus of claim 17 , wherein the prediction indication comprises one or more parameters associated with the at least one prediction model.
21 . The apparatus of claim 17 , wherein the prediction indication is received when a connection between the first wireless device and the second wireless device is established.
22 . The apparatus of claim 17 , wherein the prediction indication is received within at least one of a downlink control indicator (DCI), radio resource control (RRC) signaling, or media access control (MAC) control element (CE) (MAC-CE).
23 . The apparatus of claim 17 , wherein the prediction indication is received in a periodic configuration or an aperiodic configuration.
24 . The apparatus of claim 17 , wherein the combined frame comprises a transformed portion of the at least part of the first frame, wherein the transformed portion comprises an overlap region between the first perspective and the second perspective.
25 . The apparatus of claim 24 , wherein the transformed portion is transformed based on the at least one prediction model, wherein the second perspective comprises the transformed portion and at least a second region that extends beyond the first perspective.
26 . The apparatus of claim 25 , wherein the first perspective comprises a first region that extends beyond the overlap region and the second perspective.
27 . A method of wireless communication at a second wireless device, comprising:
receiving, from a first wireless device, a prediction indication indicating at least one prediction model used to transform at least part of a first frame associated with a first perspective of a first camera to at least part of a second frame associated with a second perspective of a second camera;
receiving, from the first wireless device, a compressed signal based on an encoder scheme, wherein an encoded signal comprising a combined frame comprising a combination of at least the first frame and the second frame; and
decoding the combined frame based on the at least one prediction model.
28 . The method of claim 27 , wherein the at least one prediction model predicts the at least part of the second frame based on the at least part of the first frame.
29 . The method of claim 27 , wherein the combined frame comprises a transformed portion of the at least part of the first frame, wherein the transformed portion comprises an overlap region between the first perspective and the second perspective.
30 . The method of claim 29 , wherein the transformed portion is transformed based on the at least one prediction model, wherein the second perspective comprises the transformed portion and at least a second region that extends beyond the first perspective, wherein the first perspective comprises a first region that extends beyond the overlap region and the second perspective.