Methods, apparatus and systems for optimizing communication between sender(s) and receiver(s) in computer-mediated reality applications

1. A method of processing audio content, the method comprising:

obtaining, using one or more processors, first pose information indicative of a first position or orientation of a user;

obtaining, using the one or more processors, predicted pose information based on the first pose information and previous pose information;

rendering, using the one or more processors, the audio content based on the predicted pose information to obtain rendered audio content; and

transmitting, using the one or more processors, the rendered audio content and predicted pose information to a first apparatus for reproduction, wherein the predicted pose information and second pose information are used to update the rendered audio content prior to reproducing the updated rendered audio content at the first apparatus.

2. The method according to claim 1 , wherein the audio content is one of First Order Ambisonics, FOA-based, Higher Order Ambisonics, HOA-based, object-based, or channel based audio content, or a combination of two or more of FOA-based, HOA-based, object-based, or channel based audio content.

3. The method according to claim 1 , wherein the rendered audio content is one of binaural audio content, FOA audio content, HOA audio content, or channel-based audio content, or a combination of two or more of binaural audio content, FOA audio content, HOA audio content, or channel-based audio content.

4. The method according to claim 1 , wherein using the predicted pose information and the second pose information to update the rendered audio content further comprises

comparing the predicted pose information to the second pose information; and

updating the rendered audio content based on a result of the comparison.

5. The method according to claim 4 , wherein the predicted pose information is predicted for an estimate of a timing at which the rendered audio content is expected to be processed by the first apparatus for reproduction; and

the second pose information is pose information obtained at a timing at which the rendered audio content is actually processed by the first apparatus for reproduction.

6. The method according to claim 1 , wherein the rendered audio content is transmitted to the first apparatus in uncompressed form.

7. The method according to claim 1 , further comprising:

encoding the rendered audio content before transmission to the first apparatus; and decoding the encoded rendered audio content after reception at the first apparatus.

8. The method according to claim 5 , wherein the estimate of the timing at which the rendered audio content is expected to be processed by the first apparatus for reproduction includes an estimation of a time that is necessary for encoding and decoding the rendered audio content and/or an estimate of a time that is necessary for transmitting the rendered audio content to the first apparatus.

9. The method according to claim 1 , wherein the predicted pose information is obtained further based on an estimate of a time that is necessary for encoding and decoding the rendered audio content and/or an estimate of a time that is necessary for transmitting the rendered audio content to the first apparatus.

10. The method according to claim 1 , further comprising:

comparing the predicted pose information that has been used for rendering the audio content to the second pose information; and

updating the rendered audio content based on a result of the comparison.

11. The method according to claim 1 , further comprising:

determining, at the second apparatus, gradient information indicative of how the rendered audio content changes in response to changes of the pose information;

transmitting the gradient information to the first apparatus together with the rendered audio content;

comparing, at the first apparatus, the pose information that has been used for rendering the audio content to current pose information; and

updating the rendered audio content based on the gradient information and a result of the comparison.

12. The method according to claim 1 , wherein

the method further comprises transmitting environmental information indicative of acoustic characteristics of an environment in which the first apparatus is located to the second apparatus; and

the rendering the audio content is further based on the environmental information.

13. The method according to claim 1 , wherein

the method further comprises transmitting morphologic information indicative of a morphology of the user or part of the user to the second apparatus; and

the rendering the audio content is further based on the morphologic information.

14. A system comprising:

one or more processors;

memory storing instructions, that when executed by one or more processors, cause the one or more processors to perform operations comprising:

obtaining first pose information indicative of a first position or orientation of a user;

obtaining predicted pose information based on the first pose information and previous pose information;

rendering the audio content based on the predicted pose information to obtain rendered audio content; and

transmitting the rendered audio content and the predicted pose information to the first apparatus for reproduction, wherein the predicted pose information and second pose information are used to update the rendered audio content prior to reproducing the updated rendered audio content at the first apparatus.

15. The system according to claim 14 , wherein the rendered audio content is one of binaural audio content, FOA audio content, HOA audio content, or channel-based audio content, or a combination of two or more of binaural audio content, FOA audio content, HOA audio content, or channel-based audio content.

16. A method of processing media content for reproduction by a first apparatus, the method comprising:

obtaining, using one or more processors, pose information indicative of a position or orientation of a user;

obtaining, using the one or more processors, predicted pose information based on the pose information and previous pose information;

rendering, using the one or more processors, the media content based on the predicted pose information to obtain rendered media content;

obtaining, using the one or more processors, gradient information indicative of how the rendered media content changes in response to changes of the pose information; and

transmitting, using the one or more processors, the rendered media content and gradient information to the first apparatus for use in updating and reproducing the updated rendered media content by the first apparatus.

17. The method of claim 16 , further comprising:

transmitting, using the one or more processors, the predicted pose to the first apparatus for use in updating and reproducing the updated rendered media content by the first apparatus.

18. A method comprising:

obtaining, using one or more processors of a first apparatus, first pose information indicative of a first position or orientation of a user;

transmitting, using the one or more processors, the first pose information to a second apparatus;

receiving, using the one or more processors, rendered audio content from the second apparatus, where the rendered audio content was rendered by the second apparatus using predicted pose information, the predicted pose information based on the first pose information;

obtaining, using the one or more processors, second pose information indicative of a second position or orientation of the user;

updating, using the one or more processors, the rendered audio content based on the second pose information; and

reproducing, using the one or more processors, the updated rendered audio content.

19. The method of claim 18 , further comprising:

receiving, using the one or more processors, the predicted pose from the second apparatus; and

updating, using the one or more processors, the rendered audio content based on a difference between the predicted pose and the second pose information.

20. A system comprising a first apparatus for reproducing rendered audio content and a second apparatus for producing the rendered audio content,

the first apparatus configured to:

obtain first pose information indicative of a first position or orientation of a user;

transmit the first pose information to the second apparatus;

receive rendered audio content from the second apparatus;

obtain second pose information indicative of a second position or orientation of the user;

update the rendered audio content using at least the second pose information; and

reproduce the updated rendered audio content.

21. The system of claim 20 , wherein the first apparatus is further configured to receive the predicted pose from the first apparatus and to update the rendered audio content based on a difference between the predicted pose and the second pose information.