IP Library › Granted Patent US 11,652,862
Granted Patent B2
US 11,652,862 · App. 17/892,019 · Granted May 16, 2023

Immersive teleconferencing and telepresence

Inventor: Ozgur Oyman (Cupertino, CA)
Assignee: Apple Inc.
H04L65/65H04L65/1016H04L65/1104H04L65/403
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,652,862
App. No.
17/892,019
Granted
May 16, 2023
Kind
B2
Abstract

Embodiments may relate to a user equipment (UE) that is configured to determine a real-time transport protocol (RTP) media flow that includes visual data related to a plurality of images concurrently taken of a location and a supplemental information enhancement (SEI) message that is to be used to display at least a portion of the visual data. The UE is further configured to visually display, based on the visual data and the SEI message, the portion of the visual data to a user of the user device. Other embodiments may be described or claimed.

Claims (29)

1. An apparatus comprising:

first circuitry to determine, based on a received real-time transport protocol (RTP) stream that includes visual data related to a plurality of images concurrently taken of a location, an elementary stream;

second circuitry to decode, based on the elementary stream, the visual data and a supplemental enhancement information (SEI) message;

third circuitry to generate, based on the visual data and the SEI message, a mapping of the visual data to a visual field; and

fourth circuitry to output, to a display device, data related to the mapping of the visual data to the visual field.

2. The apparatus of claim 1 , wherein the apparatus comprises a user equipment (UE) of a third generation partnership project (3GPP) network, and wherein the UE includes the display device.

3. The apparatus of claim 1 , wherein the SEI message is an equirectangular projection SEI message, a cubemap projection SEI message, a sphere rotation SEI message, or a region-wise packing SEI message.

4. The apparatus of claim 1 , further comprising fifth circuitry to facilitate transmission, in a RTP control protocol (RTCP) feedback message, an indication of a desired viewing orientation of the visual data.

5. The apparatus of claim 4 , wherein the visual data is based on the indication of the desired viewing orientation.

6. The apparatus of claim 1 , wherein the RTP stream is a first RTP stream that includes first visual data related to a first image of the plurality of images, and wherein the elementary stream is further based on a decoded second RTP stream that includes second visual data related to a second image of the plurality of images.

7. The apparatus of claim 1 , wherein the RTP stream includes visual data related to at least a first image and a second image of the plurality of images.

8. The apparatus of claim 1 , wherein the visual data is a high efficiency video coding (HEVC) bitstream that includes the SEI message.

9. The apparatus of claim 1 , wherein the first circuitry determines, based on the received real-time transport protocol (RTP) stream i) for a video conference between two or more devices that include the apparatus and ii) that includes the visual data related to the plurality of images concurrently taken of the location, the elementary stream.

10. The apparatus of claim 1 , wherein the visual data is based on an Omnidirectional Media Format (OMAF) video profile.

11. A method comprising:

obtaining, based on a received real-time transport protocol (RTP) stream that includes visual data related to a plurality of images a for a video conference between two or more devices, the visual data and a supplemental enhancement information (SEI) message;

generating, based on the visual data and the SEI message, a mapping of the visual data to a visual field; and

outputting, to a display device, data related to the mapping of the visual data to the visual field.

12. The method of claim 11 , wherein obtaining the visual data and the SEI message comprises obtaining, by one or more baseband processors, the visual data and the SEI message.

13. The method of claim 11 , wherein the SEI message is an equirectangular projection SEI message, a cubemap projection SEI message, a sphere rotation SEI message, or a region-wise packing SEI message.

14. The method of claim 11 , comprising transmitting, in a RTP control protocol (RTCP) feedback message, an indication of a desired viewing orientation of the visual data.

15. The method of claim 14 , wherein the visual data is based on the indication of the desired viewing orientation.

16. The method of claim 11 , wherein:

the RTP stream is a first RTP stream that includes first visual data related to a first image of the plurality of images; and

obtaining the visual data and the SEI message comprises obtaining the visual data and the SEI message based on the first RTP stream and a second RTP stream that includes second visual data related to a second image of the plurality of images.

17. The method of claim 11 , wherein the RTP stream includes visual data related to at least a first image and a second image of the plurality of images.

18. The method of claim 11 , wherein the visual data is a high efficiency video coding (HEVC) bitstream that includes the SEI message.

19. The method of claim 11 , wherein the plurality of images were substantially concurrently taken of a location.

20. The method of claim 11 , wherein the visual data is based on an Omnidirectional Media Format (OMAF) video profile.