IP Library Granted Patent US 11,528,312
Granted Patent B2
US 11,528,312 · App. 17/439,322 · Granted Dec 13, 2022

Immersive teleconferencing and telepresence

Inventor: Ozgur Oyman (Cupertino, CA)
Assignee: Apple Inc.
H04L65/65H04L65/1016H04L65/1104H04L65/403
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,528,312
App. No.
17/439,322
Granted
Dec 13, 2022
Kind
B2
Abstract

Embodiments may relate to a user equipment (UE) that is configured to determine a real-time transport protocol (RTP) media flow that includes visual data related to a plurality of images concurrently taken of a location and a supplemental information enhancement (SEI) message that is to be used to display at least a portion of the visual data. The UE is further configured to visually display, based on the visual data and the SEI message, the portion of the visual data to a user of the user device. Other embodiments may be described or claimed.

Claims (32)

1. An electronic device that comprises:

first circuitry to decode, based on a first real-time transport protocol (RTP) stream for a video conference between two or more devices that include the electronic device, first visual data related to a plurality of images concurrently taken of a location;

second circuitry to decode, based on a session description protocol (SDP) offer received from a user equipment (UE) included in the two or more devices, an indication that the UE supports immersive viewing capability; and

third circuitry to transmit, via a second RTP stream based on the SDP offer, second visual data related to the first visual data, wherein the second visual data includes a supplemental enhancement information (SEI) message to be used to display at least a portion of the second visual data.

2. The electronic device of claim 1 , wherein the first RTP stream includes visual data related to two or more of the plurality of images.

3. The electronic device of claim 1 , wherein the first RTP stream include visual data related to an image of the plurality of images, and wherein the electronic device further comprises:

fourth circuitry to decode, based on a third RTP stream, third visual data related to a another image of the plurality of images; and

fifth circuitry to stitch together the first visual data and the third visual data.

4. The electronic device of claim 1 , wherein the SEI message is an equirectangular projection SEI message, a cubemap projection SEI message, a sphere rotation SEI message, or a region-wise packing SEI message.

5. The electronic device of claim 1 , further comprising sixth circuitry to decode, based on a RTP control protocol (RTCP) feedback message received from the UE, an indication of a desired viewing orientation of the first visual data.

6. The electronic device of claim 5 , wherein the second visual data is a portion of the first visual data that is based on the indication of the desired viewing orientation.

7. The electronic device of claim 1 , wherein the first, second, and third circuitry are circuitry of a processor.

8. A method comprising:

transmitting, from a user equipment (UE) to a remote terminal and for a video conference, a session description protocol (SDP) offer that includes an indication that the UE supports immersive video;

determining, by the UE based on a SDP answer received from the remote terminal, an indication that the remote terminal supports immersive video;

determining, based at least on the SDP offer and the SDP answer, a real-time transport protocol (RTP) media flow that includes visual data related to a plurality of images concurrently taken of a location and a supplemental enhancement information (SEI) message that is to be used to display at least a portion of the visual data; and

visually displaying, by the UE based on the visual data and the SEI message, the portion of the visual data on a display device.

9. The method of claim 8 , wherein the SDP offer includes an indication that the UE supports viewport-dependent processing.

10. The method of claim 8 , wherein the SDP answer includes an indication that the remote terminal supports viewport-dependent processing.

11. The method of claim 8 , further comprising transmitting, from the UE to the remote terminal in a RTP control protocol (RTCP) feedback message, an indication of a desired field of view of the visual data.

12. The method of claim 11 , wherein the visual data of the RTP media flow is based on the indication of the desired field of view.

13. The method of claim 8 , wherein the visual data is based on an Omnidirectional Media Format (OMAF) video profile.

14. An electronic device comprising:

first circuitry to transmit, to a remote terminal and for a video conference, a session description protocol (SDP) offer that includes an indication that the electronic device supports immersive video;

second circuitry to determine, based on a SDP answer received from the remote terminal, an indication that the remote terminal supports immersive video;

third circuitry to determine, based at least on the SDP offer and the SDP answer, a real-time transport protocol (RTP) media flow that includes visual data related to a plurality of images concurrently taken of a location and a supplemental enhancement information (SEI) message that is to be used to display at least a portion of the visual data; and

fourth circuitry to visually display, based on the visual data and the SEI message, the portion of the visual data on a display device.

15. The electronic device of claim 14 , wherein the SDP offer includes an indication that the electronic device supports viewport-dependent processing.

16. The electronic device of claim 14 , wherein the SDP answer includes an indication that the remote terminal supports viewport-dependent processing.

17. The electronic device of claim 14 , further comprising fifth circuitry to transmit, to the remote terminal in a RTP control protocol (RTCP) feedback message, an indication of a desired field of view of the visual data.

18. The electronic device of claim 17 , wherein the visual data of the RTP media flow is based on the indication of the desired field of view.

19. The electronic device of claim 14 , wherein the visual data is based on an Omnidirectional Media Format (OMAF) video profile.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 22, 2022
From: INTEL CORPORATION
To: APPLE INC.
Reel/Frame 061554/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2022
From: OYMAN, OZGUR
To: INTEL CORPORATION
Reel/Frame 060993/0481 →