IP Library › Granted Patent US 11,431,956
Granted Patent B2
US 11,431,956 · App. 17/098,388 · Granted Aug 30, 2022

Interactive overlay handling for immersive teleconferencing and telepresence for remote terminals

Inventors: Rohit Abhishek (Kansas City, MO); Iraj Sodagar (Palo Alto, CA)
Assignee: TENCENT AMERICA LLC
H04N13/156H04N5/23238H04N7/15H04N13/167H04N13/172H04N13/194
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,431,956
App. No.
17/098,388
Granted
Aug 30, 2022
Kind
B2
Abstract

A method, a non-transitory computer readable medium, and a computer system is provided for handling video streams. The method may include: receiving a 360° video stream, the 360° video stream including a multiple overlay parameter; when the multiple overlay parameter is a first value, determining that overlaying the 360° video stream with two or more overlays is permissible by the user terminal, and displaying, by the user terminal, the 360° video stream with the two or more overlays; and when the multiple overlay parameter is a second value, determining that overlaying the 360° video stream with two or more overlays is not permissible and displaying the 360° video stream with zero overlays or one overlay.

Claims (52)

1. A method comprising:

receiving, by a user terminal, from an electronic device or a server, a 360° video stream, wherein the 360° video stream includes a multiple overlay parameter that indicates whether overlaying the 360° video stream with two or more overlays is permissible by the user terminal;

displaying, by the user terminal, the 360° video stream;

when the multiple overlay parameter is a first value, determining, by the user terminal, that overlaying the 360° video stream with two or more overlays is permissible by the user terminal, and displaying, by the user terminal, the 360° video stream with the two or more overlays; and

when the multiple overlay parameter is a second value, determining, by the user terminal, that overlaying the 360° video stream with two or more overlays is not permissible by the user terminal, and displaying, by the user terminal, the 360° video stream with zero overlays or one overlay,

wherein the first value is different from the second value, and

wherein the overlay or overlays include one or more of: another video stream, or an image.

2. The method of claim 1 , wherein

the 360° video stream includes an overlap overlays parameter that indicates whether overlapping one of the two or more overlays with a first overlay is permissible, and

the method further comprises:

displaying, by the user terminal, the one of the two or more overlays superimposed to overlap the first overlay with the 360° video stream when the overlap overlays parameter indicates that overlapping one of the two or more overlays with the first overlay is permissible.

3. The method of claim 1 , wherein the 360° video stream includes a content-based attribute that allows use of the two or more overlays.

4. The method of claim 1 , wherein the 360° video stream includes a flag to signal when the user is allowed to share a sender's stream info with other senders.

5. The method of claim 1 , further comprising:

receiving, by the user terminal, a request from the electronic device or server, wherein the request is a request for one or more users in a teleconference to share media stream attributes; and

transmitting, by the user terminal to the electronic device or server, information regarding media stream attributes that the user terminal authorizes sharing of.

6. The method of claim 5 , wherein the information regarding media stream attributes that the user is willing to share includes an identifier (ID) corresponding to a sender ID.

7. The method of claim 6 , further comprising:

allocating, by the electronic device or server, a unique overlay ID by combining at least one sender ID with at least one overlay ID.

8. The method of claim 1 , further comprising: negotiating the multiple overlay parameter during an initial offer-answer negotiation or during a session.

9. The method of claim 1 , wherein the multiple overlay parameter is offered in a session description protocol (SDP) of the 360° video stream.

10. The method of claim 2 , further comprising: negotiating the overlap overlays parameter during an initial offer-answer negotiation or during a session.

11. The method of claim 2 , wherein the overlap overlays parameter is offered in a session description protocol (SDP) of the 360° video stream.

12. The method of claim 3 , further comprising: negotiating the content-based attribute during an initial offer-answer negotiation or during a session.

13. The method of claim 3 , wherein the content-based attribute is offered in a session description protocol (SDP) of the 360° video stream.

14. The method of claim 4 , further comprising: communicating, by the user terminal to the electronic device or server, whether another user is authorized to share stream information shared by the user terminal with other electronic devices.

15. The method of claim 5 , further comprising: communicating, by the user terminal to the electronic device or server, whether another user is authorized to share stream information shared by the user terminal with other electronic devices.

16. The method of claim 14 , further comprising: sharing, by the electronic device or server, at least one additional overlay besides any overlay or overlays produced by the electronic device or server with the other electronic devices, wherein the at least one additional overlay includes only overlays where a sender of the corresponding overlay video authorized sharing of the overlay.

17. A user terminal for video conferencing, the computer system comprising:

a communication interface;

a display;

one or more computer-readable non-transitory storage media configured to store computer program code; and

one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:

receiving code configured to cause the one or more computer processors to receive, via the communication interface, a 360° video stream, wherein the 360° video stream includes a multiple overlay parameter that indicates whether overlaying the 360° video stream with two or more overlays is permissible by the user terminal;

displaying code configured to cause the one or more computer processors to control the display to display the received 360° video stream;

multiple overlay authorized code configured to cause the one or more computer processors to: when the multiple overlay parameter is a first value, determine that overlaying the 360° video stream with two or more overlays is permissible by the user terminal, and control the display to display the 360° video stream with the two or more overlays; and

multiple overlay unauthorized code configured to cause the one or more computer processors to: when the multiple overlay parameter is a second value, determine that overlaying the 360° video stream with two or more overlays is not permissible by the user terminal, and control the display to display the 360° video stream with zero overlays or one overlay,

wherein the first value is different from the second value, and

wherein the overlay or overlays include one or more of: another video stream, or an image.

18. The computer system of claim 17 , wherein

the 360° video stream includes an overlap overlays parameter that indicates whether overlapping one of the two or more overlays with a first overlay is permissible, and

the computer program code further comprises: overlap overlays authorized code configured to cause the at one or more processors to: control the display to display the one of the two or more overlays superimposed to overlap the first overlay with the 360° video stream when the overlap overlays parameter indicates that overlapping one of the two or more overlays with the first overlay is permissible.

19. A non-transitory computer readable medium having stored thereon a computer program for handling a video call, the computer program configured to cause one or more computer processors to:

receive a 360° video stream, wherein the 360° video stream includes a multiple overlay parameter that indicates whether overlaying the 360° video stream with two or more overlays is permissible;

control a display to display the received 360° video stream;

when the multiple overlay parameter is a first value, determine that overlaying the 360° video stream with two or more overlays is permissible, and control the display to display the 360° video stream with the two or more overlays; and

when the multiple overlay parameter is a second value, determine that overlaying the 360° video stream with two or more overlays is not permissible, and control the display to display the 360° video stream with zero overlays or one overlay,

wherein the first value is different from the second value, and

wherein the overlay or overlays include one or more of: another video stream, or an image.

20. The non-transitory computer-readable medium of claim 19 , wherein

the 360° video stream includes an overlap overlays parameter that indicates whether overlapping one of the two or more overlays with a first overlay is permissible, and

wherein the computer program is further configured to cause the one or more computer processors to: control the display to display the one of the two or more overlays superimposed to overlap the first overlay with the 360° video stream when the overlap overlays parameter indicates that overlapping one of the two or more overlays with the first overlay is permissible.

Continuity (2)
Provisional Application 63064252 · Aug 11, 2020
Related Publication 20220053176A1 · Feb 17, 2022