Granted Patent B2

US 12,432,265 · App. 17/060,516 · Granted Sep 30, 2025

Inventor: Cevat Yerli (Frankfurt am Main, DE)

Assignee: TMRW Group IP

H04L65/4053G06F3/011G06T19/003H04L65/4015G09B5/12

Abstract

Claims (60)

1. A system for virtually broadcasting from within a virtual environment, comprising:

a server computer system comprising one or more server computers, each server computer comprising at least one processor and memory, the server computer system including:

data and instructions configured to manage a data exchange between client devices; and

at least one virtual environment comprising a virtual broadcasting camera positioned within the at least one virtual environment and configured to capture incoming multimedia streams from a virtual multimedia streams source from within the at least one virtual environment, wherein the incoming multimedia streams comprise spatial, three dimensional orientation data,

wherein the virtual broadcasting camera is managed through at least one client device accessing the at least one virtual environment and is configured to allow the at least one client device to steer the point of view of the virtual broadcasting camera within the at least one virtual environment,

wherein the server computer system is configured to receive live feed data captured by at least one camera from receiving client devices and broadcast the multimedia streams captured by the virtual broadcasting camera to the receiving client devices based on the data exchange management,

wherein the broadcasted multimedia streams are configured to be displayed to corresponding user graphical representations generated from the live feed data from the receiving client devices,

wherein the point of view of the virtual broadcasting camera is different from the point of view of any of the user graphical representations,

wherein the data exchange between client devices comprises analyzing the incoming multimedia streams and forwarding outbound multimedia streams based on the analyzing of the incoming multimedia streams,

wherein a first user graphical representation is assigned a higher priority score than a second user graphical representation based on a spatial, three dimensional orientation of the first user graphical representation with respect to an incoming multimedia stream's source, and

wherein the data exchange optimizes the outbound media streams, based on the analyzing of the incoming multimedia streams, by forwarding higher resolution images to the first user graphical representation, based on the higher priority score, than to the second user graphical representation.

2. The system of claim 1 , wherein the server computer system utilizes a routing topology comprising a Selective Forwarding Unit (SFU), a Traversal Using Relay NAT (TURN), or a Spatially analyzed Media Server (SAMS).

3. The system of claim 1 , wherein the server computer system uses a media processing topology to process the outbound multimedia streams for viewing by the user graphical representations within the at least one virtual environment through the receiving client devices.

4. The system of claim 1 , wherein the server computer system uses a forwarding server topology comprising one or more Multipoint Control Units (MCUs), cloud media mixers, or cloud 3D renderers.

5. The system of claim 1 , wherein the forwarding of the outbound multimedia streams comprises modifying, upscaling or downscaling the multimedia streams for temporal, spatial, quality or color features.

6. The system of claim 1 ,

wherein the at least one virtual environment comprises a plurality of virtual broadcasting cameras, each virtual broadcasting camera providing multimedia streams from a corresponding point of view within the at least one virtual environment,

wherein the point of view of each of the plurality of virtual broadcasting cameras is different from the point of view of any of the user graphical representations, and

wherein users of the receiving client devices can select between the points of view of any of the plurality of virtual broadcasting cameras.

7. The system of claim 1 , wherein the at least one virtual environment is hosted by at least one dedicated server computer connected via a network to at least one media server computer.

8. The system of claim 1 , wherein the at least one virtual environment is hosted in a peer-to-peer infrastructure and is relayed through at least one media server computer.

9. The system of claim 1 , wherein the assigned priority score of the first user graphical representation and the assigned priority score of the second user graphical representation are further based on distance relationship data of the first user graphical representation with respect to the multimedia stream source and the second user graphical representation with respect to the multimedia stream source, respectively.

10. A method for virtually broadcasting from within a virtual environment, comprising:

capturing incoming multimedia streams from a virtual multimedia streams source with a virtual broadcasting camera positioned within at least one virtual environment connected to at least one media server, wherein the incoming multimedia streams comprise spatial, three dimensional orientation data,

wherein the virtual broadcasting camera is managed through at least one client device accessing the at least one virtual environment and is configured to allow the at least one client device to steer the point of view of the virtual broadcasting camera within the at least one virtual environment;

sending the incoming multimedia streams captured by the virtual broadcasting camera to the at least one media server for broadcasting corresponding multimedia streams to receiving client devices;

obtaining live feed data from the receiving client devices;

performing data exchange management comprising analyzing the incoming multimedia streams from within the at least one virtual environment and the live feed data and assessing the forwarding of outbound multimedia streams;

broadcasting the outbound multimedia streams to the receiving client devices based on the data exchange management,

wherein the outbound multimedia streams are displayed to user graphical representations in the at least one virtual environment,

wherein the point of view of the virtual broadcasting camera is different from the point of view of any of the user graphical representations,

wherein a first user graphical representation is assigned a higher priority score than a second user graphical representation based on a spatial, three dimensional orientation of the first user graphical representation with respect to an incoming multimedia stream's source; and

optimizing, based on the analyzing of the incoming multimedia streams, the outbound media streams by forwarding higher resolution images to the first user graphical representation, based on the higher priority score, than to the second user graphical representation.

11. The method of claim 10 , wherein the forwarding of the outbound multimedia streams comprises modifying, upscaling or downscaling the multimedia streams for temporal, spatial, quality or color features.

12. The method of claim 10 ,

wherein the at least one virtual environment comprises a plurality of virtual broadcasting cameras, each virtual broadcasting camera providing multimedia streams from a corresponding point of view within the at least one virtual environment,

wherein the point of view of each of the plurality of virtual broadcasting cameras is different from the point of view of any of the user graphical representations, and

wherein users of the receiving client devices can select between the points of view of any of the plurality of virtual broadcasting cameras.

13. The method of claim 10 , wherein the at least one virtual environment is hosted by at least one dedicated server computer connected via a network to at least one media server computer.

14. The method of claim 10 , wherein the at least one virtual environment is hosted in a peer-to-peer infrastructure and is relayed through at least one media server computer.

15. The method of claim 10 , wherein the assigned priority score of the first user graphical representation and the assigned priority score of the second user graphical representation are further based on distance relationship data of the first user graphical representation with respect to the multimedia stream source and the second user graphical representation with respect to the multimedia stream source, respectively.

16. A non-transitory computer readable medium having stored thereon instructions configured to cause at least one server computer comprising a processor and memory to perform steps comprising:

providing data exchange management data and instructions and at least one virtual environment;

obtaining a live data feed comprising incoming multimedia streams from a virtual multimedia streams source, wherein the incoming multimedia streams comprise spatial, three dimensional orientation data,

wherein the incoming multimedia streams are captured by a virtual broadcasting camera positioned within the at least one virtual environment connected to at least one media server,

wherein the virtual broadcasting camera is managed through at least one client device accessing the at least one virtual environment and is configured to allow the at least one client device to steer the point of view of the virtual broadcasting camera within the at least one virtual environment;

performing data exchange management comprising analyzing the incoming multimedia streams and assessing the forwarding of outbound multimedia streams corresponding to the multimedia streams captured by the virtual broadcasting camera;

broadcasting the outbound multimedia streams to receiving client devices within the at least one virtual environment based on the data exchange management,

wherein the outbound multimedia streams are displayed to user graphical representations in the at least one virtual environment,

wherein the point of view of the virtual broadcasting camera is different from the point of view of any of the user graphical representations,

wherein a first user graphical representation is assigned a higher priority score than a second user graphical representation based on a spatial, three dimensional orientation of the first user graphical representation with respect to an incoming multimedia stream's source; and

optimizing, based on the analyzing of the incoming multimedia streams, the outbound media streams by forwarding higher resolution images to the first user graphical representation, based on the higher priority score, than to the second user graphical representation.

17. The non-transitory computer readable medium of claim 16 , wherein the forwarding of the outbound multimedia streams comprises modifying, upscaling or downscaling the multimedia streams for temporal, spatial, quality or color features.

18. The non-transitory computer readable medium of claim 16 ,

wherein the at least one virtual environment comprises a plurality of virtual broadcasting cameras, each virtual broadcasting camera providing multimedia streams from a corresponding point of view within the at least one virtual environment,

wherein the point of view of each of the plurality of virtual broadcasting cameras is different from the point of view of any of the user graphical representations, and

wherein users of the receiving client devices can select between the points of view of any of the plurality of virtual broadcasting cameras.

19. The non-transitory computer readable medium of claim 16 , wherein the at least one virtual environment is hosted by at least one dedicated server computer connected via a network to at least one media server computer.

20. The non-transitory computer readable medium of claim 16 , wherein the at least one virtual environment is hosted in a peer-to-peer infrastructure and is relayed through at least one media server computer.

21. The non-transitory computer readable medium of claim 16 , wherein the assigned priority score of the first user graphical representation and the assigned priority score of the second user graphical representation are further based on distance relationship data of the first user graphical representation with respect to the multimedia stream source and the second user graphical representation with respect to the multimedia stream source, respectively.

Assignments (1)

Continuity (2)

References Cited (272)