Apparatus and method for client-side compositing of video streams

1. A method for client-side compositing of media streams by a video display device, the method comprising:

receiving, by the video display device, a multiplexed data stream from a remote video server, the multiplexed data stream being a signal that includes data representing substreams, the substreams including a descriptor substream, at least one media substream, and a compositing-instruction substream;

demultiplexing, by the video display device, the multiplexed data stream into the substreams according to the descriptor substream; and

displaying, by the video display device, the at least one media substream on a display screen according to the compositing-instruction substream, the compositing-instruction substream including instructions on a composition of the at least one media substream, wherein

the displaying step displays the at least one media substream as one of an on-screen display and a picture-in-picture in an area of the display screen indicated by the compositing-instruction substream.

2. The method of claim 1 , further comprising:

receiving, by the video display device, a first instruction command from a user, the first instruction command changing a composition of the at least one media substream to a composition different from the composition indicated by the compositing-instruction substream, wherein the displaying step displays the at least one media substream according to the changed composition.

3. The method of claim 2 , wherein the first instruction command indicates a removal of a media substream included in the multiplexed data stream.

4. The method of claim 2 , wherein the first instruction command indicates a change of at least one of a display size, display color, display shape, or display location of the at least one media substream.

5. The method of claim 2 , wherein the first instruction command indicates an addition of a media substream to the multiplexed data stream.

6. A method for client-side compositing of media streams by a video display device, the method comprising:

receiving, by the video display device, a multiplexed data stream from a remote video server, the multiplexed data stream being a signal that includes data representing substreams, the substreams including a descriptor substream, at least one media substream, and a compositing-instruction substream;

demultiplexing, by the video display device, the multiplexed data stream into the substreams according to the descriptor substream; and

displaying, by the video display device, the at least one media substream on a display screen according to the compositing-instruction substream, the compositing-instruction substream including instructions on a composition of the at least one media substream, wherein

the multiplexed data stream includes a plurality of media substreams, each of the plurality of media substreams includes same video data according to different display aspect ratios, the display step displays at least one of the plurality of media substreams that has a display aspect ratio that is closest to the video display device based on the compositing-instruction substream.

7. A method for client-side compositing of media streams by a video server, the method comprising:

generating, by the video server, a compositing-instruction substream, the compositing-instruction substream including one or more of instructions indicating an aspect ratio of at least one media substream and an area of a display screen to place an on screen display included in an on screen display substream, the video server being a server-side video server;

multiplexing, by the video server, a descriptor substream, the at least one media substream, and the compositing-instruction substream, the at least one media substream containing one or more of a plurality of media substreams which represent a same video according to different aspect ratios and both a video substream and the on screen display substream; and

transmitting, by the video server, the multiplexed data stream to be displayed on the display screen according to the compositing-instruction substream.

8. The method of claim 7 , wherein the compositing-instruction substream indicating the area of the display screen to display the at least one media substream is an area to display one of the on-screen display and a picture-in-picture.

9. The method of claim 7 , further comprising:

receiving, by the video server, a first instruction command from the video display device, the first instruction command changing a composition of the at least one media substream to a composition different from the composition indicated by the compositing-instruction substream, wherein the generating step generates an updated compositing-instruction substream indicating the changed composition.

10. The method of claim 9 , wherein the first instruction command indicates a removal of a media substream included in the transmitted multiplexed data stream.

11. The method of claim 9 , wherein the first instruction command indicates a change of at least one of a display size, display color, display shape, or display location of the at least one media substream included in the multiplexed data stream.

12. The method of claim 9 , wherein the first instruction command indicates an addition of a media substream to the multiplexed data stream.

13. The method of claim 7 , wherein the compositing-instruction substream includes instructions for a video display device to select one of the plurality of media substreams that has a display aspect ratio that is closest to the video display device.

14. A video display device for client-side compositing of media streams, the video display device comprising:

a demultiplexer configured to demultiplex a multiplexed data stream into substreams, the substreams including a descriptor sub-stream, at least one media substream, and a compositing-instruction substream, the compositing-instruction substream including instructions on a composition of the at least one media substream; and

a controller configured to display the at least one media substream on a display screen according to the compositing-instruction substream, wherein

the controller displays the at least one media substream as an on-screen display or a picture-in-picture in an area of the display screen indicated by the compositing-instruction substream.

15. The video display device of claim 14 , wherein the video display device is configured to receive a first instruction command from a user, the first instruction command changing a composition of the at least one media substream to a composition different from the composition indicated by the compositing-instruction substream, wherein the controller displays the at least one media substream according to the changed composition.

16. The video display device of claim 15 , wherein the first instruction command indicates a removal of a media substream included in the multiplexed data stream.

17. The video display device of claim 15 , wherein the first instruction command indicates a change of at least one of a display size, display color, display shape, or display location of the at least one media substream.

18. The video display device of claim 15 , wherein the first instruction command indicates an addition of a media substream to the multiplexed data stream.

19. A video display device for client-side compositing of media streams, the video display device comprising:

a demultiplexer configured to demultiplex a multiplexed data stream into substreams, the substreams including a descriptor sub-stream, at least one media substream, and a compositing-instruction substream, the compositing-instruction substream including instructions on a composition of the at least one media substream; and

a controller configured to display the at least one media substream on a display screen according to the compositing-instruction substream, wherein

the multiplexed data stream includes a plurality of media substreams, each of the plurality of media substreams includes same video data according to different display aspect ratios, and the controller displays at least one of the plurality of media substreams that has a display aspect ratio that is closest to the video display device based on the compositing-instruction substream.

20. A video server for client-side compositing of media streams, the video server comprising:

a controller configured to generate a compositing-instruction substream, the compositing-instruction substream including one or more of instructions indicating an aspect ratio of at least one media substream and an area of a display screen to place an on screen display substream, the video server being a server-side video server;

a multiplexer configured to multiplex a descriptor substream, the at least one media substream, and the compositing-instruction substream, the at least one media substream containing one or more of a plurality of media substreams which represent a same video according to different aspect ratios and both a video substream and the on screen display substream; and

the video server configured to transmit the multiplexed data stream to be displayed on a display screen according to the compositing-instruction substream.

21. The video server of claim 20 , wherein the compositing-instruction substream indicating the area of the display screen to display the at least one media substream is an area to display one of the on-screen display and a picture-in-picture.

22. The video server of claim 20 , wherein the video server is configured to receive a first instruction command from the video display device, the first instruction command changing a composition of the at least one media substream to a composition different from the composition indicated by the compositing-instruction substream, and the controller generates an updated compositing-instruction substream indicating the changed composition.

23. The video server of claim 22 , wherein the first instruction command indicates a removal of a media substream included in the transmitted multiplexed data stream.

24. The video server of claim 22 , wherein the first instruction command indicates a change of at least one of a display size, display color, display shape, and display location of the at least one media substream included in the multiplexed data stream.

25. The video server of claim 22 , wherein the first instruction command indicates an addition of a media substream to the multiplexed data stream.

26. The video server of claim 20 , wherein the compositing-instruction substream includes instructions for a video display device to select one of the plurality of media substreams that has a display aspect ratio that is closest to the video display device.