System, apparatus, and method for sharing a screen having multiple visual components
Embodiments of the invention provide a screen sharing technology including a local computing device and a remote computing device. The local computing device compresses pixel data captured from a frame buffer of the local computing device into a video stream. When the local computing device determines that a video visual component is to be displayed, it obtains compressed data for the video visual component, location information and temporal information. The remote computing device decodes the compressed data and the video stream in different decoder and displays the video stream and compressed data on its monitor according to the temporal information and location information. As such, the embodiment described above leads to better coding efficiency, better coding quality, and lower bit rate.
1. A method, performed by a local computing device for sharing screen images with a remote computing device, the method comprising:
capturing pixel data of a first visual component of screen content of the local computing device from a frame buffer of the local computing device;
compressing the captured pixel data of the first visual component to form a first video stream;
determining that a second visual component of the screen content is to be displayed on the screen, wherein the second visual component of the screen content is decompressed from compressed data;
obtaining the compressed data for the second visual component;
obtaining location information indicating display location of the first and second visual components, temporal information for synchronizing display of the first and second visual components, and permission information for the first visual component, the second visual component, or both, wherein the permission information indicates whether the first visual component, the second visual component, or both are to be presented on the remote computing device; and
sending the compressed first video stream for the first visual component of the screen content, the compressed data for the second visual component, the location information, the temporal information, and the permission information to the remote computing device to share screen with the remote computing device.
2. The method according to claim 1 , wherein the step of determining includes detecting that:
a command is given by a user to open compressed data;
the compressed data is being decompressed by a processor of the local computing device; or
the compressed data is decompressed and saved in the frame buffer.
3. The method according to claim 1 , wherein the compressed data is a video file while the second visual component is a video visual component.
4. The method according to claim 1 , wherein the compressed data is an image file while the second visual component is an image visual component.
5. The method according to claim 1 , wherein the compressed data is sent as a video stream.
6. The method according to claim 1 , further comprising
capturing pixel data of a third visual component taken out from the first visual component from the frame buffer of the local computing device;
compressing the captured pixel data of the third visual component into a second video stream; and
sending the compressed second video stream to the remote computing device.
7. A computing device for sharing screen images with a remote computing device, the computing device comprising:
a monitor;
a frame buffer; and
a processor coupled with the frame buffer and the monitor, the processor configured to:
capture pixel data of a first visual component of screen content to be displayed on the monitor of the computing device from the frame buffer;
compress the captured pixel data of the first visual component to form a first video stream;
determine that a second visual component of the screen content is to be displayed on the monitor, wherein the second visual component of the screen content is decompressed from compressed data;
obtain the compressed data for the second visual component;
obtain location information indicating display location of the first and second visual components, temporal information for synchronizing display of the first and second visual component, and permission information for the first visual component, the second visual component, or both, wherein the permission information indicates whether the first visual component, the second visual component, or both are to be presented on the remote computing device; and
send the compressed first video stream for the first visual component of the screen content, the compressed data for the second visual component, the location information, the temporal information, and the permission information to the remote computing device to share screen with the remote computing device.
8. The computing device according to claim 7 , wherein the processor determine that a second visual component of the screen content is to be displayed on the monitor when the processor detects:
a command is given by a user to open compressed data;
the compressed data is being decompressed by a processor of the local computing device; or
the compressed data is decompressed and saved in the frame buffer.
9. The computing device according to claim 7 , wherein the compressed data is a video file while the second visual component is a video visual component.
10. The computing device according to claim 7 , wherein the compressed data is an image file while the second visual component is an image visual component.
11. The computing device according to claim 7 , wherein the compressed data is sent as a video stream.
12. The computing device according to claim 7 , the processor further configured to:
capture pixel data of a third visual component taken out from the first visual component from the frame buffer;
compress the captured pixel data of the third visual component into a second video stream; and
send the compressed second video stream to the remote computing device.