IP Library › Granted Patent US 11,412,012
Granted Patent B2
US 11,412,012 · App. 15/995,878 · Granted Aug 9, 2022

Method, apparatus, and computer-readable medium for desktop sharing over a web socket connection in a networked collaboration workspace

Inventors: Marco Valerio Masi (Lugano Paradiso, CH); Cristiano Fumagalli (Segrate, IT)
Assignee: RE MAGO HOLDING LTD
H04L65/4015H04L67/02H04L67/38
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,412,012
App. No.
15/995,878
Granted
Aug 9, 2022
Kind
B2
Abstract

A system, method and computer-readable medium for desktop sharing over a web socket connection in a networked collaboration workspace, including transmitting a representation of a collaboration workspace hosted on a server and accessible to participants over a web socket connection, receiving a request to share at least one portion of a local desktop of a local computing device within the collaboration workspace and a selection of an area within the representation of the collaboration workspace, generating a streaming object configured to output a video stream of the at least one portion of the local desktop of the local computing device, and transmitting one or more commands to the server over the web socket connection, the one or more commands comprising the streaming object and information corresponding to the selected area and being configured to cause the server to insert the streaming object into the collaboration workspace at the selected area.

Claims (53)

1. A method for desktop sharing over a web socket connection in a networked collaboration workspace, the method comprising:

transmitting, on a user interface of a local computing device, a representation of a collaboration workspace hosted on a server and accessible to a plurality of participants on a plurality of computing devices over a web socket connection;

receiving, by the local computing device, a request to share an entire local desktop of the local computing device within the collaboration workspace at a selected area within the representation of the collaboration workspace, wherein the selected area is smaller than the representation of the collaboration workspace;

generating, by the local computing device, a streaming object configured to output a video stream of the entire local desktop of the local computing device by identifying a source identifier corresponding to the entire local desktop, retrieving a media stream corresponding to the identified source identifier, and instantiating a media stream object linked to the media stream and having spatial attributes corresponding to the selected area within the representation of the collaboration workspace, wherein the streaming object comprises a media stream configured to duplicate the entire local desktop; and

transmitting, by the local computing device, one or more commands to the server over the web socket connection, the one or more commands comprising the streaming object and information corresponding to the selected area and being configured to cause the server to embed the streaming object into the collaboration workspace, wherein a spatial location of the streaming object within the collaboration workspace corresponds to the selected area and is independent of a position of the entire local desktop relative to the collaboration workspace and wherein the user interface renders both the entire local desktop and the streaming object duplicating the entire local desktop within the collaboration workspace.

2. The method of claim 1 , wherein user input at a position within the streaming object within the collaboration workspace results in a corresponding user input being applied at a corresponding position within the entire local desktop.

3. The method of claim 2 , wherein the user input comprises input from a remote user of a remote computing device in the plurality of computing devices.

4. The method of claim 2 , wherein the user input comprises input from a local user of the local computing device.

5. The method of claim 1 , wherein the inserted streaming object is configured to receive the video stream of the entire local desktop of the local computing device and transmit the video stream of the entire local desktop of the local computing device to the plurality of computing devices over the web socket connection.

6. The method of claim 1 , further comprising:

transmitting, on the user interface of the local computing device, the inserted streaming object within the representation of the collaboration workspace, the inserted streaming object being associated with a network address of a source of the video stream; and

detecting, by a transparent layer executing on the local computing device, a user input associated with the inserted streaming object, the user input corresponding to a location within the entire local desktop, wherein the transparent layer comprises an application programming interface (API) configured to interface with one or more of: an operating system or one or more applications configured to execute on the operating system;

determining, by the transparent layer executing on the local computing device, that the network address associated with the inserted streaming object corresponds to a network address of the local computing device; and

transmitting, by the transparent layer executing on the local computing device, one or more second commands to one or more of: the operating system or one or more applications configured to execute on the operating system, the one or more second commands being configured to perform the user input at the location within the entire local desktop.

7. The method of claim 1 , further comprising:

transmitting, on a remote user interface of a remote computing device in the plurality of computing devices, the inserted streaming object within a remote representation of the collaboration workspace, the inserted streaming object being associated with a network address of a source of the video stream; and

detecting, by a remote transparent layer executing on the remote computing device, a remote user input associated with the inserted streaming object, the remote user input corresponding to a location within the entire local desktop, wherein the transparent layer comprises an application programming interface (API) configured to interface with one or more of: an operating system or one or more applications configured to execute on the operating system;

determining, by the remote transparent layer executing on the remote computing device, that the network address associated with the inserted streaming object does not correspond to a network address of the remote computing device; and

transmitting, by the remote transparent layer executing on the remote computing device, one or more second commands to the local computing device over the web socket connection, the one or more second commands being configured to cause a local transparent layer executing on the local computing device to cause performance of the user input at the location within the entire local desktop.

8. A local computing device for desktop sharing over a web socket connection in a networked collaboration workspace, the local computing device comprising:

one or more processors; and

one or more memories operatively coupled to at least one of the one or more processors and having instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to:

transmit, on a user interface of a local computing device, a representation of a collaboration workspace hosted on a server and accessible to a plurality of participants on a plurality of computing devices over a web socket connection;

receive, by the local computing device, a request to share an entire local desktop of the local computing device within the collaboration workspace at a selected area within the representation of the collaboration workspace, wherein the selected area is smaller than the representation of the collaboration workspace; generate, by the local computing device, a streaming object configured to output a video stream of the the entire local desktop of the local computing device by identifying a source identifier corresponding to the entire local desktop, retrieving a media stream corresponding to the identified source identifier, and instantiating a media stream object linked to the media stream and having spatial attributes corresponding to the selected area within the representation of the collaboration workspace, wherein the streaming object comprises a media stream configured to duplicate the entire local desktop; and

transmit, by the local computing device, one or more commands to the server over the web socket connection, the one or more commands comprising the streaming object and information corresponding to the selected area and being configured to cause the server to embed the streaming object into the collaboration workspace, wherein a spatial location of the streaming object within the collaboration workspace corresponds to the selected area and is independent of a position of the entire local desktop relative to the collaboration workspace and wherein the user interface renders both the entire local desktop and the streaming object duplicating the entire local desktop within the collaboration workspace.

9. The local computing device of claim 8 , wherein user input at a position within the streaming object within the collaboration workspace results in a corresponding user input being applied at a corresponding position within the entire local desktop.

10. The local computing device of claim 9 , wherein the user input comprises input from a remote user of a remote computing device in the plurality of computing devices.

11. The local computing device of claim 9 , wherein the user input comprises input from a local user of the local computing device.

12. The local computing device of claim 8 , wherein the inserted streaming object is configured to receive the video stream of the entire local desktop of the local computing device and transmit the video stream of the entire local desktop of the local computing device to the plurality of computing devices over the web socket connection.

13. The local computing device of claim 8 , wherein at least one of the one or more memories has further instructions stored thereon that, when executed by at least one of the one or more processors, cause at least one of the one or more processors to:

transmit, on the user interface of the local computing device, the inserted streaming object within the representation of the collaboration workspace, the inserted streaming object being associated with a network address of a source of the video stream; and

detect, by a transparent layer executing on the local computing device, a user input associated with the inserted streaming object, the user input corresponding to a location within the entire local desktop, wherein the transparent layer comprises an application programming interface (API) configured to interface with one or more of: an operating system or one or more applications configured to execute on the operating system;

determine, by the transparent layer executing on the local computing device, that the network address associated with the inserted streaming object corresponds to a network address of the local computing device; and

transmit, by the transparent layer executing on the local computing device, one or more second commands to one or more of: the operating system or one or more applications configured to execute on the operating system, the one or more second commands being configured to perform the user input at the location within the entire local desktop.

14. At least one non-transitory computer-readable medium storing computer-readable instructions that, when executed by a local computing device, cause the local computing device to:

transmit, on a user interface of a local computing device, a representation of a collaboration workspace hosted on a server and accessible to a plurality of participants on a plurality of computing devices over a web socket connection;

receive, by the local computing device, a request to share an entire local desktop of the local computing device within the collaboration workspace at a selected area within the representation of the collaboration workspace, wherein the selected area is smaller than the representation of the collaboration workspace;

generate, by the local computing device, a streaming object configured to output a video stream of the entire local desktop of the local computing device by identifying a source identifier corresponding to the entire local desktop, retrieving a media stream corresponding to the identified source identifier, and instantiating a media stream object linked to the media stream and having spatial attributes corresponding to the selected area within the representation of the collaboration workspace, wherein the streaming object comprises a media stream configured to duplicate the entire local desktop; and

transmit, by the local computing device, one or more commands to the server over the web socket connection, the one or more commands comprising the streaming object and information corresponding to the selected area and being configured to cause the server to embed the streaming object into the collaboration workspace, wherein a spatial location of the streaming object within the collaboration workspace corresponds to the selected area and is independent of a position of the entire local desktop relative to the collaboration workspace and wherein the user interface renders both the entire local desktop and the streaming object duplicating the entire local desktop within the collaboration workspace.

15. The at least one non-transitory computer-readable medium of claim 14 , wherein user input at a position within the streaming object within the collaboration workspace results in a corresponding user input being applied at a corresponding position within the entire local desktop.

16. The at least one non-transitory computer-readable medium of claim 15 , wherein the user input comprises input from a remote user of a remote computing device in the plurality of computing devices.

17. The at least one non-transitory computer-readable medium of claim 15 , wherein the user input comprises input from a local user of the local computing device.

18. The at least one non-transitory computer-readable medium of claim 14 , wherein the inserted streaming object is configured to receive the video stream of the entire local desktop of the local computing device and transmit the video stream of the entire local desktop of the local computing device to the plurality of computing devices over the web socket connection.

19. The at least one non-transitory computer-readable medium of claim 14 , further storing computer-readable instructions that, when executed by the local computing device, cause the local computing device to:

transmit, on the user interface of the local computing device, the inserted streaming object within the representation of the collaboration workspace, the inserted streaming object being associated with a network address of a source of the video stream; and

detect, by a transparent layer executing on the local computing device, a user input associated with the inserted streaming object, the user input corresponding to a location within the entire local desktop, wherein the transparent layer comprises an application programming interface (API) configured to interface with one or more of: an operating system or one or more applications configured to execute on the operating system;

determine, by the transparent layer executing on the local computing device, that the network address associated with the inserted streaming object corresponds to a network address of the local computing device; and

transmit, by the transparent layer executing on the local computing device, one or more second commands to one or more of: the operating system or one or more applications configured to execute on the operating system, the one or more second commands being configured to perform the user input at the location within the entire local desktop.

20. The at least one non-transitory computer-readable medium of claim 14 , further storing computer-readable instructions that, when executed by a remote computing device in the plurality of computing devices, cause the remote computing device to:

transmit, on a remote user interface of the remote computing device, the inserted streaming object within a remote representation of the collaboration workspace, the inserted streaming object being associated with a network address of a source of the video stream; and

detect, by a remote transparent layer executing on the remote computing device, a remote user input associated with the inserted streaming object, the remote user input corresponding to a location within the entire local desktop, wherein the transparent layer comprises an application programming interface (API) configured to interface with one or more of: an operating system or one or more applications configured to execute on the operating system;

determine, by the remote transparent layer executing on the remote computing device, that the network address associated with the inserted streaming object does not correspond to a network address of the remote computing device; and

transmit, by the remote transparent layer executing on the remote computing device, one or more second commands to the local computing device over the web socket connection, the one or more second commands being configured to cause a local transparent layer executing on the local computing device to cause performance of the user input at the location within the entire local desktop.

Assignments (2)
CHANGE OF NAME Recorded Jul 1, 2020
From: RE MAGO HOLDING LTD
To: RE MAGO LTD
Reel/Frame 053097/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2019
From: MASI, MARCO VALERIO; FUMAGALLI, CRISTIANO
To: RE MAGO HOLDING LTD
Reel/Frame 048311/0100 →
Continuity (2)
Continuation In Part 15685533 · Aug 24, 2017
Related Publication 20190068661A1 · Feb 28, 2019