Operating environment with gestural control and multiple client devices, displays, and users
Embodiments described herein includes a system comprising a processor coupled to display devices, sensors, remote client devices, and computer applications. The computer applications orchestrate content of the remote client devices simultaneously across the display devices and the remote client devices, and allow simultaneous control of the display devices. The simultaneous control includes automatically detecting a gesture of at least one object from gesture data received via the sensors. The detecting comprises identifying the gesture using only the gesture data. The computer applications translate the gesture to a gesture signal, and control the display devices in response to the gesture signal.
1. A system comprising:
a display processor that is communicatively coupled to a plurality of display devices;
a first remote client device that is communicatively coupled to the display processor, wherein the first remote client device includes a first input device and a first client application; and
a second remote client device that is communicatively coupled to the display processor, wherein the second remote client device includes a second input device and a second client application,
wherein the first remote client device is constructed to provide first video data to the display processor, wherein the first video data includes at least display output of the first client application,
wherein the second remote client device is constructed to provide second video data to the display processor, wherein the second video data includes at least display output of the second client application,
wherein the first remote client device and the second remote client device are remote to the display processor and the plurality of display devices, and
wherein the display processor is constructed to:
integrate the first video data and the second video data in a first application session of the display processor;
control the plurality of display devices to display rendered data of the first application session;
responsive to receiving first pointer data from the first input device via the first remote client device, the first pointer data corresponding to the displayed rendered data that corresponds to the second client application, controlling the second client application based on the received first pointer data; and
responsive to receiving second pointer data from the second input device via the second remote client device, the second pointer data corresponding to the displayed rendered data that corresponds to the first client application, controlling the first client application based on the received second pointer data.
2. The system of claim 1 , wherein the display processor is communicatively coupled to a plurality of sensors.
3. The system of claim 1 , wherein each of the first remote client device and the second remote client device provide an element of a workflow of the first application session.
4. The system of claim 3 , wherein the display processor is constructed to integrate pixels of the element of the first remote client device and the element of the second remote client device into the workflow of the first application session.
5. The system of claim 4 , wherein the display processor is constructed to provide a collaborative interface for a plurality of remote client devices communicatively coupled to the display processor, wherein the collaborative interface is constructed to provide any remote client device of the plurality of remote client devices with direct interaction with applications running on any other remote client device of the plurality of remote client devices, wherein direct interaction with an application running on a remote client device comprises control of an element of the remote client device that is included in the workflow of the first application session.
6. A method comprising: at a display processor that is communicatively coupled to a plurality of display devices:
accessing first video data from a first remote client device that is communicatively coupled to the display processor, wherein the first video data includes at least display output of a first client application of the first remote client device;
integrating the accessed first video data in a first application session of the display processor;
controlling the plurality of display devices to display rendered data of the first application session;
receiving pointer data from an input device of a second remote client device, wherein the received pointer data corresponds to the displayed rendered data that corresponds to the first client application; and
controlling the first client application of the first remote client device based on the received pointer data,
wherein the first remote client device and the second remote client device are remote to the display processor and the plurality of display devices, and
wherein the first remote client device and the second remote client device are communicatively coupled to the display processor.
7. The method of claim 6 , wherein the second remote client device is constructed to:
generate a first un-typed data structure that includes the pointer data of the input device of the second remote client device; and
provide the first un-typed data structure to the display processor,
wherein the pointer data has a data type that corresponds to a second client application of the second remote client device, and
wherein the first un-typed data structure that includes the pointer data has a format that is application-independent.
8. The method of claim 7 , wherein the display processor controls the plurality of display devices to display rendered data of the first application session by using a first application of the display processor, wherein the first application of the display processor corresponds to a type that is different from a type of the second client application of the second remote client device.
9. The method of claim 8 , further comprising: at the display processor:
receiving pointer data from an input device of a third remote client device, wherein the received pointer data from the input device of the third remote client device corresponds to the displayed rendered data that corresponds to the first client application; and
controlling the first client application of the first remote client device based on the received pointer data of the input device of the third remote client device,
wherein the first remote client device, the second remote client device, and the third remote client device are remote to the display processor and the plurality of display devices, and
wherein the third remote client device is communicatively coupled to the display processor.
10. The method of claim 8 , further comprising: at the display processor:
accessing second video data from a third remote client device that is communicatively coupled to the display processor, wherein the second video data includes at least display output of a third client application of the third remote client device;
integrating the accessed first video data and the accessed second video data in the first application session of the display processor;
controlling the plurality of display devices to display rendered data of the first application session;
receiving pointer data from the input device of the second remote client device, wherein the received pointer data from the input device of the second remote client device corresponds to the displayed rendered data that corresponds to the third client application; and
controlling the third client application of the third remote client device based on the received pointer data from the input device of the second remote client device,
wherein the first remote client device, the second remote client device, and the third remote client device are remote to the display processor and the plurality of display devices, and
wherein the third remote client device is communicatively coupled to the display processor.
11. The method of claim 8 , further comprising: at the display processor:
accessing second video data from the second remote client device, wherein the second video data includes at least display output of the second client application of the second remote client device;
integrating the accessed first video data and the accessed second video data in the first application session of the display processor;
controlling the plurality of display devices to display rendered data of the first application session;
receiving pointer data from an input device of the first remote client device, wherein the received pointer data from the input device of the first remote client device corresponds to the displayed rendered data that corresponds to the second client application; and
controlling the second client application of the second remote client device based on the received pointer data from the input device of the first remote client device.
12. The method of claim 11 , wherein the type of the first application of the display system processor is different from a type of the first client application of the first remote client device.
13. The method of claim 11 , further comprising:
the display processor authenticating the first remote client device for access to the first application session; and
the display processor authenticating the second remote client device for access to the first application session.
14. The method of claim 6 , wherein the first application session is an application session of a collaborative application.
15. The method of claim 6 , wherein the display processor is communicatively coupled to a plurality of sensors.
16. The method of claim 6 , wherein each of the first remote client device and the second remote client device provide an element of a workflow of the first application session.
17. The method of claim 16 , wherein the display processor integrates pixels of the element of the first remote client device and the element of the second remote client device into the workflow of the first application session.
18. The method of claim 17 , wherein the display processor provides a collaborative interface for a plurality of remote client devices communicatively coupled to the display processor, wherein the collaborative interface provides any remote client device of the plurality of remote client devices with direct interaction with applications running on any other remote client device of the plurality of remote client devices, wherein direct interaction with an application running on a remote client device comprises control of an element of the remote client device that is included in the workflow of the first application session.