Managing connections of input and output devices in a physical room
A device management server computer (“server”) is programmed to manage a plurality of input devices and output devices in a physical room. The server is programmed to analyze media data capturing actions performed by a user in real time as a participant in the physical room, determine how the user would like to connect at least one of the input devices and one of the output devices from the analysis, and enable the connection. The sever is programmed to interpret the actions and derive commands for connecting two or more devices based on predetermined data regarding the input devices and output devices and rules for referring to and connecting these devices.
1. A computer-implemented method of managing multiple devices in a physical room with multimedia communications in real time, comprising:
receiving and storing, by a processor, input data generated in real time by one of a plurality of input devices in a physical room,
the input data including digital image data or digital sound data;
identifying, by the processor, in real time from the input data a user performing an action with respect to at least one of the plurality of input devices or at least one of a plurality of output devices in the physical room, comprising:
extracting a reference to an input or output (I/O) device;
computing a first confidence score for determining the reference;
when the first confidence score is less than a first threshold, sending a first request for user input to improve the first confidence score;
when the first confidence score is no less than the first threshold, extracting a handle for a specific I/O device and computing a second confidence score for determining the handle;
when the second confidence score is less than a second threshold, sending a second request for user input to improve the second confidence score; and
when the second confidence score is no less than the second threshold, determining that the user action is performed with respect to the specific I/O device;
deriving in real time from the action a request from the user for a direct or network connection between the at least one input device and a set of the plurality of output devices or a direct or network connection between the at least one output device and a set of the plurality of input devices, comprising:
determining whether the action includes an indication to connect (I/O) devices; and
in response to determining that the action includes an indication to connect I/O devices:
determining that the action includes a reference to only to a first I/O device of the plurality of input devices and the plurality of output devices; and
selecting a second I/O device of the plurality of input devices and the plurality of output devices to connect with the first I/O device for the request;
automatically establishing the direct or network connection in real time.
2. The computer-implemented method of claim 1 , further comprising obtaining device data regarding a new input or output (I/O) device in the physical room, the device data including a handle for the new I/O device that includes a spoken word or a gesture different from an identifier provided by a vendor of the new I/O device.
3. The computer-implemented method of claim 2 , the device data further including a default compatible device, a set of collaborating devices, or a hierarchical rank.
4. The computer-implemented method of claim 1 , the identifying comprising:
extracting from the input data a mechanism for a relative reference to an input or output (I/O) device;
mapping the mechanism to the at least one input device or the at least one output device.
5. The computer-implemented method of claim 4 , the mechanism including touching the I/O device, looking or pointing at the I/O device, looking or pointing in a direction leading to the I/O device, describing a positional relationship of the I/O device to a portion of the physical room, or describing an attribute other than an identifier of the I/O device.
6. The computer-implemented method of claim 1 , the handle being a shorthand name for referring to the specific I/O device.
7. The computer-implemented method of claim 1 , the deriving further comprising, in response to determining that the action does not include an indication to connect I/O devices, determining whether the action includes references to two I/O devices that can be connected only with each other.
8. The computer-implemented method of claim 1 , the deriving further comprising, in response to determining that the action includes references to more than two I/O devices that can be connected with one another, selecting at least one direct or network connection each between a pair of the more than two I/O devices for the request.
9. The computer-implemented method of claim 1 , the deriving comprising automatically including in the set of input devices or the set of output devices an input or output (I/O) device that is an only operational input device or only operational output device, that is located closest to a majority of participants in the physical room, that has a best or only capability for handling data being communicated through the direct or network connection, or that is a most compatible I/O device to one of the at least one input device or at least one output device.
10. The computer-implemented method of claim 1 , the automatically establishing comprising:
receiving data from one of the at least one input device or the set of input device;
transmitting the data to one of the set of output devices or the at least one output device.
11. The computer-implemented method of claim 1 , the automatically establishing comprising transmitting an instruction to a media switch to establish the direct or network connection.
12. A system for managing multiple devices in a physical room with multimedia communications in real time, comprising:
a processor;
a memory coupled with the processor and storing computer-implemented instructions which when executed cause the processor to perform:
receiving and storing input data generated in real time by one of a plurality of input devices in a physical room,
the input data including digital image data or digital sound data;
identifying in real time from the input data a user performing an action with respect to at least one of the plurality of input devices or at least one of a plurality of output devices in the physical room, comprising:
extracting a reference to an input or output (I/O) device;
computing a first confidence score for determining the reference;
when the first confidence score is less than a first threshold, sending a first request for user input to improve the first confidence score; and
when the first confidence score is no less than the first threshold, extracting a handle for a specific I/O device and computing a second confidence score for determining the handle;
when the second confidence score is less than a second threshold, sending a second request for user input to improve the second confidence score; and
when the second confidence score is no less than the second threshold, determining that the user action is performed with respect to the specific I/O device;
deriving in real time from the action a request from the user for a direct or network connection between the at least one input device and a set of the plurality of output devices or a direct or network connection between the at least one output device and a set of the plurality of input devices: comprising:
determining whether the action includes an indication to connect (I/O) devices; and
in response to determining that the action includes an indication to connect I/O devices:
determining that the action includes a reference to only to a first I/O device of the plurality of input devices and the plurality of output devices; and
selecting a second I/O device of the plurality of input devices and the plurality of output devices to connect with the first I/O device for the request;
automatically establishing the direct or network connection in real time.
13. The system of claim 12 ,
the plurality of input devices including a microphone, a camera, a thermometer, a mouse, a keyboard, or another device that measures a physical aspect of any portion of the physical room,
the plurality of output devices including a screen, a light, a speaker, or another device that communicates information.
14. The system of claim 12 , further comprising the plurality of input devices and the plurality of output devices.
15. The system of claim 12 , the automatically establishing comprising transmitting an instruction to a media switch to connect the at least one input device and the set of output devices or to connect the at least one output device and the set of output devices.
16. The system of claim 15 , further comprising the media switch.
17. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause performance of a method of managing multi-role activities in a physical room with multimedia communications in real time, the method comprising:
receiving and storing input data generated in real time by one of a plurality of input devices in a physical room,
the input data including digital image data or digital sound data;
identifying in real time from the input data a user performing an action with respect to at least one of the plurality of input devices or at least one of a plurality of output devices in the physical room, comprising:
extracting a reference to an input or output (I/O) device;
computing a first confidence score for determining the reference;
when the first confidence score is less than a first threshold, sending a first request for user input to improve the first confidence score; and
when the first confidence score is no less than the first threshold, extracting a handle for a specific I/O device and computing a second confidence score for determining the handle;
when the second confidence score is less than a second threshold, sending a second request for user input to improve the second confidence score; and
when the second confidence score is no less than the second threshold, determining that the action is performed with respect to the specific I/O device;
deriving in real time from the action a request from the user for a direct or network connection between the at least one input device and a set of the plurality of output devices or a direct or network connection between the at least one output device and a set of the plurality of input devices, comprising:
determining whether the action includes an indication to connect (I/O) devices; and
in response to determining that the action includes an indication to connect I/O devices:
determining that the action includes a reference to only to a first I/O device of the plurality of input devices and the plurality of output devices; and
selecting a second I/O device of the plurality of input devices and the plurality of output devices to connect with the first I/O device for the request;
automatically establishing the direct or network connection in real time.
18. The one or more non-transitory storage media of claim 17 , the method further comprising obtaining device data regarding a new input or output (I/O) device in the physical room, the device data including a handle for the new I/O device that includes a spoken word or a gesture different from an identifier provided by a vendor of the new I/O device.
19. The one or more non-transitory storage media of claim 17 , the identifying comprising:
extracting from the input data a mechanism for a relative reference to an input or output (I/O) device;
mapping the relative reference to the at least one input device or the at least one output device.
20. The one or more non-transitory storage media of claim 17 , the handle being a shorthand name for referring to the specific I/O device.
21. The one or more non-transitory storage media of claim 17 , the deriving further comprising, in response to determining that the action does not include an indication to connect I/O devices, determining whether the action includes references to two I/O devices that can be connected only with each other.
22. The one or more non-transitory storage media of claim 17 , the deriving comprising automatically including in the set of input devices or the set of output devices an input or output (I/O) device that is an only operational input device or only operational output device, that is located closest to a majority of participants in the physical room, that has a best or only capability for handling data being communicated through the direct or network connection, or that is a most compatible I/O device to one of the at least one input device or at least one output device.