Robotic emulation device
A robotic emulation device includes a processor, memory and multimedia interfaces to capture a video signal from a target device, analyze it, and transmit manipulation control signals to the target device input by emulating a peripheral input device, wherein the manipulation control signals determine manipulation or selection of a target interface control associated with a first user interface encoded and transmitted in the video signal. Furthermore, the robotic emulation device may be connected via an embedded transceiver to at least one computer or computer tenant for relaying the captured video data for analysis, and based on the analysis, applying inferred manipulation semantics.
1. A robotic emulation device, comprising:
a processor, a memory, a first transceiver and a second transceiver, the first transceiver being coupled to a first physical connector;
the first physical connector being configured for connection to a target device, wherein the robotic emulation device, when the first physical connector is connected to the target device, is arranged to receive a video signal from the target device;
the robotic emulation device further being arranged to transmit electric signals to a peripheral input device via the second transceiver, the peripheral input device being coupled to the target device via a third transceiver;
the peripheral input device having an associated peripheral input device identification which is indicative of a computer keyboard or mouse;
the processor being configured to (1) interpret the video signal received via the first physical connector, wherein the interpretation includes identifying based on semantic analysis a target interface control associated with a first user interface encoded and transmitted in the video signal, (2) determine manipulation data associated to the target interface control, and (3) transmit one or more manipulation data electric signals to the peripheral input device via the second transceiver, the one or more manipulation data electric signals including key codes associated with an emulated performance of one or more manipulation actions at the peripheral input device;
wherein the peripheral input device is configured to form one or more encoded inputs from the one or more manipulation data electric signals and to provide the encoded inputs to the target device via the third transceiver.
2. The robotic emulation device of claim 1 , wherein the peripheral input device is configured to emulate a peripheral device.
3. The robotic emulation device of claim 1 , wherein the peripheral input device further comprises a peripheral device driver.
4. The robotic emulation device of claim 1 , wherein the peripheral input device comprises a hardware bus core driver.
5. The robotic emulation device of claim 4 , wherein the peripheral input device operates software implementing a human device interface profile or function.
6. The robotic emulation device of claim 1 , wherein at least one among the target device or the robotic emulation device is communicatively coupled with at least one computer host or tenant, wherein the robotic emulation device manipulates the target device user interface to manage and configure computer hosts, tenants, virtual machines, communication, resources, tasks, resource allocations, responses or flux infrastructure comprising the least one computer host or tenant.
7. The robotic emulation device of claim 1 , wherein the target interface control is associated with an interest semantic.
8. The robotic emulation device of claim 1 , wherein the manipulation data is inferred and applied based on interpreting video signals encoding pixels associated with a digital representation of a user agenda or task.
9. The robotic emulation device of claim 1 , wherein the manipulation data is inferred and applied based on inputs from a sensor capturing data associated with a rendering of a user agenda or task.
10. The robotic emulation device of claim 1 , wherein the manipulation data is determined based on interpreting video signals encoding a digital representation of an agenda or task management application output.
11. The robotic emulation device of claim 1 , wherein the robotic emulation device is communicatively coupled with at least one computer host or tenant, wherein the robotic emulation device communicates imaging data encoded in the video signal with the at least one computer host or tenant.
12. The robotic emulation device of claim 1 , wherein the target interface control is determined based on a first inferred semantic and further based on semantic drift between the first inferred semantic and a semantic associated with the target interface control.
13. The robotic emulation device of claim 12 , wherein the target interface control is determined based on an interest semantic.
14. The robotic emulation device of claim 13 , wherein the target interface control is associated with an inferred semantic identity or activity, wherein the semantic identity or activity is inferred and associated with the target interface control based on video signal analysis, wherein the target interface control is determined based on a semantic drift between the semantic identity or activity and the interest semantic.
15. The robotic emulation device of claim 1 , wherein the target interface control is associated with an inferred semantic identity or activity, wherein the semantic identity or activity is inferred and associated with the target interface control based on a document analysis.
16. The robotic emulation device of claim 15 , wherein the document analysis is based on launching a document viewing application on the target device and acquiring and analyzing video signals from the target device comprising the document viewing application user interface.
17. The robotic emulation device of claim 15 , wherein the document analysis is based on capturing optical sensor inputs comprising renderings of pages of the document and analyzing the optical sensor inputs.
18. The robotic emulation device of claim 15 , wherein the document analysis is based on downloading and analyzing the document data from a web server.
19. The robotic emulation device of claim 1 , wherein robotic emulation device is communicatively coupled with at least one computer host or tenant via a fourth transceiver, and wherein the video signal comprises a plurality of video signals, and further wherein the robotic emulation device is configured to forward video data associated with one or more of the plurality of received video signals to the at least one computer host or tenant, the at least one computer host or tenant applying image analysis to the video data to determine user interface control semantics and to return user interface control semantics signals to the robotic emulation device to cause manipulation action signals inferences by the robotic emulation device.
20. The robotic emulation device of claim 1 , wherein
the processor is being further configured to determine content associated to the target user interface control and forming one or more data signals based on the determined content, and transmitting the one or more data signals to the peripheral input device;
wherein the peripheral input device is configured to form, based on the one or more data signals, one or more encoded inputs related to the emulated peripheral input device, and to provide the one or more encoded inputs to the target device via the third transceiver.
21. A robotic emulation device, comprising:
a processor, a memory and at least one transceiver;
the robotic emulation device being arranged to receive a video signal from the target device via the at least one transceiver;
the robotic emulation device further being arranged to transmit electric signals to the target device via the at least one transceiver;
the robotic emulation device further being arranged to communicate data with at least one computer host or tenant via the at least one transceiver;
the robotic emulation device storing an associated peripheral input device identification which is indicative of an emulated peripheral input device;
the processor being configured to control a movement and selection by a cursor of a target user interface control on a first user interface by: (1) interpreting the video signal received via the at least one transceiver, wherein the interpreting includes identifying based on semantic analysis a target interface control associated with the first user interface encoded and transmitted in the video signal, (2) determining content associated to the target user interface control and forming one or more manipulation signals based on the determined content, and (3) forming, based on the one or more manipulation signals, one or more encoded inputs related to the emulated peripheral input device; and
(4) transmitting via the at least one transceiver the one or more encoded inputs to the target device,
wherein the video signal comprises a plurality of video signals, and further wherein the robotic emulation device is configured to forward data associated with one or more of the plurality of received video signals to the at least one computer host or tenant, the at least one computer host or tenant applying image analysis to the data to determine user interface control semantics and to return user interface control semantics signals to the robotic emulation device to cause manipulation action signals inferences by the robotic emulation device.
22. The robotic emulation device of claim 21 , wherein the content is determined based on an inferred first semantic at an endpoint, the first semantic being inferred based on an input from a sensor.
23. The robotic emulation device of claim 22 , wherein the content is further determined based on a semantic matching between the first semantic and a second semantic comprised in a user preference or profile.
24. The robotic emulation device of claim 21 , wherein the target user interface control is determined based on an inferred first semantic at an endpoint, the first semantic being inferred based on an input from a sensor.
25. The robotic emulation device of claim 24 , wherein the target user interface control is further determined based on a semantic matching between the first semantic and a second semantic comprised in a user preference or profile.
26. The robotic emulation device of claim 21 , wherein the content or the target user interface control is further determined based on an indication from a user.
27. The robotic emulation device of claim 21 , wherein the peripheral input device identification is stored as a device descriptor.
28. The robotic emulation device of claim 21 , wherein the robotic emulation device having an associated peripheral device designation and wherein the processor is further being configured to: determine one or more manipulation actions associated to the target interface control, and form and transmit one or more manipulation action signals to the target device via the at least one transceiver, the one or more manipulation action signals being based on the peripheral device designation and including key codes associated with the an emulated performance of the one or more manipulation actions by the robotic emulation device.
29. A robotic emulation device, comprising:
a processor, a memory, a first wireless transceiver and a first physical connector;
the first physical connector being configured for connection to a target device, wherein the robotic emulation device, when the first physical connector is connected to the target device, is arranged to receive a video signal from the target device;
the robotic emulation device further being arranged to transmit wireless signals to a peripheral input device via the first wireless transceiver, the peripheral input device being coupled to the target device via a second physical connector;
the peripheral input device having an associated peripheral input device identification which is indicative of a computer keyboard or mouse;
the processor being configured to (1) interpret the video signal received via the first physical connector, wherein the interpretation includes identifying based on semantic analysis a target interface control associated with a first user interface encoded and transmitted in the video signal, (2) determine one or more manipulation actions associated to the target interface control, and (3) transmit one or more manipulation action wireless signals to the peripheral input device via the first wireless transceiver, the one or more manipulation action wireless signals including key codes associated with an emulated performance of the one or more manipulation actions by the peripheral input device;
wherein the peripheral input device is configured to form one or more encoded inputs from the one or more manipulation actions and to provide the encoded inputs to the target device via the second physical connector.
30. A robotic emulation device, comprising:
a processor, a memory and a first physical connector;
a memory storing a plurality of manipulation action semantics;
the first physical connector being configured for connection to a target device, wherein the robotic emulation device, when the first physical connector is connected to the target device, is arranged to receive a video signal from the target device;
the robotic emulation device further being arranged to transmit signals to the target device via the first physical connector;
the robotic emulation device having an associated designation of a peripheral input device;
the processor being configured to (1) interpret the video signal received via the first physical connector, wherein the interpretation includes identifying based on semantic analysis a target interface control associated with a first user interface encoded and transmitted in the video signal, (2) determine one or more manipulation actions associated to the target interface control, and (3) form and transmit one or more manipulation action signals to the target device via the first physical connector, the one or more manipulation action signals comprising encoded inputs based on the peripheral input device designation, and including key codes associated with the performance of the one or more manipulation actions by the target device,
wherein the one or more manipulation actions are determined based on a plurality of semantic factorizations of one or more manipulation action semantics among the plurality of the stored manipulation action semantics.