Systems and methods for real-time remote control of mobile applications
Systems and methods for real-time, remote-control of mobile applications are provided. A communication session between a network device and a terminal device can be established. The network device can be configured to execute a mobile application. For example, the mobile application can include an input element at an initial state. The mobile application can be remotely controlled by the terminal device. Further, a data stream including content data can be received and transmitted during the communication session. For example, the content data can include an object presented by the mobile application. The content data can be displayed on the terminal device. An input event associated with the content data can be detected. Control data can be generated based on the detected input event. The control data can be received at the network device and the initial state of the input element can be modified.
1. A computer-implemented method, comprising:
facilitating, at a communication server, a communication session between a mobile device and a terminal device, wherein the mobile device is configured to execute a mobile application, wherein the mobile application includes an input element at an initial state, and wherein the mobile application is remotely controlled by the terminal device;
receiving a data stream during the communication session, wherein the data stream includes content data, wherein the content data includes an object presented by the mobile application, and wherein the object represents the input element;
determining whether the input element includes sensitive information;
transmitting the data stream during the communication session, wherein when the input element does not include sensitive information, when the data stream is received at the terminal device, the content data causes the object to be displayed at the terminal device, and wherein when the input element includes sensitive information, when the data stream is received at the terminal device, the data stream includes the content data and a field mask that causes the object to be masked when the content data is displayed at the terminal device;
detecting an input event associated with the object;
generating control data when the input event is detected; and
transmitting the control data, wherein when the control data is received at the mobile device, the control data modifies the initial state of the input element included in the mobile application.
2. The computer-implemented method of claim 1 , wherein when the control data is received at the mobile device, the input event is replicated at the mobile device.
3. The computer-implemented method of claim 1 , wherein transmitting the data stream during the communication session comprises:
displaying the object representing the input element, wherein the object includes content data; and
preventing the content data included in the input element from being manipulated.
4. The computer-implemented method of claim 1 , wherein when the control data is received at the mobile device, the control data causes a synthetic event to be generated at the mobile device, and wherein the synthetic event modifies the initial state of the input element included in the mobile application.
5. The computer-implemented method of claim 4 , wherein the synthetic event is generated by an operating system of the mobile device, and wherein the synthetic event replicates the detected input event.
6. The computer-implemented method of claim 1 , wherein when the control data is received at the mobile device, the method further comprises:
determining whether the control data corresponds to the input element; and
modifying the initial state of the input element when the control data corresponds to the input element, wherein modifying the initial state of the input element includes changing the initial state to a subsequent state.
7. A system, comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
facilitating, at a communication server, a communication session between a mobile device and a terminal device, wherein the mobile device is configured to execute a mobile application, wherein the mobile application includes an input element at an initial state, and wherein the mobile application is remotely controlled by the terminal device;
receiving a data stream during the communication session, wherein the data stream includes content data, wherein the content data includes an object presented by the mobile application, and wherein the object represents the input element;
determining whether the input element includes sensitive information;
transmitting the data stream during the communication session, wherein when the input element does not include sensitive information, when the data stream is received at the terminal device, the content data causes the object to be displayed at the terminal device, and wherein when the input element includes sensitive information, when the data stream is received at the terminal device, the data stream includes the content data and a field mask that causes the object to be masked when the content data is displayed at the terminal device;
detecting an input event associated with the object;
generating control data based on the detected input event; and
transmitting the control data, wherein when the control data is received at the mobile device, the control data modifies the initial state of the input element included in the mobile application.
8. The system of claim 7 , wherein when the control data is received at the mobile device, the input event is replicated at the mobile device.
9. The system of claim 7 , wherein transmitting the data stream during the communication session comprises:
displaying the object representing the input element, wherein the object includes content data; and
preventing the content data included in the input element from being manipulated.
10. The system of claim 7 , wherein when the control data is received at the mobile device, the control data causes a synthetic event to be generated at the mobile device, and wherein the synthetic event modifies the initial state of the input element included in the mobile application.
11. The system of claim 10 , wherein the synthetic event is generated by an operating system of the mobile device, and wherein the synthetic event replicates the detected input event.
12. The system of claim 7 , wherein when the control data is received at the mobile device, the operations further comprise:
determining whether the control data corresponds to the input element; and
modifying the initial state of the input element when the control data corresponds to the input element, wherein modifying the initial state of the input element includes changing the initial state to a subsequent state.
13. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:
facilitating, at a communication server, a communication session between a mobile device and a terminal device, wherein the mobile device is configured to execute a mobile application, wherein the mobile application includes an input element at an initial state, and wherein the mobile application is remotely controlled by the terminal device;
receiving a data stream during the communication session, wherein the data stream includes content data, wherein the content data includes an object presented by the mobile application, and wherein the object represents the input element;
determining whether the input element includes sensitive information;
transmitting the data stream during the communication session, wherein when the input element does not include sensitive information, when the data stream is received at the terminal device, the content data causes the object to be displayed at the terminal device, and wherein when the input element includes sensitive information, when the data stream is received at the terminal device, the data stream includes the content data and a field mask that causes the object to be masked when the content data is displayed at the terminal device;
detecting an input event associated with the object;
generating control data based on the detected input event; and
transmitting the control data, wherein when the control data is received at the mobile device, the control data modifies the initial state of the input element included in the mobile application.
14. The computer-program product of claim 13 , wherein when the control data is received at the mobile device, the input event is replicated at the mobile device.
15. The computer-program product of claim 13 , wherein transmitting the data stream during the communication session comprises:
displaying the object representing the input element, wherein the object includes content data; and
preventing the content data included in the input element from being manipulated.
16. The computer-program product of claim 13 , wherein when the control data is received at the mobile device, the control data causes a synthetic event to be generated at the mobile device, and wherein the synthetic event modifies the initial state of the input element included in the mobile application.
17. The computer-program product of claim 16 , wherein the synthetic event is generated by an operating system of the mobile device, and wherein the synthetic event replicates the detected input event.