Multi-modal synchronization
View Patent ↗A system for synchronizing multiple modalities is described. A user may use multiple modalities, such as voice and browser, to interact with data on a network, such as the World Wide Web. All of the modalities may be synchronized so that all are updated when the user enters information in just one. A method of communicating between devices includes receiving a request for first-modality data that includes first content, and sending a message in response to receiving the request, the message including information allowing the request of second-modality data that includes second content overlapping the first content. Another method includes requesting first data for a first modality, the first data including first content, and automatically requesting second data for a second modality, wherein the second data includes second content that overlaps the first content.
1. A computer-implemented method of performing multi-modal synchronization, the method comprising:
enabling, at a given point in time, a first-modality gateway and a second-modality gateway to output the same specific information from the same Uniform Resource Locator (URL) as both first-modality content and second-modality content, and to receive input to the output same specific information using either of the first-modality gateway and the second-modality gateway, further comprising:
opening a first-modality session via a first connection from a first-modality program on a client device to the first-modality gateway;
opening a second-modality session via a second connection from a second-modality program on the client device to the second-modality gateway on a server;
acquiring a unique identifier of the client device by the first-modality gateway and the second-modality gateway;
transmitting the unique identifier from the first-modality gateway to a synchronization handler;
creating a unique messaging channel on a third connection, based upon the unique identifier at the synchronization handler; and
subscribing the first-modality gateway and the second-modality program to the unique messaging channel at a synchronization handler based on the unique identifier; and
automatically determining a first-modality version of a resource if the input is received using the second-modality gateway, and a second-modality version of the resource if the input is received using the first-modality gateway.
2. The method of claim 1 further including associating the first-modality session and the second-modality session with the unique messaging channel, at the server.
3. The method of claim 1 further including publishing an indication of the second-modality content to the first-modality gateway on the unique messaging channel.
4. The method of claim 3 wherein the indication is published on the unique messaging channel by the server.
5. The method of claim 3 , further comprising requesting first-modality content corresponding to the second-modality content at the server from the first-modality gateway.
6. The method of claim 3 , wherein the published indication includes the Uniform Resource Locator (URL).
7. The method of claim 6 , wherein the URL points to a dummy page.
8. The method of claim 1 , wherein the first-modality gateway or the second-modality gateway are a voice gateway or a web server.
9. The method of claim 1 , wherein the first-modality program or a second-modality program are a voice client or a web browser.
10. The method of claim 1 , wherein the unique identifier is a network address of the client device.
11. The method of claim 1 , wherein the unique identifier is a user ID of the user operating the client device.
12. The method of claim 1 , wherein the unique identifier is a hardware identifier of the client device.
13. The method of claim 1 , wherein the unique identifier is a combination of a firewall network address and a network address of the client device.
14. A computer-implemented method of performing multi-modal synchronization, the method comprising:
enabling, at a given point in time, a first-modality gateway and a second-modality gateway to output the same specific information from the same Uniform Resource Locator (URL) as both first-modality content and second-modality content, and to receive input to the output same specific information using either of the first-modality gateway and the second-modality gateway, further comprising:
opening a first-modality session via a first connection from a first-modality program on a client device to the first-modality gateway;
opening a second-modality session via a second connection from a second-modality program on the client device to the second-modality gateway on a server;
acquiring a unique identifier of the client device by the first-modality gateway and the second-modality gateway;
transmitting the unique identifier from the first-modality gateway to a synchronization handler;
creating a unique messaging channel on a third connection, based upon the unique identifier at the synchronization handler;
subscribing the first-modality gateway and the second-modality program to the unique messaging channel at the synchronization handler based on the unique identifier;
associating the unique messaging channel with the second-modality session, at the server;
transmitting the second-modality content from the sewer to the second-modality program via the second connection;
publishing an indication of the second-modality content to the first-modality gateway on the unique messaging channel; and
requesting the first-modality content corresponding to the second-modality content at the sewer from the first-modality gateway; and
automatically determining a first-modality version of a resource if the input is received using the second-modality gateway, and a second-modality version of the resource if the input is received using the first-modality gateway.
15. The method of claim 14 , wherein the first-modality is a voice modality.
16. The method of claim 14 , wherein the first-modality connection is established using the International Telecommunications Union-T Recommendation H.323 (H.323) standard protocol or a Session Initiation Protocol (SIP).
17. The method of claim 14 , wherein the unique identifier is an Internet Protocol (IP) address.
18. The method of claim 14 , wherein acquiring the unique identifier further comprises pulling the unique identifier out of a connection header.
19. The method of claim 14 , wherein acquiring the unique identifier further comprises querying the client device for the unique identifier.
20. The method of claim 14 , wherein the second-modality content is a dummy page that does not include content.
21. The method of claim 14 wherein the second-modality content includes a web page.
22. The method of claim 14 , wherein the server is a web server.
23. The method of claim 14 , wherein the second-modality connection is established in response to a user entering the Uniform Resource Locator (URL) into the client device.
24. The method of claim 14 , further comprising determining the first-modality content corresponding to the second-modality content, at the server.
25. The method of claim 14 , wherein the client device is a mobile device.
26. The method of claim 14 , further comprising:
requesting a response from the server, at the first-modality gateway; and
transmitting the response from the server to the first-modality gateway.
27. A system for performing multi-modal synchronization, the system comprising:
a client device including a first-modality program and a second-modality program;
a first-modality gateway configured to:
open a first-modality session from the first-modality program via a first connection,
acquire a unique identifier of the client device, and
transmit the unique identifier;
a server including a second-modality gateway configured to:
open a second-modality session from the second-modality program to a second-modality gateway via a second connection;
acquire the unique identifier of the client device at the second-modality gateway; and
automatically determine a first-modality version of a resource if input is received using the second-modality gateway, and a second-modality version of the resource if the input is received using the first-modality gateway; and
a synchronization handler configured to enable, at a given point in time, a first-modality gateway and a second-modality gateway to output the same specific information from the same Uniform Resource Locator as both first-modality content and second-modality content, and to receive the input to the output same specific information using either of the first-modality gateway and the second-modality gateway, by:
receiving the unique identifier from the first-modality gateway,
creating a unique messaging channel on a third connection, based upon the unique identifier handler, and
subscribing the first-modality gateway and the second-modality program to the unique messaging channel based on the unique identifier.
28. A computer program product tangibly embodied in a machine-readable storage medium, the computer program product for performing multi-modal synchronization, where the computer program product comprises instructions that, when read by a machine, operate to cause a data processing apparatus to:
enable, at a given point in time, a first-modality gateway and a second-modality gateway to output the same specific information from the same Uniform Resource Locator (URL) as both first-modality content and second-modality content, and to receive input to the output same specific information using either of the first-modality gateway and the second-modality gateway, by:
opening a first-modality session via a first connection from a first-modality program on a client device to the first-modality gateway;
opening a second-modality session via a second connection from a second-modality program on the client device to the second-modality gateway on a server;
acquiring a unique identifier of the client device by the first-modality gateway and the second-modality gateway;
transmitting the unique identifier from the first-modality gateway to a synchronization handler;
creating a unique messaging channel on a third connection, based upon the unique identifier at the synchronization handler; and
subscribing the first-modality gateway and the second-modality program to the unique messaging channel at a synchronization handler based on the unique identifier; and automatically determine a first-modality version of a resource if the input is received using the second-modality gateway, and a second-modality version of the resource if the input is received using the first-modality gateway.