SYSTEM FOR DEVICE-AGNOSTIC SYNCHRONIZATION OF AUDIO AND ACTION OUTPUT
A system and method are disclosed for device-agnostic synchronizing of audio and physical actions at a client device. Different client devices may synthesize audio from text at different rates, and different client devices may perform physical actions such as gestures and other physical movements at different rates. The system of the present technology enables synchronization of audio with physical actions at different client devices, where audio and/or physical actions may be synthesized at different rates.
1 . A client device for synchronizing audio output with movement performed at the client device, the client device comprising:
a memory; and
a processor, the processor configured to execute instructions to:
receive text to speech (TTS) data for audio output at the client device,
receive movement commands for causing virtual or real movement at the client device, the movement commands being device-agnostic; and
synchronize the audio output with the virtual or real movement at the client device, using timing data at the client device related to the virtual or real movement.
2 . The client device of claim 1 , wherein the client device synchronizes a beginning of the audio output with a beginning of the virtual or real movement at the client device.
3 . The client device of claim 1 , wherein the client device synchronizes an ending of the audio output with an ending of the virtual or real movement at the client device.
4 . The client device of claim 1 , wherein the client device slows one of the audio output and virtual or real movement to synchronize a beginning and ending of the audio output with a beginning and ending of the virtual or real movement at the client device.
5 . The client device of claim 4 , wherein slowing one of the audio output and virtual or real movement comprises slowing a motor affecting the movement in the client device.
6 . The client device of claim 4 , wherein slowing one of the audio output and virtual or real movement comprises slowing a speed at which the audio is output.
7 . The client device of claim 1 , wherein the client device synchronizes a beginning of a first portion of the audio output with a first portion of the virtual or real movement at the client device, and the client device synchronizes a beginning of a second portion of the audio output with a second portion of the virtual or real movement at the client device.
8 . The client device of claim 1 , wherein the processor and memory further receive a locality classification affecting synchronization of the audio output with the movement command at the client device.
9 . The client device of claim 1 , wherein the processor and memory further receive a mood classification affecting synchronization of the audio output with the movement command at the client device.
10 . The client device of claim 1 , wherein the client device is a mechanical device with moving parts.
11 . The client device of claim 10 , wherein the mechanical device comprises a motor, the timing data comprising an operating speed of the motor.
12 . The client device of claim 10 , wherein the mechanical device is a robot having at least at least one of limbs, hands and a facial feature.
13 . The client device of claim 1 , wherein the client device comprises a computer with a graphical display screen, the movement being virtual movement of a character displayed on the display screen.
14 . A platform system for synchronizing audio output with movement performed at a client device, the system comprising one or more servers, and the system comprising:
a memory; and
a processor, the processor configured to execute instructions to:
receive device-agnostic text to speech (TTS) data for audio output at the client device,
receive device-agnostic movement commands for causing the virtual or real movement at the client device; and
cause the transmission of the device-agnostic TTS data and movement commands to the client device to enable the client device to synchronize the audio output with the virtual or real movement at the client device, using timing data available at the client device related to the virtual or real movement.
15 . The platform system of claim 14 , wherein causing the transmission of the TTS data and movement commands to the client device enables the client device to synchronize a beginning of the audio output with a beginning of the virtual or real movement at the client device.
16 . The platform system of claim 14 , wherein causing the transmission of the TTS data and movement commands to the client device enables the client device to synchronize an ending of the audio output with an ending of the virtual or real movement at the client device.
17 . The platform system of claim 14 , wherein causing the transmission of the TTS data and movement commands to the client device enables the client device to slow one of the audio output and virtual or real movement to synchronize a beginning and ending of the audio output with a beginning and ending of the virtual or real movement at the client device.
18 . The platform system of claim 17 , wherein slowing one of the audio output and virtual or real movement comprises slowing a motor affecting the movement in the client device.
19 . The platform system of claim 17 , wherein slowing one of the audio output and virtual or real movement comprises slowing a speed at which the audio is output.
20 . The platform system of claim 14 , wherein causing the transmission of the TTS data and movement commands to the client device enables the client device to synchronize a beginning of a first portion of the audio output with a first portion of the virtual or real movement at the client device, and enabling the client device to synchronize a beginning of a second portion of the audio output with a second portion of the virtual or real movement at the client device.
21 . The platform system of claim 14 , wherein the processor and memory further receive a locality classification and cause transmission of the locality classification to the client device, the locality classification affecting synchronization of the audio output with the movement command at the client device.
22 . The platform system of claim 14 , wherein the processor and memory further receive a mood classification and cause transmission of the mood classification to the client device, the mood classification affecting synchronization of the audio output with the movement command at the client device.
23 . A method of synchronizing audio output with movement performed at a client device, comprising:
receiving device-agnostic text to speech (TTS) data for audio output at the client device;
receiving device-agnostic movement commands for causing virtual or real movement at the client device; and
causing the transmission of the TTS data and movement commands to the client device to enable the client device to synchronize the audio output with the virtual or real movement at the client device, using timing data received at the client device related to the virtual or real movement.