Synchronizing remote audio with fixed video
View Patent ↗A multimedia device ( 100 ) including a separating entity configured to separate a multimedia stream into audio frames and video frames, a sequencing entity configured to add a sequence number to at least one audio frame, a transceiver configured to transmit audio frames to a remote audio device, a controller coupled to a video player, the controller configured to determine a delay associated with transmitting the audio frames to the remote audio device based upon the sequence number and to control the presentation of the video frames at the video player based on the delay.
1. A method in a handheld multimedia device, the method comprising:
separating multimedia content into audio frames and video frames;
adding a sequence number to at least one of the audio frames and at least one of the video frames;
presenting the video frames on a user interface of the multimedia device;
transmitting the audio frames to a wireless audio device, which is separate from the multimedia device;
measuring and tracking a transmission delay associated with the transmission of the audio frames to the wireless audio device;
assessing the synchronization between the audio and video frames using the measured transmission delay through a correlation of the sequence numbers added to the audio and video frames; and
controlling the presentation of the video frames on the user interface, to reduce or eliminate a lack of synchronization, by delaying a transfer of video frames presented to the multimedia device in a manner that synchronizes the audio and video frames when the measured transmission delay of the audio frames exceeds a specified threshold.
2. The method of claim 1 , controlling the presentation of the video frames on the user interface by delaying the presentation of the video frames.
3. The method of claim 1 ,
determining an estimated delay between the audio and video frames before determining the measured transmission delay associated with transmitting the audio frames to the wireless audio device,
delaying the presentation of the video frames on the user interface of the multimedia device based upon the estimated delay before compensating for the measured transmission delay.
4. The method of claim 3 , determining the estimated delay based upon estimated delay information stored in a look-up table of the multimedia device.
5. The method of claim 1 ,
substantially reducing a processing delay of the multimedia device by obtaining a list of supported codecs on the wireless audio headset that do not require transcoding on the multimedia device;
transmitting the audio frames in the native coding scheme rather than the mandatory encoding scheme only if the native coding scheme is supported by both the wireless audio device and by the multimedia device.
6. The method of claim 1 ,
transmitting the audio frames to the wireless audio device for presentation on the wireless audio device,
determining remote delay associated with processing audio frames on the wireless audio device,
controlling the presentation of the video frames at the multimedia device based on the remote delay.
7. The method of claim 6 ,
determining local delay associated with processing audio frames on the multimedia device,
controlling the presentation of the video frames at the multimedia device based on the local delay.
8. The method of claim 1 , determining the delay includes measuring a transmission time of the audio frames having the sequence number to the wireless audio device.
9. The method of claim 1 , determining the delay based on a comparison of the number of video frames consumed to a number of audio frames consumed during a common time interval.
10. A multimedia device, comprising:
a separating entity that separates a multimedia content into audio frames and video frames;
a sequencing entity coupled to the separating entity, the sequencing entity that adds a sequence number to at least one audio frame and at least one video frame;
a video player coupled to the separating entity, the video player that presents video frames received from the separating entity;
a transceiver coupled to the sequencing entity, the transceiver that transmits audio frames to a wireless audio device, which is separate from the multimedia device;
a controller coupled to the video player,
the controller that measures and track a transmission delay associated with the audio frames being transmitted to the wireless audio device and assessing the synchronization between the audio and video frames using the measured transmission delay through a correlation of the sequence numbers added to the audio and video frames; and
the controller that controls the presentation of the video frames at the video player and to reduce or eliminate a lack of synchronization, by delaying a transfer of video frames presented to the video player in a manner that synchronizes the audio and video frames when the measured transmission delay of the audio frames exceeds a specified threshold.
11. The device of claim 10 , the controller that controls the presentation of the video frames on the video player by delaying the presentation.
12. The device of claim 10 ,
the controller that determines an estimated delay between the audio and video frames before measuring the transmission delay associated with transmitting the audio frames to the wireless audio device,
delaying the presentation of the video frames on the video player based upon the estimated delay before compensating for the measured transmission delay.
13. The device of claim 12 ,
a look-up table accessible by the controller, the look-up table storing estimated delay information,
the controller that determines the estimated delay for the wireless audio device using wireless audio device delay information stored in the look-up table.
14. The device of claim 10 ,
the controller that substantially reduces a processing delay of the multimedia device by obtaining a list of supported codecs on the wireless audio headset that do not require transcoding on the multimedia device;
the transmitter that transmits the audio frames in the native coding scheme rather than the mandatory encoding scheme only if the native coding schemes is supported by both the wireless audio device and by the multimedia device.
15. The device of claim 10 ,
the controller that determines a remote delay associated with processing audio frames on the wireless audio device,
the controller that controls the presentation of the video frames at the video player based on the remote delay.
16. The method of claim 15 ,
the controller that determines a local delay associated with processing audio frames on the multimedia device,
controlling the presentation of the video frames at the video player based on the local delay.
17. The device of claim 10 , the multimedia device is a handheld electronics device.
18. The device of claim 10 , the controller that determines the delay by measuring a transmission time of the audio frames having the sequence number to the wireless audio device.
19. The device of claim 10 , the controller that determines the delay based on a comparison of a number of video and audio frames consumed during a common time interval.