Wireless external multi-microphone system for mobile device environment
View Patent ↗A method to merge data recorded using a wireless external multi-microphone system is described. The method includes determining a communication delay between a mobile device and an accessory device. In response to activation of a camera application, an activation indication is sent to the accessory device and, in response, the accessory device starts recording audio. In response to a camera event, a camera event indication is sent to the accessory device. The camera event includes starting a video recording, pausing/resuming a video recording, ending a video recording or taking a photograph. In response to receiving the camera event indication, the accessory device creates a marker corresponding to the reception time. The method also includes receiving, at the mobile device, the audio data and the marker; and merging image data and the audio based on the marker and the communication delay. Apparatus and computer readable media are also described.
1. A method comprising:
determining a communication delay between a first device and a second device;
in response to activation of a camera application on the first device, sending an indication of the activation to the second device;
in response to a camera event, sending an indication of the camera event to the second device, where the camera event comprises taking a photograph;
receiving, at the first device from the second device, audio data and a marker corresponding to a time when the second device received the indication of the camera event; and
merging image data and the audio data, where merging the image data and the audio data comprises:
determining a first point in the audio data corresponding to the time the photograph was taken based at least in part on the marker and the communication delay and merging the photograph with a segment of the audio data which spans a time which precedes the first point in the audio data by a first pre-configured amount of time and which follows the first point in the audio data by a second pre-configured amount of time while cross-correlating audio data from the first device with the audio data from the second device.
2. The method according to claim 1 , where the communication delay comprises a transmission time delay and a processing delay.
3. The method according to claim 1 , where determining the communication delay comprises performing a loop-back procedure.
4. The method according to claim 1 , where receiving the audio data comprises receiving a streaming audio file.
5. The method according to claim 1 , further comprising in response to de-activation of the camera application, sending an indication of the deactivation to the second device.
6. The method according to claim 5 , where receiving the audio data and the marker is performed in response to sending the indication of the deactivation.
7. The method according to claim 1 , where the audio data comprises a plurality of audio data feeds recorded by a plurality of microphones.
8. The method according to claim 1 , where merging the image data and the audio data comprises determining a point in the audio data corresponding to the time the camera event occurred based at least in part on the marker and the communication delay.
9. A method comprising:
determining a communication delay between a first device and a second device;
in response to receiving, at the second device, an indication of an activation of a camera application on the first device, starting recording audio data using a plurality of microphones;
in response to receiving an indication of a camera event, creating a marker corresponding to a time when the second device received the indication of the camera event, where the camera event comprises taking a photograph; and
sending, to the first device from the second device, the audio data and the marker then cross-correlating audio data from the plurality of microphones and the audio data from the first device.
10. The method according to claim 9 , where determining the communication delay comprises performing a loop-back procedure.
11. The method according to claim 9 , further comprising in response to receiving an indication of a deactivation of a camera application on the first device, sending the audio data and the marker.
12. An apparatus, comprising at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following:
to determine a communication delay between a first device and a second device;
in response to activation of a camera application on the first device, to send an indication of the activation to the second device;
in response to a camera event, to send an indication of the camera event to the second device, where the camera event comprises taking a photograph;
to receive, at the first device from the second device, audio data and a marker corresponding to a time when the second device received the indication of the camera event; and
to merge image data and the audio data based at least in part on the marker and the communication delay while cross-correlating audio data from the first device with the audio data from the second device.
13. The apparatus according to claim 12 , where, when determining the communication delay, the at least one memory and the computer program code are further configured to cause the apparatus to perform a loop-back procedure.
14. The apparatus according to claim 12 , where the at least one memory and the computer program code are further configured to cause the apparatus, in response to de-activation of the camera application, to send an indication of the deactivation to the second device.
15. The apparatus according to claim 14 , where receiving the audio data and the marker is performed in response to sending the indication of the deactivation.
16. The apparatus according to claim 12 , where, when merging the image data and the audio data, the at least one memory and the computer program code are further configured to cause the apparatus to determine a point in the audio data corresponding to the time the camera event occurred based at least in part on the marker and the communication delay.
17. A non-transitory computer readable medium tangibly encoded with a computer program executable by a processor to perform actions comprising:
determining a communication delay between a first device and a second device;
in response to activation of a camera application on the first device, sending an indication of the activation to the second device;
in response to a camera event, sending an indication of the camera event to the second device, where the camera event comprises taking a photograph;
receiving, at the first device from the second device, audio data and a marker corresponding to a time when the second device received the indication of the camera event; and
merging image data and the audio data based at least in part on the marker and the communication delay while cross-correlating audio data from the first device with the audio data from the second device.
18. The non-transitory computer readable medium according to claim 17 , where determining the communication delay comprises performing a loop-back procedure.
19. The non-transitory computer readable medium according to claim 17 , where the actions further comprise: in response to de-activation of the camera application, sending an indication of the deactivation to the second device, and
where receiving the audio data and the marker is performed in response to sending the indication of the deactivation.