IP Library Granted Patent US 9,007,477
Granted Patent B2
US 9,007,477 · App. 13/380,513 · Granted Apr 14, 2015

Audio-controlled image capturing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,007,477
App. No.
13/380,513
Filed
Dec 22, 2011
Granted
Apr 14, 2015
Kind
B2
Art Unit
2663
USPC
348/211.99
Abstract

A method comprising: receiving a plurality of images corresponding to a time period covering the intended moment for releasing the shutter; receiving an audio signal associated with the plurality of images using audio capturing means; analyzing the received audio signal in order to determine an auditory event associated with a desired output image; and selecting at least one of the plurality of images on the basis of the analysis of the received audio signal for further processing in order to obtain the desired output image.

Claims (54)

1. A method comprising:

receiving a plurality of images corresponding to a time period covering an intended moment;

receiving an audio signal associated with the plurality of images;

estimating the direction of arrival of the audio signal in order to determine when an object producing the audio signal is in a desired part of an image;

analyzing the received audio signal in order to determine an auditory event associated with a desired output image; and

selecting at least one of the plurality of images on the basis of the analysis of the received audio signal for further processing in order to obtain the desired output image.

2. The method according to claim 1 , wherein

receiving the plurality of images comprises capturing the plurality of images during the time period covering the intended moment using an image capturing unit in response to a user of an image capturing device giving a command to release a shutter at the intended moment; and

receiving the audio signal comprises capturing the audio signal associated with the plurality of images using audio capturing means.

3. The method according to claim 1 , further comprising:

receiving a plurality of images corresponding to a first time instant before the intended moment.

4. The method according to claim 1 , further comprising:

receiving a plurality of images corresponding to a second time instant after the intended moment.

5. The method according to claim 1 , further comprising:

estimating energy of the received audio signal in order to determine an event of interest within the plurality of images.

6. The method according to claim 5 , wherein an abrupt change in the energy of the received audio signal is used to determine a desired moment of the event of interest.

7. The method according to claim 1 , further comprising:

estimating a fundamental frequency of the received audio signal in order to determine at least one image from the plurality of images in which an object producing the audio signal is closest to the image capturing device providing the plurality of images.

8. The method according to claim 7 , further comprising:

using the estimation of the fundamental frequency to select an image in which the object producing the audio signal is approaching the image capturing device providing the plurality of images.

9. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive a plurality of images corresponding to a time period covering an intended moment;

receive an audio signal associated with the plurality of images;

estimate the direction of arrival of the audio signal, captured by a microphone array, in order to determine when the object producing the audio signal is in a desired part of the image;

analyze the received audio signal in order to determine an auditory event associated with a desired output image; and

select at least one of the plurality of images on the basis of the analysis of the received audio signal for further processing in order to obtain the desired output image.

10. The apparatus according to claim 9 , wherein

the apparatus is further caused to:

receive a plurality of images corresponding to a first time instant before the intended moment.

11. The apparatus according to claim 9 , wherein

the apparatus is further caused to:

receive a plurality of images corresponding to a second time instant after the intended moment.

12. The apparatus according to claim 9 , wherein

the apparatus is further caused to:

determine that the object is in desired part of the image when the object producing the audio signal residing substantially perpendicular to the microphone array.

13. The apparatus according to claim 9 , wherein the apparatus is further caused to:

estimate energy of the received audio signal in order to determine an event of interest within the plurality of images.

14. The apparatus according to claim 13 , wherein

the apparatus is further caused to:

determine that the desired moment of the event of interest is when there is an abrupt change in the energy of the received audio signal.

15. The apparatus according to claim 9 , wherein the apparatus is further caused to:

estimate a fundamental frequency of the received audio signal in order to determine at least one image from the plurality of images in which an object producing the audio signal is closest to the image capturing device providing the plurality of images.

16. The apparatus according to claim 15 , wherein

the apparatus is further caused to:

use the estimation of the fundamental frequency to select an image in which the object producing the audio signal is approaching the image capturing device providing the plurality of images.

17. A computer program product comprising a non-transitory computer-readable medium bearing computer program code embodied therein for use with a computer, the computer program code configured for:

receiving a plurality of images corresponding to a time period covering an intended moment;

receiving an audio signal associated with the plurality of images;

estimating the direction of arrival of the audio signal, received from a microphone array consisting of two or more microphones, in order to determine when an object producing the audio signal is in a desired part of an image;

analyzing the received audio signal in order to determine an auditory event associated with a desired output image; and

selecting at least one of the plurality of images on the basis of the analysis of the received audio signal for further processing in order to obtain the desired output image.