IP Library Granted Patent US 9,282,279
Granted Patent B2
US 9,282,279 · App. 14/358,760 · Granted Mar 8, 2016

Quality enhancement in multimedia capturing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,282,279
App. No.
14/358,760
Granted
Mar 8, 2016
Kind
B2
Abstract

A method for controlling capturing of multimedia content, the method comprising: capturing multimedia content by an apparatus, said multimedia content comprising at least an audio signal and a video signal; obtaining control information based on at least either of said audio signal or video signal; controlling pre-processing of the audio signal based on the control information obtained from the video signal; and/or controlling pre-processing of the video signal based on the control information obtained from the audio signal.

Claims (60)

1. A method, comprising:

capturing multimedia content by an apparatus, said multimedia content comprising at least an audio signal and a video signal;

obtaining control information based on at least either of said audio signal or video signal;

controlling pre-processing of the audio signal based on the control information obtained from the video signal; and/or

controlling pre-processing of the video signal based on the control information obtained from the audio signal.

2. A method according to claim 1 , wherein

the pre-processing of the audio signal is one of the following: noise suppression, voice level adjustment, adjustment of dynamic range of voice, directing a microphone beamform of a multi-microphone arrangement towards an audio source.

3. A method according to claim 1 , the method further comprising

determining a priority value for at least one audio source appearing on a video scene represented by the video signal in proportion to an image area covered by the audio source in said video scene; and

adjusting the pre-processing of the audio signal according to the priority value such that an audio component originating from an audio source covering largest image area of the video scene is emphasized in the pre-processing.

4. A method according to claim 1 , the method further comprising

determining a priority value for at least one audio source appearing on a video scene represented by the video signal in proportion to an image area covered by the audio source in said video scene; and

adjusting the pre-processing of the audio signal according to the priority value such that an audio component contributing less to an overall video scene is de-emphasized in the pre-processing.

5. A method according to claim 1 , the method further comprising

detecting at least a part of a human face in a video scene represented by the video signal; and

adjusting the pre-processing of the audio signal in proportion to an image area covered by the human face in said video scene.

6. A method according to claim 5 , wherein said pre-processing of the audio signal is noise suppression, and the method further comprises

adjusting attenuation of background noise in proportion to the image area covered by the human face in said video scene.

7. A method according to claim 1 , the method further comprising

obtaining control information for the audio pre-processor control signal from a plurality of points of a processing chain of the video signal, said plurality of points being located in at least one of the following points: prior to video signal pre-processing, prior to video signal encoding, during video encoding and the encoded parameter values of the video signal.

8. A method according to claim 1 , wherein

the pre-processing of the video signal is one of the following: smoothening details of image frames, adjustment of dynamic range of colours, reducing a colour gamut of the video signal or removing less essential parts of the video signal.

9. A method according to claim 1 , the method further comprising

determining a priority value for at least one object appearing on a video scene represented by the video signal in proportion to an audio component contributed by said object to an overall audio scene; and

adjusting the pre-processing of the video signal according to the priority value such that an object contributing less to an overall audio scene is de-emphasized in the pre-processing.

10. An apparatus comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

capture multimedia content, said multimedia content comprising at least an audio signal and a video signal;

obtain control information based on at least either of said audio signal or video signal;

control pre-processing of the audio signal based on the control information obtained from the video signal; and/or

control pre-processing of the video signal based on the control information obtained from the audio signal.

11. An apparatus according to claim 10 , wherein

the pre-processing of the audio signal is one of the following: noise suppression, voice level adjustment, adjustment of dynamic range of voice, directing a microphone beamform of a multi-microphone arrangement towards an audio source.

12. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

determine a priority value for at least one audio source appearing on a video scene represented by the video signal in proportion to an image area covered by the audio source in said video scene; and

adjust the pre-processing of the audio signal according to the priority value such that an audio component originating from an audio source covering largest image area of the video scene is emphasized in the pre-processing.

13. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

determine a priority value for at least one audio source appearing on a video scene represented by the video signal in proportion to an image area covered by the audio source in said video scene; and

adjust the pre-processing of the audio signal according to the priority value such that an audio component contributing less to an overall video scene is de-emphasized in the pre-processing.

14. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

detect at least a part of a human face in a video scene represented by the video signal; and

adjust the pre-processing of the audio signal in proportion to an image area covered by the human face in said video scene.

15. An apparatus according to claim 14 , wherein said pre-processing of the audio signal is noise suppression, and the apparatus further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

adjust attenuation of background noise in proportion to the image area covered by the human face in said video scene.

16. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

obtain control information for the audio pre-processor control signal from a plurality of points of a processing chain of the video signal, said plurality of points being located in at least one of the following points: prior to video signal pre-processing, prior to video signal encoding, during video encoding and the encoded parameter values of the video signal.

17. An apparatus according to claim 10 , wherein

the pre-processing of the video signal is one of the following: smoothening details of image frames, adjustment of dynamic range of colours, reducing a colour gamut of the video signal or removing less essential parts of the video signal.

18. An apparatus according to claim 10 , further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

determine a priority value for at least one object appearing on a video scene represented by the video signal in proportion of an audio component contributed by said object to an overall audio scene; and

adjust the pre-processing of the video signal according to the priority value such that an object contributing less to an overall audio scene is de-emphasized in the pre-processing.

19. A non-transitory computer readable storage medium tangibly encoded with a computer program executable, which when executed by a processor of an apparatus, causes the apparatus to perform:

capturing multimedia content, said multimedia content comprising at least an audio signal and a video signal;

obtaining control information based on at least either of said audio signal or video signal;

controlling pre-processing of the audio signal based on the control information obtained from the video signal; and/or

controlling pre-processing of the video signal based on the control information obtained from the audio signal.

20. An apparatus comprising:

means for capturing multimedia content, said multimedia content comprising at least an audio signal and a video signal;

means for obtaining control information based on at least either of said audio signal or video signal;

means for controlling pre-processing of the audio signal based on the control information obtained from the video signal; and/or

means for controlling pre-processing of the video signal based on the control information obtained from the audio signal.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2015
From: NOKIA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 035305/0622 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2014
From: JÄRVINEN, KARI
To: NOKIA CORPORATION
Reel/Frame 032910/0057 →