IP Library Granted Patent US 9,838,737
Granted Patent B2
US 9,838,737 · App. 15/147,040 · Granted Dec 5, 2017

Filtering wind noises in video content

Inventors: Elad Eban (Sunnyvale, CA); Aren Jansen (Los Altos, CA); Sourish Chaudhuri (San Francisco, CA)
Assignee: Google Inc.
H04N21/4398H04N5/602H04N21/4394
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,838,737
App. No.
15/147,040
Granted
Dec 5, 2017
Kind
B2
Abstract

Implementations disclose filtering wind noises in video content. A method includes receiving video content comprising an audio component and a video component, detecting, by a processing device, occurrence of a wind noise artifact in a segment of the audio component, identifying duration of the wind noise artifact and intensity of the wind noise artifact, selecting, by the processing device, a wind noise replacement operation based on the identified duration and intensity of the wind noise artifact, and applying, by the processing device, the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment.

Claims (51)

1. A method comprising:

receiving video content comprising an audio component and a video component;

detecting, by a processing device, occurrence of a wind noise artifact in a segment of the audio component;

identifying duration of the wind noise artifact and intensity of the wind noise artifact, wherein the duration comprises a time length of the wind noise artifact in the segment and wherein the intensity is based on frequency values of the wind noise artifact;

selecting, by the processing device, a wind noise replacement operation based on the identified duration and intensity of the wind noise artifact; and

applying, by the processing device, the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment.

2. The method of claim 1 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

3. The method of claim 1 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

4. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the segment with silence.

5. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

6. The method of claim 1 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

7. The method of claim 6 , wherein a user is prompted to select the different audio component.

8. The method of claim 1 , wherein selecting the wind noise replacement operation further comprises:

deriving a plurality of signals from the identified duration and the identified intensity of the wind noise artifact;

mapping the derived signals to a corresponding set of threshold values; and

selecting the wind noise replacement operation that corresponds to the set of thresholds values mapped to the derived signals.

9. The method of claim 8 , wherein the derived signals comprise one or more frequencies of the wind noise artifact and a signal-to-noise ratio corresponding to the wind noise artifact.

10. The method of claim 1 , wherein machine learning is used to detect the occurrence of the wind noise artifact.

11. The method of claim 1 , wherein deep learning is used to detect the occurrence of the wind noise artifact.

12. The method of claim 1 , wherein spectrogram analysis is used to detect the occurrence of the wind noise artifact.

13. A system comprising:

a memory; and

a processing device coupled to the memory, wherein the processing device is to:

extract an audio component from video content;

analyze the audio component to identify occurrence of a wind noise artifact in a segment of the audio component;

identify characteristics of the segment, wherein the characteristics comprise a duration and an intensity of the wind noise artifact in the segment, the duration comprising a time length of the wind noise artifact in the segment and the intensity is based on frequency values of the wind noise artifact;

select a wind noise replacement operation based on the identified characteristics; and

remove the wind noise artifact from the segment via application of the selected wind noise replacement operation to the segment.

14. The system of claim 13 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

15. The system of claim 13 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

16. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

17. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the segment with silence.

18. The system of claim 13 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

19. The system of claim 13 , wherein the characteristics of the segment further comprise at least one of time markers of the segment or an amplitude of the segment.

20. The system of claim 13 , wherein the processing device to select the wind noise replacement operation further comprises:

deriving a plurality of signals from the identified characteristics;

mapping the derived signals to corresponding set of threshold values; and

selecting the wind noise replacement operation that corresponds to the set of thresholds values mapped to the derived signals.

21. A non-transitory machine-readable storage medium storing instructions which, when executed, cause a processing device to perform operations comprising:

extracting an audio component from video content;

detecting, by the processing device, occurrence of a wind noise artifact in a segment of the audio component;

identifying, by the processing device, characteristics of the wind noise artifact, wherein the characteristics comprise a duration and an intensity of the wind noise artifact in the segment, the duration comprising a time length of the wind noise artifact in the segment and the intensity is based on frequency values of the wind noise artifact;

selecting, by the processing device, a wind noise replacement operation based on the identified characteristics of the wind noise artifact;

applying the selected wind noise replacement operation to the segment of the audio component to remove the wind noise artifact from the segment and generate a modified audio component;

combining the modified audio component with the video content; and

transmitting, by the processing device, the video content to a content sharing platform.

22. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises in-filling the segment with an interpolation of audio signal extracted from other segments of the audio component surrounding the segment.

23. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises filtering the segment to remove the wind noise artifact from the segment.

24. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the segment with silence.

25. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the wind noise artifact with audio signal extracted from another segment of the audio component.

26. The non-transitory machine-readable storage medium of claim 21 , wherein the wind noise replacement operation comprises replacing the audio component with a different audio component.

Assignments (2)
CHANGE OF NAME Recorded Dec 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044695/0115 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2016
From: EBAN, ELAD; JANSEN, AREN; CHAUDHURI, SOURISH
To: GOOGLE INC.
Reel/Frame 038476/0478 →
Continuity (1)
Related Publication 20170324990A1 · Nov 9, 2017