Vibrato detection modules in a system for automatic transcription of sung or hummed melodies
View Patent ↗The technology disclosed relates to audio signal processing. It includes a series of modules that individually are useful to solve audio signal processing problems. Among the problems addressed are buzz removal, selecting a pitch candidate among pitch candidates based on local continuity of pitch and regional octave consistency, making small adjustments in pitch, ensuring that a selected pitch is consistent with harmonic peaks, determining whether a given frame or region of frames includes harmonic, voiced signal, extracting harmonics from voice signals and detecting vibrato. One environment in which these modules are useful is transcribing singing or humming into a symbolic melody. Another environment that would usefully employ some of these modules is speech processing. Some of the modules, such as buzz removal, are useful in many other environments as well.
1. A method of vibrato detection applied to a sequence of detected pitches, the method including:
processing electronically a sequence of detected pitches for frames and estimating a rate and a pitch depth of oscillations in the sequence;
comparing the estimated rate and pitch depth to a predetermined vibrato detection envelope and determining whether the sequence of detected pitches would be perceived as vibrato;
wherein the predetermined vibrato detection envelope maps combinations of a dominant pitch variation rate and a pitch depth at the dominant pitch on a perceptual basis to whether the combinations are likely to be perceived by a listener as vibrato;
repeatedly determining vibrato perception of successive sequences of frames and repeating the processing and comparing actions; and
outputting data regarding whether the successive sequences would be perceived as vibrato.
2. The method of claim 1 , wherein the estimating of the rate and the pitch depth of oscillations further includes:
applying a zero-padded FFT to the sequence of detected pitches, producing an FFT output including raw rate and pitch depth data for oscillations in the sequence; and
interpolating the rate and pitch depth of oscillation centered on at least one peak in the FFT output to produce the estimated rate and pitch depth.
3. The method of claim 2 , wherein the interpolation is a quadratic interpolation.
4. The method of claim 2 , further including the action of excluding from vibrato perception those sequences of frames in which a plurality of peaks in the FFT output indicate a wave form that would not be perceived as vibrato.
5. The method of claim 2 ,
wherein the a median magnitude value is subtracted from the sequence of detected pitches before the applying of the zero-padded FFT,
further including the action of excluding from vibrato perception those sequences of frames in which a DC component in the FFT output indicates a new tone that would not be perceived as vibrato.
6. The method of claim 1 , further including, after repeatedly determining vibrato perception of the successive sequences, filtering out isolated sequences of vibrato that persist for fewer than a predetermined vibrato streak length of successive sequences.
7. An electronic signal processing component for detecting vibrato in frames that represent an audio signal, the component including:
an input port adapted to receive a stream of data frames including detected pitches;
an FFT processor coupled to the input that processes sequences of data frames in the stream and estimates rate and pitch depth of oscillations in pitch;
a comparison processor including data representing an envelope of combinations of rates and pitch depths of oscillation that would be perceived by listeners as vibrato, the comparison processor coupled to the estimates of rate and pitch depth of oscillations in pitch and operative to compare the estimates to the data representing the envelope;
wherein the envelope of combinations maps combinations of a dominant pitch variation rate and a pitch depth at the dominant pitch on a perceptual basis to whether the combinations are likely to be perceived by a listener as vibrato;
an output port coupled to the comparison processor that outputs results of the comparisons.
8. The component of claim 7 , wherein the FFT processor is implemented using a digital signal processor (DSP).
9. The component of claim 7 , wherein the FFT processor is implemented using software running on a general purpose central processing unit (hereinafter “CPU”) and the input and output ports are software running on the CPU.
10. The component of claim 7 , wherein the FFT processor is implemented using a gate array.
11. The component of claim 7 , wherein:
the FFT processor applies a zero-padded FFT to the detected pitches the sequence of frames; and
the comparison processor interpolates the rate and pitch depth of oscillation centered on at least one peak in output from the FFT processor to produce the estimates of rate and pitch depth of oscillation.
12. The component of claim 7 , further including a first exclusion filter coupled to output of the FFT processor that senses when a plurality of peaks in the estimates indicate a non-sinusoidal wave form that would not be perceived as vibrato and excludes the corresponding sequence of frames from being reported as containing vibrato.
13. The component of claim 7 , further including:
a normalizing component that subtracts from the sequence of detected pitches a median magnitude value before the sequence is processed by the FFT processor; and
a second exclusion filter coupled to output of the FFT processor that senses when a DC component in the estimates indicates a new tone that would not be perceived as vibrato and excludes the corresponding sequence of frames from being reported as containing vibrato.
14. The component of claim 13 , further including:
a normalizing component that subtracts from the sequence of detected pitches a median magnitude value before the sequences are processed by the FFT processor; and
a second exclusion filter coupled to output of the FFT processor that senses when a plurality a DC component in the estimates indicates a new tone that would not be perceived as vibrato and excludes the corresponding sequence from being reported as containing vibrato.
15. An electronic signal processing component for detecting vibrato in frames that represent an audio signal, the component including:
an input port adapted to receive a stream of data frames including detected pitches;
an FFT means for processing the sequences of data frames in the stream, coupled to the input port, and for estimating rate and pitch depth of oscillations in pitch;
a comparison means for evaluating whether estimated pitch variation rates and pitch depth at a dominant pitch would be perceived as vibrato, based on comparison to data representing a psychoacoustic envelope of perceived vibrato; and
an output port to which the comparison means reports results.
16. The component of claim 15 , further including:
first exclusion means, coupled to output of the FFT means, for detecting and excluding from containing vibrato the sequences in which the FFT output includes a plurality of peaks that indicate a non-sinusoidal wave form that would not be perceived as vibrato;
second exclusion means, also coupled to output of the FFT means, for detecting and excluding from containing vibrato the sequences in which a DC component in the estimates indicates a new tone that would not be perceived as vibrato.
17. A computer readable non-volatile storage medium including program instructions for carrying out a method including:
processing a sequence of detected pitches for frames and estimating a rate and a pitch depth of oscillations in the sequence;
comparing the estimated rate and pitch depth to a predetermined vibrato detection envelope and determining whether the sequence of detected pitches would be perceived as vibrato;
wherein the predetermined vibrato detection envelope maps combinations of a dominant pitch variation rate and a pitch depth at the dominant pitch on a perceptual basis to whether the combinations are likely to be perceived by a listener as vibrato;
repeatedly determining vibrato perception of successive sequences of frames by indexing through the sequences of frame and repeating the processing and comparing actions; and
outputting data regarding whether the successive sequences would be perceived as vibrato.
18. The computer readable non-volatile storage medium of claim 17 , wherein at least some of the program instructions are adapted to run on a digital signal processor (hereinafter “DSP”).
19. The computer readable non-volatile storage medium of claim 17 , wherein the program instructions are adapted to produce a gate array.