Methods and apparatus to segment audio and determine audio segment similarities

1. A method, comprising:

developing features characterizing beats of audio with a neural network, the neural network trained to develop the features characterizing beats of audio based on an anchor index beat corresponding to a selected beat in training audio;

computing a self-similarity matrix based on the features; and

identifying peaks in audio corresponding to beats of audio based on thresholding a crest factor at points in the self-similarity matrix, the peaks to be mapped to timestamps of the audio.

2. The method of claim 1 , wherein identifying the peaks includes filtering with a median filter.

3. The method of claim 1 , wherein identifying the peaks includes convolving the self-similarity matrix with a checkerboard kernel, wherein the checkerboard kernel represents a two-dimensional circularly-symmetric Gaussian function multiplied elementwise by a checkerboard change kernel.

4. The method of claim 1 , wherein identifying the peaks includes:

filtering the self-similarity matrix with a median filter to form a median-filtered self-similarity matrix;

convolving columns of the median-filtered self-similarity matrix with a smoothing function to form a smoothed matrix; and

computing absolute differences between rows of the smoothed matrix to form a novelty function vector, the novelty function vector including the beats of audio.

5. The method of claim 1 , wherein identifying the peaks includes:

filtering the self-similarity matrix with a median filter to form a median-filtered self-similarity matrix;

convolving the median-filtered self-similarity matrix with a checkerboard kernel to form a novelty function vector, the novelty function vector including the beats of audio; and

identifying the peaks based on the novelty function vector.

6. The method of claim 1 , wherein developing the features includes:

computing first time-frequency data for a first plurality of samples of the audio for a first plurality of time points between a first pair of beats; and

inputting the first time-frequency data into the neural network to develop the features, wherein the features include deep features of the neural network.

7. The method of claim 6 , further including detecting the first pair of beats in the audio.

8. The method of claim 6 , further including:

computing second time-frequency data for a second plurality of samples of the audio for a second plurality of time points between a second pair of beats; and

inputting the first time-frequency data together with the second time-frequency data into the neural network to develop the features.

9. The method of claim 1 , further including:

convolving (a) checkerboard kernel or (b) a smoothing function across the self-similarity matrix to form a one dimensional novelty function vector that includes novel beats of the beats of audio;

identifying a predetermined window length of beats in the one dimensional novelty function vector to be analyzed; and

calculating the crest factor for the predetermined window length of beats in the one dimensional novelty function vector to determine a peak metric, the crest factor indicative of a minimum acceptable peak value in the one dimensional novelty function vector to be identified as a peak.

10. An apparatus comprising:

a neural network to develop features characterizing beats of audio, the neural network trained to develop the features characterizing beats of audio based on an anchor index beat corresponding to a randomly selected beat in training audio;

a matrix former to form a self-similarity matrix based on the features; and

a peak picker to identify peaks in audio corresponding to the beats of audio based on thresholding a crest factor at points in the self-similarity matrix, the peaks to be mapped to timestamps of the audio.

11. The apparatus of claim 10 , further including a median filter to filter the self-similarity matrix to form a median-filtered self-similarity matrix.

12. The apparatus of claim 10 , further including:

a filter to filter the self-similarity matrix with a median filter to form a median-filtered self-similarity matrix;

a convolver to convolve columns of the median-filtered self-similarity matrix with a smoothing function to form a smoothed matrix;

a differencer to compute absolute differences between rows of the smoothed matrix to form a novelty function vector, the novelty function vector including the beats of audio; and

peak predictor to identify the peaks based on the novelty function vector.

13. The apparatus of claim 10 , further including:

a median filter to filter the self-similarity matrix to form a median-filtered self-similarity matrix;

a convolver to convolve the median-filtered self-similarity matrix with a checkerboard kernel to form a novelty function vector, the novelty function vector including the beats of audio; and

a peak predictor to identify the peaks based on the novelty function vector.

14. The apparatus of claim 10 , further including a frequency analyzer to develop first time-frequency data for a first plurality of samples of the audio for a first plurality of time points between a first pair of beats, wherein the neural network is to develop the features based on the first time-frequency data, wherein the features include deep features of the neural network.

15. The apparatus of claim 14 , wherein the frequency analyzer is to develop second time-frequency data for a second plurality of samples of the audio for a second plurality of time points between a second pair of beats, wherein the neural network is to develop second features based on the second time-frequency data.

16. The apparatus of claim 10 , further including:

a convolver to convolve (a) a checkerboard kernel or (b) a smoothing function over the self-similarity matrix to form a one dimensional novelty function vector that includes novel beats of the beats of audio; and

a peak picker to:

identify a predetermined window length of beats in the one dimensional novelty function vector to be analyzed; and

calculate the crest factor for the predetermined window length of beats in the one dimensional novelty function vector to determine a peak metric, the crest factor indicative of a minimum acceptable peak value in the one dimensional novelty function vector to be identified as a peak.

17. A non-transitory computer-readable storage medium comprising a set of instructions that, when executed, cause a processor to at least:

develop features characterizing beats of audio with a neural network, the neural network trained to develop the features characterizing beats of audio based on an anchor index beat corresponding to a selected beat in training audio;

compute a self-similarity matrix based on the features; and

identify peaks in audio corresponding to the beats of audio based on thresholding a crest factor at points in the self-similarity matrix, the peaks to be mapped to timestamps of the audio.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the set of instructions, when executed, cause the processor to identify the peaks based on filtering with a median filter.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the set of instructions, when executed, cause the processor to:

filter the self-similarity matrix with a median filter to form a median-filtered self-similarity matrix;

convolve columns of the median-filtered self-similarity matrix with a smoothing function to form a smoothed matrix;

compute absolute differences between rows of the smoothed matrix to form a novelty function vector, the novelty function vector including the beats of audio; and

identify the peaks based on the novelty function vector.

20. The non-transitory computer-readable storage medium of claim 17 , wherein the set of instructions, when executed, cause the processor to identify the peaks by:

filtering the self-similarity matrix with a median filter to form a median-filtered self-similarity matrix;

convolving the median-filtered self-similarity matrix with a checkerboard kernel to form a novelty function vector, the novelty function vector including the beats of audio; and

identifying the peaks based on the novelty function vector.

21. The non-transitory computer-readable storage medium of claim 17 , wherein the set of instructions, when executed, cause the processor to develop the features by:

computing first time-frequency data for a first plurality of samples of the audio for a first plurality of time points between a first pair of beats; and

inputting the first time-frequency data into the neural network to develop the features, wherein the features include deep features of the neural network.

22. The non-transitory computer-readable storage medium of claim 21 , wherein the set of instructions, when executed, cause the processor to:

compute second time-frequency data for a second plurality of samples of the audio for a second plurality of time points between a second pair of beats; and

input the first time-frequency data together with the second time-frequency data into the neural network to develop the features.

23. The non-transitory computer-readable storage medium of claim 17 , wherein the set of instructions, when executed, cause the processor to:

convolve (a) checkerboard kernel or (b) a smoothing function across the self-similarity matrix to form a one dimensional novelty function vector that includes novel beats of the beats of audio;

identify a predetermined window length of beats in the one dimensional novelty function vector to be analyzed; and

calculate the crest factor for the predetermined window length of beats in the one dimensional novelty function vector to determine a peak metric, the crest factor indicative of a minimum acceptable peak value in the one dimensional novelty function vector to be identified as a peak.