Audio processing for detecting occurrences of loud sound characterized by brief audio bursts

1. A method for identifying a boundary of a highlight of audiovisual content depicting an event, the method comprising:

at a data store, storing audio data depicting at least part of the event;

at a processor, automatically analyzing the audio data to detect one or more audio events representing one or more occurrences to be included in the highlight, wherein each audio event is characterized by a high-energy audio burst of limited duration; and

at the processor, designating a time index, within the audiovisual content, defining the boundary, the boundary comprising one of a beginning of the highlight and an end of the highlight;

wherein automatically analyzing the audio data to detect the one or more audio events comprises:

performing digital filtering of the audio data for at least one of a time-domain analysis and a frequency-domain analysis;

performing the time-domain analysis and the frequency-domain analysis to detect occurrences of high energy audio events in the audio data and to detect time spacing between the high energy audio events; and

skipping the detected occurrences of the high energy audio events with time spacing below a minimum time threshold.

2. The method of claim 1 , wherein the audiovisual content comprises a television broadcast.

3. The method of claim 1 , wherein the audiovisual content comprises an audiovisual stream, and wherein the method further comprises, prior to storing the audio data depicting at least part of the event, extracting the audio data from the audiovisual stream.

4. The method of claim 1 , wherein the audiovisual content comprises stored audiovisual content, and wherein the method further comprises, prior to storing the audio data depicting at least part of the event, extracting the audio data from the stored audiovisual content.

5. The method of claim 1 , wherein:

the event comprises a sporting event; and

the highlight depicts a portion of the sporting event deemed to be of particular interest to at least one user.

6. The method of claim 5 , further comprising, at an output device, playing at least one of the audiovisual content and the highlight.

7. The method of claim 1 , further comprising, prior to detecting the audio events, pre-processing the audio data by resampling the audio data to a desired sampling rate.

8. The method of claim 1 , further comprising, prior to detecting the audio events, pre-processing the audio data by filtering the audio data to perform at least one of:

reducing noise; and

selecting a spectral band of interest.

9. The method of claim 1 , wherein performing the time-domain analysis comprises:

selecting an analysis time window size;

selecting an analysis window overlap region size;

sliding an analysis time window along the audio data;

computing a normalized magnitude for window samples at each position of the analysis time window; and

calculating an average sample magnitude at each position of the analysis time window.

10. The method of claim 1 , further comprising:

processing the audio data to generate a spectrogram for the audio data; and

analyzing the audio data and the spectrogram in a joint time-frequency domain to identify audio events comprising distinct energy burst events detected in the time domain.

11. The method of claim 10 , wherein analyzing the audio data and the spectrogram in the joint time-frequency domain comprises:

constructing a 2-D diamond-shaped spectrogram area filter to facilitate detection and selection of pronounced time-frequency magnitude peaks;

sliding the area filter along time and frequency spectrogram axes;

checking a central peak magnitude against remaining peak magnitudes at each time-frequency position of the area filter;

retaining only central peak magnitudes that are greater than all other peak magnitudes at each time-frequency position of the area filter; and

populating a spectral event vector with all retained central peak magnitudes.

12. The method of claim 10 , further comprising, in the time domain and in the frequency domain, performing joint analysis of audio events detected in the time domain.

13. The method of claim 12 , further comprising:

determining a spectrogram time-spread range around each of the audio events; and

using the time-spread ranges for event qualifier computation.

14. The method of claim 13 , wherein using the time-spread ranges for event qualifier computation comprises:

counting spectral event vector elements positioned in the spectrogram time-spread range around the audio events detected in the time domain;

recording the spectral event vector elements as qualifiers for each of the audio events;

counting a number of spectrogram magnitude peaks within a time spread range to obtain a count; and

generating a revised event vector containing only time-domain event points at which the count is below a threshold.

15. The method of claim 14 , wherein using the time-spread ranges for event qualifier computation further comprises:

comparing the qualifier, associated with each of the audio events detected in the time domain, against a threshold;

suppressing all time-domain detected events with a qualifier above the threshold; and

generating a qualifier revised event vector.

16. The method of claim 15 , further comprising:

processing the qualifier revised event vector according to a schedule of minimal time distances between adjacent events; and

suppressing undesirable, redundant audio events to obtain a final desired event timeline for the event.

17. The method of claim 1 , further comprising automatically appending at least one of the audio events, the time index, and an indicator of each occurrence to metadata associated with the highlight.

18. The method of claim 1 , wherein the event comprises a sporting event.

19. The method of claim 18 , wherein the event comprises a tennis game, and each occurrence comprises a tennis serve.

20. The method of claim 1 , further comprising, prior to performing the at least one of the time-domain analysis and the frequency-domain analysis:

generating an array of audio spectrograms on chunks of the filtered audio data;

storing at least one time-frequency coefficient for each spectrogram; and

wherein at least one of the time-domain analysis and the frequency-domain analysis is performed using the stored time-frequency coefficients.

21. A non-transitory computer-readable medium for identifying a boundary of a highlight of audiovisual content depicting an event, comprising instructions stored thereon, that when performed by a processor, perform the steps of:

causing a data store to store audio data depicting at least part of the event;

automatically analyzing the audio data to detect one or more audio events representing one or more occurrences to be included in the highlight, wherein each audio event is characterized by a high-energy audio burst of limited duration; and

designating a time index, within the audiovisual content, defining the boundary, the boundary comprising one of a beginning of the highlight and an end of the highlight;

wherein automatically analyzing the audio data to detect the one or more audio events comprises:

performing digital filtering of the audio data for at least one of a time-domain analysis and a frequency-domain analysis;

performing the time-domain analysis and the frequency-domain analysis to detect occurrences of high energy audio events in the audio data and to detect time spacing between the high energy audio events; and

skipping the detected occurrences of the high energy audio events with time spacing below a minimum time threshold.

22. The non-transitory computer-readable medium of claim 21 , wherein:

the event comprises a sporting event; and

the highlight depicts a portion of the sporting event deemed to be of particular interest to at least one user.

23. The non-transitory computer-readable medium of claim 21 , further comprising instructions stored thereon, that when executed by a processor, prior to detection of the audio events:

pre-process the audio data prior to detecting the audio events by resampling the audio data to a desired sampling rate; and

pre-process the audio data by filtering the audio data to perform at least one of:

reducing noise; and

selecting a spectral band of interest.

24. The non-transitory computer-readable medium of claim 21 , wherein performing the time-domain analysis comprises:

selecting an analysis time window size;

selecting an analysis window overlap region size;

sliding an analysis time window along the audio data;

computing a normalized magnitude for window samples at each position of the analysis time window; and

calculating an average sample magnitude at each position of the analysis time window.

25. The non-transitory computer-readable medium of claim 21 , further comprising instructions stored thereon, that when executed by a processor, perform the steps of:

process the audio data to generate a spectrogram for the audio data; and

analyze the audio data and the spectrogram in a joint time-frequency domain to identify audio events comprising distinct energy burst events detected in the time domain.

26. The non-transitory computer-readable medium of claim 25 , wherein analyzing the audio data and the spectrogram in the joint time-frequency domain comprises:

constructing a 2-D diamond-shaped spectrogram area filter to facilitate detection and selection of pronounced time-frequency magnitude peaks;

sliding the area filter along time and frequency spectrogram axes;

checking a central peak magnitude against remaining peak magnitudes at each time-frequency position of the area filter;

retaining only central peak magnitudes that are greater than all other peak magnitudes at each time-frequency position of the area filter; and

populating a spectral event vector with all retained central peak magnitudes.

27. The non-transitory computer-readable medium of claim 25 , further comprising instructions stored thereon, that when executed by a processor, perform joint analysis, in the time domain and in the frequency domain, of audio events detected in the time domain.

28. The non-transitory computer-readable medium of claim 21 , wherein:

the event comprises a tennis game; and

each occurrence comprises a tennis serve.

29. The non-transitory computer-readable medium of claim 21 , further comprising instructions stored thereon, that when performed by a processor, perform the steps of, prior to performing the at least one of the time-domain analysis and the frequency-domain analysis:

generating an array of audio spectrograms on chunks of the filtered audio data;

storing at least one time-frequency coefficient for each spectrogram; and

wherein at least one of the time-domain analysis and the frequency-domain analysis is performed using the stored time-frequency coefficients.

30. A system for identifying a boundary of a highlight of audiovisual content depicting an event, the system comprising:

a data store configured to store audio data depicting at least part of the event; and

a processor, communicatively coupled to the data store, configured to:

automatically analyze the audio data to detect one or more audio events representing one or more occurrences to be included in the highlight, wherein each audio event is characterized by a high-energy audio burst of limited duration; and

designate a time index, within the audiovisual content, defining the boundary, the boundary comprising one of a beginning of the highlight and an end of the highlight;

wherein automatically analyzing the audio data to detect the one or more audio events comprises:

performing digital filtering of the audio data for at least one of a time-domain analysis and a frequency-domain analysis;

performing the time-domain analysis and the frequency-domain analysis to detect occurrences of high energy audio events in the audio data and to detect time spacing between the high energy audio events; and

skipping the detected occurrences of the high energy audio events with time spacing below a minimum time threshold.

31. The system of claim 30 , wherein:

the event comprises a sporting event; and

the highlight depicts a portion of the sporting event deemed to be of particular interest to at least one user.

32. The system of claim 30 , wherein the processor is further configured to, prior to detecting the audio events:

pre-process the audio data by resampling the audio data to a desired sampling rate; and

pre-process the audio data by filtering the audio data to perform at least one of:

reducing noise; and

selecting a spectral band of interest.

33. The system of claim 30 , wherein the processor is further configured to perform the time-domain analysis by:

selecting an analysis time window size;

selecting an analysis window overlap region size;

sliding an analysis time window along the audio data;

computing a normalized magnitude for window samples at each position of the analysis time window; and

calculating an average sample magnitude at each position of the analysis time window.

34. The system of claim 30 , wherein the processor is further configured to:

process the audio data to generate a spectrogram for the audio data; and

analyze the audio data and the spectrogram in a joint time-frequency domain to identify audio events comprising distinct energy burst event detected in the time domain.

35. The system of claim 34 , wherein the processor is further configured to analyze the audio data and the spectrogram in the joint time-frequency domain by:

constructing a 2-D diamond-shaped spectrogram area filter to facilitate detection and selection of pronounced time-frequency magnitude peaks;

sliding the area filter along time and frequency spectrogram axes;

checking a central peak magnitude against remaining peak magnitudes at each time-frequency position of the area filter;

retaining only central peak magnitudes that are greater than all other peak magnitudes at each time-frequency position of the area filter; and

populating a spectral event vector with all retained central peak magnitudes.

36. The system of claim 34 , wherein the processor is further configured to, in the time domain and in the frequency domain, perform joint analysis of audio events detected in the time domain.

37. The system of claim 30 , wherein:

the event comprises a tennis game; and

each occurrence comprises a tennis serve.

38. The system of claim 30 , wherein the processor is further configured to, prior to performing the at least one of the time-domain analysis and

the frequency-domain analysis:

generate an array of audio spectrograms on chunks of the filtered audio data;

cause the data store to store at least one time-frequency coefficient for each spectrogram; and

wherein at least one of the time-domain analysis and the frequency-domain analysis is performed using the stored time-frequency coefficients.