Identifying shifts in audio content via machine learning
A method and system for identifying the beginning and ending of songs via a machine learning analysis. A machine learning model analyzes streaming audio (such as a radio broadcast) in overlapping, 3-second samples. Each sample is labeled into groups such as “song,” “talk,” “commercial” and “transition.” Based on the location of the transition samples, an exact second a given song begins and ends in the audio stream is derivable. The model further identifies when two songs shift between one another.
1 . A method for classifying segments of an audio stream of a radio program comprising:
labeling a plurality of consecutive audio samples of the audio stream with a trained machine learning model via successive inspection, the trained machine learning model configured to output a label corresponding to each audio sample indicating whether each respective audio sample is a song portion, a talk portion, or a commercial portion of the audio stream resulting in a sequence of labels;
executing a first probabilistic correction on the sequence of labels based on patterns represented within the sequence of labels and resulting in a corrected sequence of labels;
identifying a set of consecutive audio samples as having song portion labels; and
determining, via the trained machine learning model, whether the set of consecutive audio samples are a matching song.
2 . The method of claim 1 , further comprising:
in response to identification that the set of consecutive audio samples belong to different songs, determining, via the trained machine learning model, a transition time between two different songs through use of consecutive overlapping audio samples.
3 . The method of claim 2 , wherein determining the transition time between the two different songs includes executing a second probabilistic correction on a sequence of comparisons of contiguous audio samples.
4 . The method of claim 2 , wherein the transition time between the two different songs further comprises:
comparing a set of contiguous audio samples of the plurality of audio samples.
5 . The method of claim 1 , wherein the plurality of consecutive audio samples are overlapping.
6 . The method of claim 2 , wherein said determining the transition time further includes:
inserting a marker at an end of a song where the consecutive audio samples transition between songs.
7 . The method of claim 1 , wherein the successive inspection of consecutive audio samples further comprises:
advancing a frame of inspection by a temporal period that is shorter than a temporal length of each audio sample.
8 . The method of claim 6 , wherein the successive inspections overlap by 1 second.
9 . A computing device for classifying segments of an audio stream of a radio program comprising:
a processor; and
a non-transitory computer-readable medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations including:
label a plurality of consecutive audio samples of the audio stream with a trained machine learning model via successive inspection, the trained machine learning model configured to output a label corresponding to each audio sample indicating whether each respective audio sample is a song portion, a talk portion, or a commercial portion of the audio stream resulting in a sequence of labels;
execute a first probabilistic correction on the sequence of labels based on patterns represented within the sequence of labels and resulting in a corrected sequence of labels;
identify a set of consecutive audio samples as having song portion labels; and
determine, via the trained machine learning model, whether the set of consecutive audio samples are a matching song.
10 . The computing device of claim 9 , the instructions further comprising:
in response to identification that the set of consecutive audio samples belong to different songs, determining, via the trained machine learning model, a transition time between two different songs through use of consecutive overlapping audio samples.
11 . The computing device of claim 10 , wherein determining the transition time between the two different songs includes executing a second probabilistic correction on a sequence of comparisons of contiguous audio samples.
12 . The computing device of claim 10 , wherein the transition time between the two different songs further comprises:
comparing a set of contiguous audio samples of the plurality of audio samples.
13 . The computing device of claim 9 , wherein the plurality of consecutive audio samples are overlapping.
14 . The computing device of claim 10 , wherein said determining the transition time further includes:
inserting a marker at an end of a song where the consecutive audio samples transition between songs.
15 . A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processor to perform operations for classifying segments of an audio stream of a radio program comprising:
labeling a plurality of consecutive audio samples of the audio stream with a trained machine learning model via successive inspection, the trained machine learning model configured to output a label corresponding to each audio sample indicating whether each respective audio sample is a song portion, a talk portion, or a commercial portion of the audio stream resulting in a sequence of labels;
executing a first probabilistic correction on the sequence of labels based on patterns represented within the sequence of labels and resulting in a corrected sequence of labels;
identifying a set of consecutive audio samples as having song portion labels; and
determining, via the trained machine learning model, whether the set of consecutive audio samples are a matching song.
16 . The non-transitory computer-readable medium of claim 15 , the instructions further comprising:
in response to identification that the set of consecutive audio samples belong to different songs, determining, via the trained machine learning model, a transition time between two different songs through use of consecutive overlapping audio samples.
17 . The non-transitory computer-readable medium of claim 16 , wherein determining the transition time between the two different songs includes executing a second probabilistic correction on a sequence of comparisons of contiguous audio samples.
18 . The non-transitory computer-readable medium of claim 16 , wherein the transition time between the two different songs further comprises:
comparing a set of contiguous audio samples of the plurality of audio samples.
19 . The non-transitory computer-readable medium of claim 15 , wherein the plurality of consecutive audio samples are overlapping.
20 . The non-transitory computer-readable medium of claim 16 , wherein said determining the transition time further includes:
inserting a marker at an end of a song where the consecutive audio samples transition between songs.