Lyrics analyzer
A lyrics analyzer generates tags and explicitness indicators for a set of tracks. These tags may indicate the genre, mood, occasion, or other features of each track. The lyrics analyzer does so by generating an n-dimensional vector relating to a set of topics extracted from the lyrics and then using those vectors to train a classifier to determine whether each tag applies to each track. The lyrics analyzer may also generate playlists for a user based on a single seed song by comparing the lyrics vector or the lyrics and acoustics vectors of the seed song to other songs to select songs that closely match the seed song. Such a playlist generator may also take into account the tags generated for each track.
1. A method, comprising:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating a first explicitness indicator for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
training a second, different, classifier based on the lyrics vectors of at least one of the training tracks;
generating a second explicitness indicator for at least one of the plurality of tracks by applying each second classifier to the lyrics vectors of at least one of the plurality of tracks; and
generating a final explicitness indicator based on the combination of the first explicitness indicator and the second explicitness indicator.
2. The method according to claim 1 , wherein the generative statistical model is a Latent Dirichlet Allocation (LDA) model.
3. The method according to claim 1 , further comprising:
processing the lyrics of the plurality of tracks, wherein the processing including: (i) white-space standardizing, (ii) lowercasing, (iii) removing stopwords, (iv) removing punctuation, (v) lemmatizing, (vi) removing character repetition based on a dictionary, or (vii) any combination of (i), (ii), (iii), (iv), (v), and (vi).
4. The method according to claim 1 , further comprising:
displaying one or more explicitness indicators generated for at least one of the plurality of tracks.
5. The method according to claim 1 , wherein the classifier is based on at least one of (i) a Random Forest classifier, (ii) a Naïve Bayes classifier, (iii) a Decision Tree classifier, (iv) a Stochastic Gradient Descent classifier, (v) a K-Nearest Neighbors classifier, or (vi) any combination of (i), (ii), (iii), (iv), and (v).
6. The method according to claim 1 , wherein generating a final explicitness indicator is based on the union of the first and second explicitness indicators.
7. The method according to claim 1 , wherein generating a final explicitness indicator is based on the intersection of the first and second explicitness indicators.
8. The method according to claim 1 , further comprising:
receiving, for at least one of the plurality of tracks, an acoustic vector;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying the classifier to the hybrid vectors of at least one of the plurality of tracks.
9. The method according to claim 8 , further comprising:
training a second, different, classifier for each of a set of tags based on the lyrics vectors of at least one of the training tracks for each tag;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks for each tag; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying each classifier to the hybrid vector of at least one of the plurality of tracks.
10. The method according to claim 1 , wherein the indicator of explicitness is generated based on titles of playlists containing the associated training tracks.
11. The method according to claim 1 , wherein indicator of explicitness is generated based on expert analysis of the associated training tracks.
12. A system comprising:
a computer-readable memory storing executable instructions; and
one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least perform:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating a first explicitness indicator for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
training a second, different, classifier based on the lyrics vectors of at least one of the training tracks;
generating a second explicitness indicator for at least one of the plurality of tracks by applying each second classifier to the lyrics vectors of at least one of the plurality of tracks; and
generating a final explicitness indicator based on the combination of the first explicitness indicator and the second explicitness indicator.
13. The system according to claim 12 , wherein the generative statistical model is a Latent Dirichlet Allocation (LDA) model.
14. The system according to claim 12 , wherein the one or more processors are further programmed by the executable instructions to perform:
processing the lyrics of the plurality of tracks, wherein the processing including: (i) white-space standardizing, (ii) lowercasing, (iii) removing stopwords, (iv) removing punctuation, (v) lemmatizing, (vi) removing character repetition based on a dictionary, or (vii) any combination of (i), (ii), (iii), (iv), (v), and (vi).
15. The system according to claim 12 , wherein the one or more processors are further programmed by the executable instructions to perform:
displaying one or more explicitness indicators generated for at least one of the plurality of tracks.
16. The system according to claim 12 , wherein the classifier is based on at least one of (i) a Random Forest classifier, (ii) a Naïve Bayes classifier, (iii) a Decision Tree classifier, (iv) a Stochastic Gradient Descent classifier, (v) a K-Nearest Neighbors classifier, or (vi) any combination of (i), (ii), (iii), (iv), and (v).
17. The system according to claim 12 , wherein generating a final explicitness indicator is based on the union of the first and second explicitness indicators.
18. The system according to claim 12 , wherein generating a final explicitness indicator is based on the intersection of the first and second explicitness indicators.
19. The system according to claim 12 , wherein the one or more processors are further programmed by the executable instructions to perform:
receiving, for at least one of the plurality of tracks, an acoustic vector;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying the classifier to the hybrid vectors of at least one of the plurality of tracks.
20. The system according to claim 19 , wherein the one or more processors are further programmed by the executable instructions to perform:
training a second, different, classifier for each of a set of tags based on the lyrics vectors of at least one of the training tracks for each tag;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks for each tag; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying each classifier to the hybrid vector of at least one of the plurality of tracks.
21. The system according to claim 12 , wherein the indicator of explicitness is generated based on titles of playlists containing the associated training tracks.
22. The system according to claim 12 , wherein indicator of explicitness is generated based on expert analysis of the associated training tracks.
23. A non-transitory computer-readable medium having stored thereon one or more sequences of instructions for causing one or more processors to perform:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating a first explicitness indicator for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
training a second, different, classifier based on the lyrics vectors of at least one of the training tracks;
generating a second explicitness indicator for at least one of the plurality of tracks by applying each second classifier to the lyrics vectors of at least one of the plurality of tracks; and
generating a final explicitness indicator based on the combination of the first explicitness indicator and the second explicitness indicator.
24. A method, comprising:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating a first explicitness indicator for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
receiving, for at least one of the plurality of tracks, an acoustic vector;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying the classifier to the hybrid vectors of at least one of the plurality of tracks.
25. The method according to claim 24 , wherein the generative statistical model is a Latent Dirichlet Allocation (LDA) model.
26. The method according to claim 24 , further comprising:
processing the lyrics of the plurality of tracks, wherein the processing including: (i) white-space standardizing, (ii) lowercasing, (iii) removing stopwords, (iv) removing punctuation, (v) lemmatizing, (vi) removing character repetition based on a dictionary, or (vii) any combination of (i), (ii), (iii), (iv), (v), and (vi).
27. The method according to claim 24 , further comprising:
displaying one or more explicitness indicators generated for at least one of the plurality of tracks.
28. The method according to claim 24 , wherein the classifier is based on at least one of (i) a Random Forest classifier, (ii) a Naïve Bayes classifier, (iii) a Decision Tree classifier, (iv) a Stochastic Gradient Descent classifier, (v) a K-Nearest Neighbors classifier, or (vi) any combination of (i), (ii), (iii), (iv), and (v).
29. The method according to claim 24 , further comprising:
training a second, different, classifier based on the lyrics vectors of at least one of the training tracks;
generating a second explicitness indicator for at least one of the plurality of tracks by applying each second classifier to the lyrics vectors of at least one of the plurality of tracks; and
generating a final explicitness indicator based on the combination of the first explicitness indicator and the second explicitness indicator.
30. The method according to claim 29 , wherein generating a final explicitness indicator is based on the union of the first and second explicitness indicators.
31. The method according to claim 29 , wherein generating a final explicitness indicator is based on the intersection of the first and second explicitness indicators.
32. The method according to claim 24 , further comprising:
training a second, different, classifier for each of a set of tags based on the lyrics vectors of at least one of the training tracks for each tag;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks for each tag; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying each classifier to the hybrid vector of at least one of the plurality of tracks.
33. The method according to claim 24 , wherein the indicator of explicitness is generated based on titles of playlists containing the associated training tracks.
34. The method according to claim 24 , wherein indicator of explicitness is generated based on expert analysis of the associated training tracks.
35. A system comprising:
a computer-readable memory storing executable instructions; and
one or more processors in communication with the computer-readable memory, wherein the one or more processors are programmed by the executable instructions to at least perform:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating a first explicitness indicator for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
receiving, for at least one of the plurality of tracks, an acoustic vector;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying the classifier to the hybrid vectors of at least one of the plurality of tracks.
36. The system according to claim 35 , wherein the generative statistical model is a Latent Dirichlet Allocation (LDA) model.
37. The system according to claim 35 , further comprising:
processing the lyrics of the plurality of tracks, wherein the processing including: (i) white-space standardizing, (ii) lowercasing, (iii) removing stopwords, (iv) removing punctuation, (v) lemmatizing, (vi) removing character repetition based on a dictionary, or (vii) any combination of (i), (ii), (iii), (iv), (v), and (vi).
38. The system according to claim 35 , further comprising:
displaying one or more explicitness indicators generated for at least one of the plurality of tracks.
39. The system according to claim 35 , wherein the classifier is based on at least one of (i) a Random Forest classifier, (ii) a Naïve Bayes classifier, (iii) a Decision Tree classifier, (iv) a Stochastic Gradient Descent classifier, (v) a K-Nearest Neighbors classifier, or (vi) any combination of (i), (ii), (iii), (iv), and (v).
40. The system according to claim 35 , wherein the one or more processors are further programmed by the executable instructions to perform:
training a second, different, classifier based on the lyrics vectors of at least one of the training tracks;
generating a second explicitness indicator for at least one of the plurality of tracks by applying each second classifier to the lyrics vectors of at least one of the plurality of tracks; and
generating a final explicitness indicator based on the combination of the first explicitness indicator and the second explicitness indicator.
41. The system according to claim 40 , wherein generating a final explicitness indicator is based on the union of the first and second explicitness indicators.
42. The system according to claim 40 , wherein generating a final explicitness indicator is based on the intersection of the first and second explicitness indicators.
43. The system according to claim 35 , wherein the one or more processors are further programmed by the executable instructions to perform:
training a second, different, classifier for each of a set of tags based on the lyrics vectors of at least one of the training tracks for each tag;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks for each tag; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying each classifier to the hybrid vector of at least one of the plurality of tracks.
44. The system according to claim 35 , wherein the indicator of explicitness is generated based on titles of playlists containing the associated training tracks.
45. The system according to claim 35 , wherein indicator of explicitness is generated based on expert analysis of the associated training tracks.
46. A non-transitory computer-readable medium having stored thereon one or more sequences of instructions for causing one or more processors to perform:
receiving a plurality of tracks at an information storage and retrieval platform via an electronic communication from a datastore of tracks, a plurality of the tracks including lyrics;
extracting n topics summarizing the lyrics of the plurality of tracks, each topic consisting of a plurality of words, where n is an integer;
generating, for each of the plurality of tracks, an n-dimensional vector using a generative statistical model based on the association of the lyrics of the track with the n topics, thereby generating a plurality of n-dimensional lyrics vectors;
receiving a set of training tracks from among the plurality of tracks, each training track having an indicator of explicitness;
training a classifier for determining whether a track is explicit, based on the lyrics vectors of each of the training tracks;
generating an indicator of explicitness for each of the plurality of tracks by applying the classifier to the lyrics vectors of each of the plurality of tracks;
receiving, for at least one of the plurality of tracks, an acoustic vector;
calculating a hybrid vector for at least one of the plurality of tracks by concatenating the lyrics vector and the acoustic vector for at least one of the plurality of tracks;
wherein training a classifier is based on the hybrid vectors of at least one of the training tracks; and
wherein generating an indicator of explicitness for at least one of the plurality of tracks is accomplished by applying the classifier to the hybrid vectors of at least one of the plurality of tracks.