Systems and methods for identifying and remediating sound masking
Some embodiments of the invention are directed to enabling a user to easily identify the frequency range(s) at which sound masking occurs, and addressing the masking, if desired. In this respect, the extent to which a first stem is masked by one or more second stems in a frequency range may depend not only on the absolute value of the energy of the second stem(s) in the frequency range, but also on the relative energy of the first stem with respect to the second stem(s) in the frequency range. Accordingly, some embodiments are directed to modeling sound masking as a function of the energy of the stem being masked and of the relative energy of the masked stem with respect to the masking stem(s) in the frequency range, such as by modeling sound masking as loudness loss, a value indicative of the reduction in loudness of a stem of interest caused by the presence of one or more other stems in a frequency range.
1. A method comprising acts of:
(A) receiving a multi-track audio recording comprising a plurality of tracks synchronized over a time interval, a first track of the plurality of tracks corresponding to a first audio recording of the multi-track audio recording, a second track of the plurality of tracks corresponding to a second audio recording of the multi-track audio recording, the first track and the second track comprising sound produced by different instruments;
(B) determining a loudness of the first audio recording at each of a plurality of frequencies during a particular time period of the time interval;
(C) determining a loudness of the second audio recording at each of the plurality of frequencies during the particular time period;
(D) determining a partial loudness of the first audio recording at a first frequency of the plurality of frequencies during the particular time period, based at least in part on the loudness of the second audio recording, at the first frequency during the particular time period, determined in the act (C);
(E) determining a loudness loss for the first audio recording at the first frequency during the particular time period based at least in part on the partial loudness determined in the act (D); and
(F) creating a processed multi-track audio recording by applying one or more measures to the multi-track audio recording received in the act (A), based upon the loudness loss determined in the act (E).
2. The method of claim 1 , wherein the act (B) comprises transforming a digital representation of the first audio recording to the frequency domain, and generating a first excitation pattern using a result of the transforming.
3. The method of claim 2 , wherein the transforming comprises transforming the digital representation using a Fourier transform having a variable frequency resolution across the plurality of frequencies.
4. The method of claim 1 , wherein the act (B) comprises determining the loudness of the first audio recording at each of the plurality of frequencies using an audibility threshold and an extreme loudness threshold, the audibility threshold representing a minimum excitation level that a human ear perceives at each of the plurality of frequencies, the extreme loudness threshold representing a highest excitation level that a human ear tolerates at each of the plurality of frequencies.
5. The method of claim 1 , wherein the loudness determined in the act (B) comprises at least one of an instantaneous loudness, a time-smoothed long-/short-term loudness, and a specific loudness.
6. The method of claim 1 , wherein the act (D) comprises determining a partial loudness of the first audio recording at a first frequency of the plurality of frequencies during the time period based at least in part on (1) the loudness of the second audio recording during the given time period at the first frequency determined in the act (C) and (2) a masking threshold representing a minimum threshold of human audibility at each of the plurality of frequencies.
7. The method of claim 6 , wherein the masking threshold is based at least in part upon a psychoacoustic model.
8. The method of claim 1 , wherein the partial loudness determined in the act (D) comprises at least one of an instantaneous loudness, a time-smoothed long-/short-term loudness, and a specific loudness.
9. The method of claim 1 , wherein the act (B) comprises determining a loudness at the first frequency measured in phons, the act (D) comprises determining a partial loudness at the first frequency measured in phons, and the act (E) comprises determining a loudness loss by subtracting the partial loudness determined in the act (D) from the loudness determined in the act (A).
10. The method of claim 1 , wherein the act (F) comprises causing display of a graphical user interface depicting the loudness loss determined in the act (E), the graphical user interface enabling the user to provide input for at least partially remediating the loudness loss.
11. The method of claim 10 , wherein the input relates to one or more measures comprising filtering, mixing, equalizing, attenuating, amplifying, panning, compressing and phase rotating at least one of the first digital representation and the second digital representation at the first frequency.
12. The method of claim 1 , wherein the instrument producing the sound included in each of the first track and the second track is a musical instrument, human vocals, a microphone, or a computer system.
13. The method of claim 1 , wherein the act (F) comprises applying one or more measures specified via user input.
14. The method of claim 1 , wherein the act (F) comprises automatically applying one or more measures.
15. The method of claim 1 , wherein:
the act (A) comprises receiving a multi-track audio recording comprising more than two tracks synchronized over a time interval, each track corresponding to a different audio recording, each track comprising sound produced by a different instrument;
the act (C) comprises determining a loudness of each of the audio recordings other than the first audio recording at each of the plurality of frequencies during the particular time period; and
the act (D) comprises determining a partial loudness of the first audio recording at a first frequency of the plurality of frequencies during the particular time period, based at least in part on the loudness of another of the audio recordings, at the first frequency during the particular time period, determined in the act (C).