Electronic device, method and computer program
An electronic device having a circuitry configured to perform audio source separation on an audio input signal to obtain a vocals signal and an accompaniment signal and to perform a confidence analysis on a user's voice signal based on the vocals signal to provide guidance to the user.
1 . An electronic device comprising:
circuitry configured to
perform audio source separation on an audio input signal (x(n)) to obtain a vocals signal (s O (n)) and an accompaniment signal (s A (n)); and
perform a confidence analysis on a user's voice signal (s U (n)) based on the vocals signal (s O (n)) to provide real-time guidance to the user by adjusting a level of the vocals signal in real-time based on the confidence analysis such that the level of the vocals signal is reduced as a confidence value resulting from the confidence analysis increases.
2 . The electronic device of claim 1 , wherein the circuitry is further configured to obtain an adjusted vocals signal (s′ O (n)) from the vocals signal (s O (n)) and to perform mixing of the adjusted vocals signal (s′ O (n)) with the accompaniment signal (s A (n)), to obtain an adjusted audio signal (s″ O (n)) for providing guidance to the user.
3 . The electronic device of claim 2 , wherein the circuitry is further configured to play back the adjusted audio signal (s″ O (n)) for providing guidance to the user.
4 . The electronic device of claim 2 , wherein the circuitry is further configured to perform a gain control on the vocals signal (s O (n)) based on the confidence analysis to obtain the adjusted vocals signal (s′ O (n)).
5 . The electronic device of claim 1 , wherein the circuitry is further configured to generate a guidance control signal based on the confidence analysis and to perform a visual or audio guidance based on the guidance control signal for providing guidance to the user.
6 . The electronic device of claim 1 , wherein the circuitry is further configured to perform pitch analysis on the vocals signal (s O (n)) to obtain a vocals pitch analysis result (ω fO (n)), to perform a pitch analysis ( 303 ) on the user's voice (s U (n)) to obtain a user's pitch analysis result (ω fU (n)), and to perform a vocals pitch comparison based on the vocals pitch analysis result (ω fO (n)) and the user's pitch analysis result (ω fU (n) to obtain a pitch error (e P (n)), wherein the pitch analysis includes calculating a frequency ration in cents and performing a modulo-600 operation to maintain the frequency ratio within half an octave.
7 . The electronic device of claim 1 , wherein the circuitry is further configured to perform rhythm analysis on the vocals signal (s O (n)) to obtain a vocals rhythm analysis result ({tilde over (d)} O (n)), to perform a rhythm analysis on the user's voice (s U (n)) to obtain a user's rhythm analysis result ({tilde over (d)} U (n)), and to perform a vocals rhythm comparison based on the vocals rhythm analysis result ({tilde over (d)} O (n)) and the user's rhythm analysis result ({tilde over (d)} U (n)) to obtain a rhythm error (e R (n)).
8 . The electronic device of claim 5 , wherein the circuitry is further configured to perform a confidence estimation based on the pitch analysis result (ω fO (n), ω fU (n)) and based on the rhythm analysis result ({tilde over (d)} O (n), {tilde over (d)} U (n)) to obtain a confidence value (e tot (n)).
9 . The electronic device of claim 8 , wherein the circuitry is further configured to perform a confidence-to-gain mapping based on the confidence value (e tot (n)) to obtain a gain control signal.
10 . The electronic device of claim 8 , wherein the circuitry is further configured to perform a guidance logic based on the confidence value (e tot (n)) to obtain a guidance control signal.
11 . An electronic device comprising:
circuitry configured to
perform audio source separation on an audio input signal (x(n)) to obtain a vocals signal (s O (n)) and an accompaniment signal (s A (n));
perform a confidence analysis on a user's voice signal (s U (n)) based on the vocals signal (s O (n)) to provide guidance to the user; and
perform voice activity detection on the user's voice signal (s U (n)) to obtain a trigger signal that triggers the confidence analysis.
12 . The electronic device of claim 8 , wherein the circuitry is configured to provide guidance to the user based on the confidence value (e tot (n)).
13 . The electronic device of claim 9 , wherein the confidence-to-gain mapping is configured to set the gain control signal in such a way that the user receives no guidance if the user is singing in perfect pitch.
14 . The electronic device of claim 1 , wherein the audio input signal (x(n)) comprises a mono and/or stereo audio input signal (x(n)) or the audio input signal (x(n)) comprises an object-based audio input signal.
15 . The electronic device of claim 1 , wherein the circuitry is further configured to perform echo cancellation on the user's voice (s U (n)) to obtain an echo free user's voice.
16 . The electronic device of claim 1 , wherein the circuitry is further configured to perform a vocal characteristic analysis on the vocals signal (s O (n)) to obtain a vocal characteristic analysis result, to perform a vocal characteristic analysis on the user's voice (s U (n)) to obtain a user's vocal characteristic analysis result ({tilde over (d)} U (n)).
17 . The electronic device of claim 1 , wherein the circuitry comprises a microphone configured to capture the user's vocals signal (s U (n)).
18 . A method comprising:
performing audio source separation on an audio input signal (x(n)) to obtain a vocals signal (s O (n)) and an accompaniment signal (s A (n)); and
performing confidence analysis on a user's voice (s U (n)) based on the vocals signal (s O (n)) to provide real-time guidance to the user by adjusting a level of the vocals signal in real-time based on the confidence analysis such that the level of the vocals signal is reduced as a confidence value resulting from the confidence analysis increases.
19 . A non-transitory computer-readable medium including computer program comprising instructions, the instructions when executed by circuitry causing the circuitry to perform the method of claim 18 .
20 . The electronic device of claim 1 , wherein the circuitry is configured to trigger the confidence analysis when the user starts singing above a predetermined threshold and stop the confidence analysis when the user stops singing or lowers voice below the predetermined threshold.