Neural network device for speaker recognition and operating method of the same

1. A neural network device for speaker recognition, the neural network device comprising:

a memory configured to store at least one program; and

a processor configured to execute the at least one program to:

receive first training information including a first input speech signal;

train a first neural network using the first training information to generate a trained first neural network, the first neural network being a skip connection-based neural network configured to remove noise from the first input speech signal,

receive second training information including a second input speech signal;

train a second neural network using the second training information to generate a trained second neural network, the second neural network configured to recognize a speaker in the second input speech signal,

generate a third neural network by combining the trained first neural network with the trained second neural network by connecting an input layer of the trained second neural network to a final output layer of the trained first neural network,

receive third training information including a third input speech signal; and

train the third neural network using the third training information to generate the trained third neural network, the third neural network including a skip connection in which a value of an input layer of the third neural network is added to a value of an intermediate layer of the third neural network, and the third neural network configured to perform speech enhancement using the skip connection and recognize a speaker in the third input speech signal, which is a noisy speech signal.

2. The neural network device of claim 1 , wherein the first neural network comprises a plurality of layers, and wherein the first neural network is further configured to add a value of an input layer, among the plurality of layers, to a value of an output layer, among the plurality of layers, and output the value of the output layer as a value of a final output layer based on a skip connection.

3. The neural network device of claim 1 , wherein the first training information further includes a first target speech signal that is obtained by removing noise from the first input speech signal as target information, and

wherein the processor is further configured to train the first neural network based on the first input speech signal and the first target speech signal.

4. The neural network device of claim 1 , wherein

the second training information further includes speaker identification information corresponding to the second input speech signal, and

wherein the processor is further configured to train the second neural network based on the second input speech signal and the speaker identification information.

5. The neural network device of claim 4 , wherein the second input speech signal comprises an estimated clean speech signal that is output from the first neural network during training of the first neural network.

6. The neural network device of claim 1 , wherein

the third training information further includes speaker identification information corresponding to the third input speech signal, and

wherein the processor is further configured to train the third neural network based on the third input speech signal and the speaker identification information.

7. The neural network device of claim 1 , wherein the processor is further configured to generate the third neural network by setting a final output layer of the trained first neural network as an input layer of the trained second neural network.

8. The neural network device of claim 1 , further comprising:

an acoustic sensor configured to sense a noisy speech signal,

wherein the processor is configured to:

obtain feature information for speaker recognition from the sensed noisy speech signal by using the trained third neural network, and

recognize a speaker of the sensed noisy speech signal by comparing the feature information for speaker recognition with a pre-registered feature information for speaker recognition.

9. The neural network device of claim 8 , wherein the processor is further configured to obtain feature information for speaker recognition from the sensed noisy speech signal by extracting a value of a last hidden layer of the trained third neural network to which the sensed noisy speech signal is input.

10. The neural network device of claim 8 , wherein the processor is further configured to obtain a clean speech signal from the sensed noisy speech signal by extracting a value of a layer of the trained third neural network to which information about the sensed noisy speech signal is input.

11. The neural network device of claim 8 , wherein the acoustic sensor comprises at least one of a wide-band microphone, a resonator microphone, or a narrow-band resonator microphone array.

12. The neural network device of claim 1 , further comprising:

an acoustic sensor configured to sense a noisy speech signal of a speaker,

wherein the processor is further configured to obtain feature information for speaker recognition from the noisy speech signal of the speaker by using the trained third neural network, and store the obtained feature information for speaker recognition in the memory together with the speaker identification information to register the speaker.

13. An operating method of a neural network device for speaker recognition, the operating method comprising:

receiving first training information including a first input speech signal;

training a first neural network using the first training information to generate a trained first neural network, the first neural network being a skip connection-based neural network configured to remove noise from the first input speech signal;

receiving second training information including a second input speech signal;

training a second neural network using the second training information to generate a trained second neural network, the second neural network configured to recognize a speaker in the second input speech signal;

generating a third neural network by combining the trained first neural network with the trained second neural network by connecting an input layer of the trained second neural network to a final output layer of the trained first neural network;

receiving third training information including a third input speech signal; and

training the third neural network using the third training information to generate the trained third neural network, the third neural network including a skip connection in which a value of an input layer of the third neural network is added to a value of an intermediate layer of the third neural network, and the third neural network configured to perform speech enhancement using the skip connection and recognize a speaker in the third input speech signal, which is a noisy speech signal.

14. The operating method of claim 13 , wherein the first training information further includes a first target speech signal that is obtained by removing noise from the first input speech signal, and

wherein the training the first neural network comprises training the first neural network based on the first input speech signal and the first target speech signal.

15. The operating method of claim 13 ,

wherein the second training information further includes speaker identification information corresponding to the second input speech signal, and

wherein the training the first neural network comprises training the second neural network based on the second input speech signal and the speaker identification information.

16. The operating method of claim 15 , wherein the second input speech signal comprises an estimated clean speech signal that is output from the first neural network during the training of the first neural network.

17. The operating method of claim 13 , wherein the generating of the third neural network comprises generating the third neural network by setting a final output layer of the trained first neural network as an input layer of the trained second neural network.

18. The operating method of claim 13 , further comprising:

sensing a noisy speech signal;

obtaining feature information for speaker recognition according to the sensed noisy speech signal by using the trained third neural network; and

recognizing a speaker of the sensed noisy speech signal by comparing the feature information for speaker recognition with a pre-registered feature information for speaker recognition.

19. The operating method of claim 18 , further comprising:

obtaining a clean speech signal about the sensed noisy speech signal by extracting a value of a layer of the trained third neural network to which information about the sensed noisy speech signal is input.

20. A non-transitory computer-readable recording medium having a recorded program for executing the operating method of claim 13 in a computer.