Device and method for voice-based trauma screening using deep-learning
This application relates to a device and a method for voice-based trauma screening using deep learning. The device and method for voice-based trauma screening using deep learning screen for trauma through voices that may be obtained in a non-contact manner without limitations of space or situation. In one aspect, the device includes a memory configured to store at least one program and a processor configured to perform an operation by executing the at least one program. The processor can obtain voice data, pre-process the voice data, convert pre-processed voice data into image data, and input the image data to a deep learning model and obtain a trauma result value as an output value of the deep learning model.
1. A voice-based trauma screening device utilizing deep learning comprising:
a memory configured to store at least one program; and
a processor configured to perform an operation by executing the at least one program, the processor configured to:
obtain voice data,
pre-process the voice data,
convert pre-processed voice data into image data,
input the image data to a deep learning model and obtain a trauma result value as an output value of the deep learning model, the output value comprising an emotion result value corresponding to a first emotion class, a second emotion class, or a third emotion class,
obtain the emotion result value only in response to the emotion result value being maintained in correspondence to a set window size,
determine existence of trauma of a speaker of the voice data in response to the obtained emotion result value corresponding to the second emotion class or the third emotion class,
determine that the trauma of the speaker of the voice data is in an early stage in response to the obtained emotion result value corresponding to the second emotion class, and
determine that the trauma of the speaker of the voice data is beyond the early stage in response to the obtained emotion result value corresponding to the third emotion class.
2. The voice-based trauma screening device utilizing deep learning of claim 1 , wherein the processor is configured to pre-process the voice data by shifting the voice data by a predetermined time unit such that the voice data becomes data having a predetermined length.
3. The voice-based trauma screening device utilizing deep learning of claim 1 , wherein the processor is configured to:
generate 2-dimensional data by performing Short-Time Fourier Transformation on the pre-processed voice data, and
input the 2-dimensional data to input to the deep learning model as image data.
4. The voice-based trauma screening device utilizing deep learning of claim 1 , wherein the first emotion class includes one or more first mental states of the speaker having low correlation with trauma, wherein the second emotion class includes one or more second mental states of the speaker being highly correlated with trauma, and wherein the third emotion class includes one or more third mental states of the speaker configured to become prominent after a predetermined time is elapsed after trauma.
5. The voice-based trauma screening device utilizing deep learning of claim 4 , wherein the one or more first mental states of the speaker comprise neutrality and happiness, wherein the one or more second mental states of the speaker comprise fear and surprise, and wherein the one or more third mental states of the speaker comprise sadness and anger.
6. A voice-based trauma screening method utilizing deep learning comprising:
obtaining, at a processor, voice data;
pre-processing, at the processor, the voice data;
converting, at the processor, pre-processed voice data into image data;
inputting, at the processor, the image data to a deep learning model and obtaining a trauma result value as an output value of the deep learning model, the output value comprising an emotion result value corresponding to a first emotion class, a second emotion class, or a third emotion class;
obtaining the emotion result value only in response to the emotion result value being maintained in correspondence to a set window size;
determining existence of trauma of a speaker of the voice data in response to the obtained emotion result value corresponding to the second emotion class or the third emotion class;
determining that the trauma of the speaker of the voice data is in an early stage in response to the obtained emotion result value corresponding to the second emotion class; and
determining that trauma of the speaker of the voice data is beyond the early stage in response to the obtained emotion result value corresponding to the third emotion class.
7. A non-transitory computer-readable recording medium storing instructions, when executed by one or more processors, to perform a voice-based trauma screening method utilizing deep learning, the method comprising:
obtaining voice data;
pre-processing the voice data;
converting pre-processed voice data into image data;
inputting the image data to a deep learning model and obtaining a trauma result value as an output value of the deep learning model, the output value comprising an emotion result value corresponding to a first emotion class, a second emotion class, or a third emotion class;
obtaining the emotion result value only in response to the emotion result value being maintained in correspondence to a set window size;
determining existence of trauma of a speaker of the voice data in response to the obtained emotion result value corresponding to the second emotion class or the third emotion class;
determining that the trauma of the speaker of the voice data is in an early stage in response to the obtained emotion result value corresponding to the second emotion class; and
determining that trauma of the speaker of the voice data is beyond the early stage in response to the obtained emotion result value corresponding to the third emotion class.