User identification method using electrocardiogram and electromyogram
View Patent ↗Provided is a method of identifying a user by converting an electrocardiogram (ECG) and an electromyogram (EMG) into images, respectively, and inputting each converted image to a multi-stream convolutional neural network (multi-stream CNN). According to an embodiment of the present disclosure, the user identification method using the ECG and the EMG includes acquiring one-dimensional ECG and EMG signals for a user, converting the ECG and EMG signals into two-dimensional images, respectively, and identifying a user by inputting the two-dimensional images to a multi-stream convolutional neural network (multi-stream CNN), respectively.
1. A user identification method using an electrocardiogram and an electromyogram comprising the steps of:
acquiring, by a processor, one-dimensional electrocardiogram (ECG) and electromyogram (EMG) signals for a user;
converting, by the processor, the ECG and EMG signals into two-dimensional images, respectively; and
identifying, by the processor, a user by inputting the two-dimensional images to a multi-stream convolutional neural network (multi-stream CNN), respectively,
wherein the multi-stream CNN is supervised-learned by a training dataset using images of the ECG and EMG signals as input data and using user identification information as label data.
2. The user identification method of claim 1 , wherein the acquiring of the one-dimensional ECG and EMG signals includes acquiring the ECG and EMG signals from a sensor in a vehicle coming into contact with a user's body.
3. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes
normalizing the ECG signal and the EMG signal at the same sampling rate; and
converting the normalized ECG signal and EMG signal into the two-dimensional images, respectively.
4. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes
removing noise in the ECG and EMG signals through a Butterworth filter; and
converting the noise-removed ECG signal and EMG signal into the two-dimensional images, respectively.
5. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes
segmenting the ECG signal and the EMG signal by a preset time through non-fiducial segmentation; and
converting the segmented ECG signal and EMG signal into the two-dimensional images, respectively.
6. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes converting ECG and EMG signals expressed in amplitudes to the time into two-dimensional images expressed by a time axis and a frequency axis, respectively.
7. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes converting the ECG and EMG signals into two-dimensional spectrograms.
8. The user identification method of claim 1 , wherein the converting of the ECG and EMG signals into the two-dimensional images, respectively includes converting the ECG and EMG signals into two-dimensional spectrograms by applying constant Q-transform (CQT) to the ECG and EMG signals.
9. The user identification method of claim 8 , wherein the converting of the ECG and EMG signals into the two-dimensional spectrograms, respectively includes converting the ECG and EMG signals into spectrograms according to Equation 1 below:
X
(
l
,
ω
)
=
∫
-
∞
∞
w
(
t
,
ω
)
x
(
t
+
lM
)
e
-
i
2
π
Qt
dt
[
Equation
1
]
(X represents a spectrogram value, ω represents each frequency, l represents a time index, w represents a window function, x represents an ECG or EMG signal, Q represents a quality factor, and M represents the number of frames to which a window is applied in the ECG or EMG signal).
10. The user identification method of claim 1 , wherein the inputting of the two-dimensional images to the multi-stream CNN includes inputting an image for the ECG signal to a first stream neural network, and inputting an image for the EMG signal to a second stream neural network.
11. The user identification method of claim 10 , wherein the identifying of the user includes concatenating features extracted from the first and second stream neural networks, respectively, and identifying the user based on the concatenated features.
12. The user identification method of claim 11 , wherein the concatenating of the extracted features includes element-wise summing a feature vector extracted from the first stream neural network and the feature vector extracted from the second stream neural network.