Apparatus and method for reasoning music based on brain computer interface

1 . A brain-computer interface (BCI)-based music inferring method in which at least part of each operation is performed by a processor, the method comprising:

receiving brain signal data of a subject imagining a melody;

obtaining an event-related potential (ERP) peak envelope and a delta band peak envelope by processing the brain signal data; and

estimating a melodic contour of the melody on a basis of the ERP peak envelope and the delta band envelope,

wherein the estimating of the melodic contour of the melody comprises estimating the melodic contour of the melody by inputting partial brain signal data into a machine learning-based learning model, and

wherein the learning model is a learning model trained with training brain signal data obtained from a plurality of training subjects listening to music corresponding to a preset melodic contour label as training data.

2 . The BCI-based music inferring method of claim 1 , further comprising:

measuring a length of the brain signal data; and

identifying whether the length of the brain signal data is six seconds or longer.

3 . The BCI-based music inferring method of claim 1 , comprising:

determining partial brain signal data having musical significance having a length corresponding to at least two bars from the brain signal data;

determining meter information about the melody on a basis of the brain signal data corresponding to the partial brain signal data; and

retrieving music on a basis of the meter information about the melody and the melodic contour of the melody.

4 . The BCI-based music inferring method of claim 3 , wherein the determining of the meter information further comprises determining a beat and a meter of the melody on a basis of the brain signal data.

5 . The BCI-based music inferring method of claim 4 , wherein the determining of the beat and the meter comprises:

comparing reference brain signal data obtained from the subject while the subject listens to music having a preset beat and meter with the brain signal data;

determining a frequency region in which a power spectrum is increased from the brain signal data on a basis of comparison with the reference brain signal data; and

determining the beat and the meter of the melody imagined by the subject on a basis of the frequency region in which the power spectrum is increased.

6 . The BCI-based music inferring method of claim 4 , wherein the estimating of the melodic contour comprises:

determining the melodic contour based on the partial brain signal data corresponding to a motif on a basis of the ERP peak envelope and the delta band peak envelope.

7 . The BCI-based music inferring method of claim 6 , wherein the determining of the partial brain signal data corresponding to the motif comprises:

determining a point of the brain signal data at which a low point of the ERP peak envelope is positioned as a start point of the beat;

determining a point of the brain signal data at which a high point of the delta band peak envelope is positioned as a start point of the meter; and

determining a position of the brain signal data corresponding to a bar on a basis of the start point of the meter and the start point of the beat, and determining the partial brain signal data corresponding to the motif on a basis of the position of the brain signal data corresponding to the bar.

8 . The BCI-based music inferring method of claim 1 , wherein the training data is based on at least one feature among a preset band power amplitude, an event-related potential (ERP), and brain connectivity from the training brain signal data obtained from the plurality of training subjects.

9 . A brain-computer interface (BCI)-based music inferring method in which at least part of each operation is performed by a processor, the method comprising:

receiving brain signal data of a subject;

when the received brain signal data is brain signal data of a subject imagining a melody, determining partial brain signal data having musical significance from the brain signal data, and estimating a melodic contour of the melody based on the partial brain signal data; and

when the received brain signal data is brain signal data of a subject measured without musical imagination, determining base meter information from the brain signal data,

wherein the determining of the partial brain signal data comprises obtaining an event-related potential (ERP) peak envelope and a delta band peak envelope by processing the brain signal data, and

wherein the estimating of the melodic contour comprises determining the melodic contour based on the partial brain signal data corresponding to a motif on a basis of the ERP peak envelope and the delta band peak envelope.

10 . The BCI-based music inferring method of claim 9 , wherein the determining of the base meter information comprises determining the base meter information on a basis of peak values in a preset frequency band from the brain signal data.

11 . A brain-computer interface (BCI)-based music inferring apparatus comprising:

a processor; and

a memory configured to be electrically connected to the processor and to store at least one code performed by the processor,

wherein the memory is configured to store a code which, when executed by the processor, causes the processor to:

receive brain signal data of a subject imagining a melody;

obtain an event-related potential (ERP) peak envelope and a delta band peak envelope by processing the brain signal data;

determine partial brain signal data having musical significance from the brain signal data using the ERP peak envelope and the delta band peak envelope; and

estimate a melodic contour of the melody on a basis of the partial brain signal data, and

wherein the memory is configured to cause the processor to estimate the melodic contour of the melody by inputting the partial brain signal data into a machine learning-based learning model, and

wherein the learning model is a learning model trained with training brain signal data obtained from at least one second subject listening to music corresponding to a preset melodic contour label as training data.

12 . The BCI-based music inferring apparatus of claim 11 , wherein the memory is configured to further store a code which causes the processor to measure a length of the brain signal data; and to identify whether the length of the brain signal data is six seconds or longer.

13 . The BCI-based music inferring apparatus of claim 12 , wherein the memory is configured to:

further store a code which causes the processor to determine the partial brain signal data having musical significance having a length corresponding to at least two bars from the brain signal data;

determine meter information about the melody on a basis of the brain signal data corresponding to the partial brain signal data; and

retrieve music on a basis of the meter information about the melody and the melodic contour of the melody.

14 . The BCI-based music inferring apparatus of claim 13 , wherein the memory is configured to further store a code which causes the processor to determine a beat and a meter of the melody on a basis of the brain signal data.

15 . The BCI-based music inferring apparatus of claim 14 , wherein the memory is configured to further store a code which causes the processor to determine a frequency region in which a power spectrum is increased from the brain signal data on a basis of a result of comparing reference brain signal data obtained from at least one first subject while the first subject listens to music having a preset beat and meter with the brain signal data, and to determine the beat and the meter of the melody imagined by the subject on a basis of the frequency region in which the power spectrum is increased.

16 . The BCI-based music inferring apparatus of claim 11 , wherein the memory is configured to further store a code which causes the processor to determine the melodic contour based on the partial brain signal data corresponding to a motif on a basis of the ERP peak envelope and the delta band peak envelope.

17 . The BCI-based music inferring apparatus of claim 16 , wherein the memory is configured to:

further store a code which causes the processor to determine a point of the brain signal data at which a high point of the ERP peak envelope is positioned as a start point of the meter;

determine a point of the brain signal data at which a low point of the delta band peak envelope is positioned as a start point of each beat;

determine a position of the brain signal data corresponding to a bar on a basis of the start point of the meter and the start point of each beat; and

determine the partial brain signal data corresponding to the motif on a basis of the position of the brain signal data corresponding to the bar.

18 . The BCI-based music inferring apparatus of claim 11 , wherein the training data is based on at least one feature among a preset band power amplitude, an event-related potential (ERP), and brain connectivity from the training brain signal data obtained from the second subject.