Motion data generation device, motion data generation method, and recording medium

1 . A motion data generation device comprising:

a memory storing instructions, and

a processor connected to the memory and configured to execute the instructions to:

acquire a plurality of pieces of motion data to be data converted;

group the plurality of pieces of motion data into one or more of a plurality of motion classes, each motion class representing a different type of target motion for data augmentation;

set, for each motion class, at least one piece of the motion data grouped for each motion class as reference data;

set, for each motion class, at least one piece of the motion data different from the reference data among the grouped motion data as data to be converted;

generate extension motion data in which the data to be converted is synchronized with reference to motion timing of the reference data within each motion class; and

output the generated extension motion data, wherein

the processor is configured to execute the instructions to

group the plurality of motion data into a plurality of clusters by a predetermined clustering method,

pair two of the clusters included in the plurality of clusters grouped according to the number of pieces of the motion data included in the clusters,

randomly extract at least one piece of the motion data from each of the paired two clusters,

pair the motion data extracted from each of the two clusters, and

generate the extension motion data in which the paired motion data is synchronized with each other, and wherein

the processor is configured to execute the instructions to

calculate an inverse ratio from a ratio of the number of pieces of the motion data included in the plurality of clusters, and

pair a cluster having a maximum calculated inverse ratio value and any one of the clusters having a non-maximum inverse ratio value.

2 . The motion data generation device according to claim 1 , wherein

the processor is configured to execute the instructions to

set all pieces of the motion data grouped for each motion class for the reference data,

set all pieces of the motion data different from the reference data among the grouped motion data for the data to be converted, and

generate the extension motion data in which all pieces of the data to be converted set for the reference data are synchronized with each other with reference to motion timings of all pieces of the reference data.

3 . The motion data generation device according to claim 1 , wherein

the processor is configured to execute the instructions to

extract a preset number of samples from the plurality of pieces of motion data,

calculate an index value representing a relationship between samples with respect to the extracted samples,

pair the motion data to be converted according to the index value, and

generate the extension motion data in which the paired motion data is synchronized with each other.

4 . The motion data generation device according to claim 3 , wherein

the processor is configured to execute the instructions to

calculate a degree of similarity between samples as the index value, and

pair two pieces of the motion data having a degree of similarity smaller than a preset similarity degree threshold value.

5 . The motion data generation device according to claim 3 , wherein

the processor is configured to execute the instructions to

calculate a distance between samples as the index value, and

pair two pieces of the motion data having a distance larger than a preset distance threshold value.

6 . The motion data generation device according to claim 1 , wherein

the processor is configured to execute the instructions to

normalize posture data estimated for each frame constituting motion data including a target motion to an angle expression,

calculate a feature amount in an embedded space by input the posture data normalized to the angle expression to an encoder including a graph convolutional network,

calculate a distance between a feature amount calculated for each frame constituting the reference data and a feature amount calculated for each frame constituting synchronization target data, and

calculate an optimal path for each frame based on the calculated distance, and

synchronize the synchronization target data with the reference data by aligning timing of frames connected by the optimal path, wherein

the encoder is configured to

convolve the posture data normalized to an angle expression by graph convolution, and

output embedding in an embedded space as a feature amount, and wherein

the processor is configured to execute the instructions to

calculate a distance between a feature amount related to a frame constituting the reference data and a feature amount related to a frame constituting the synchronization target data in a brute-force manner.

7 . A motion data generation method executed by a computer, the method comprising:

acquiring a plurality of pieces of motion data to be data converted;

grouping the plurality of pieces of motion data into one or more of a plurality of motion classes, each motion class representing a different type of target motion for data augmentation;

setting, for each motion class, at least one piece of the motion data grouped for each motion class as reference data;

setting, for each motion class, at least one piece of the motion data different from the reference data among the grouped motion data as data to be converted;

generating extension motion data in which the data to be converted is synchronized with reference to motion timing of the reference data within each motion class; and

outputting the generated extension motion data, wherein

grouping the plurality of motion data into a plurality of clusters by a predetermined clustering method,

pairing two of the clusters included in the plurality of clusters grouped according to the number of pieces of the motion data included in the clusters,

randomly extracting at least one piece of the motion data from each of the paired two clusters,

pairing the motion data extracted from each of the two clusters, and

generating the extension motion data in which the paired motion data is synchronized with each other, and wherein

calculating an inverse ratio from a ratio of the number of pieces of the motion data included in the plurality of clusters, and

pairing a cluster having a maximum calculated inverse ratio value and any one of the clusters having a non-maximum inverse ratio value.

8 . A non-transitory recording medium storing a program for causing a computer to execute the steps of:

acquiring a plurality of pieces of motion data to be data converted;

grouping the plurality of pieces of motion data into one or more of a plurality of motion classes, each motion class representing a different type of target motion for data augmentation;

setting, for each motion class, at least one piece of the motion data grouped for each motion class as reference data;

setting, for each motion class, at least one piece of the motion data different from the reference data among the grouped motion data as data to be converted;

generating extension motion data in which the data to be converted is synchronized with reference to motion timing of the reference data within each motion class; and

outputting the generated extension motion data, wherein

grouping the plurality of motion data into a plurality of clusters by a predetermined clustering method,

pairing two of the clusters included in the plurality of clusters grouped according to the number of pieces of the motion data included in the clusters,

randomly extracting at least one piece of the motion data from each of the paired two clusters,

pairing the motion data extracted from each of the two clusters, and

generating the extension motion data in which the paired motion data is synchronized with each other, and wherein

calculating an inverse ratio from a ratio of the number of pieces of the motion data included in the plurality of clusters, and

pairing a cluster having a maximum calculated inverse ratio value and any one of the clusters having a non-maximum inverse ratio value.