Device, method and non-transitory recording medium for building a hidden semi-Markov model to estimate human action segments using hybrid learning
A hidden semi-Markov model includes plural second hidden Markov models each containing plural first hidden Markov models using types of movement of a person as states. The plural second hidden Markov models each use partial actions that are parts of actions determined by combining plural movements as states. In the hidden semi-Markov model observation probabilities are leant for each type of the movements of the plural first hidden Markov models using unsupervised learning. The learnt observation probabilities are fixed, and input first supervised data is augmented to give second supervised data, and transition probabilities of the movements of the first hidden Markov models are learned by supervised learning in which the second supervised data is employed. The learnt observation probabilities and transition probabilities are employed to build the hidden semi-Markov model that is a model for estimating segments of the partial actions.
1 . A partial action segment estimation model building device, comprising:
a memory; and
a processor connected to the memory, the processor being configured to:
in a hidden semi-Markov model (HSMM) including a plurality of second hidden Markov models each containing a plurality of first hidden Markov models using types of movement of a person as states, and the plurality of second hidden Markov models each using partial actions that are parts of actions defined by combining a plurality of the movements as states, learn observation probabilities for each of the movement types of the plurality of first hidden Markov models using unsupervised learning;
fix the learnt observation probabilities, augment input first supervised data so as to give second supervised data, and learn transition probabilities of the movements of the first hidden Markov models by supervised learning in which the second supervised data is employed;
build the hidden semi-Markov model that is a model for estimating segments of the partial actions by using the learnt observation probabilities and the learnt transition probabilities;
extract feature vectors from sensor data generated by detecting postures of a person using sensors;
take a series of the extracted feature vectors as observation data, and estimate successive durations of respective action phases by comparison against the built HSMM;
compute successive durations for the respective actions by adding the successive durations for the action phases contained in each action; and
estimate time segments for each action from the successive durations of each action state, and the built HSMM is applied for estimating the time segments of the action that contains partial actions.
2 . The partial action segment estimation model building device of claim 1 , wherein the partial actions are each generated by dividing the actions so as to contain durations of equal lengths.
3 . The partial action segment estimation model building device of claim 1 , wherein the partial actions are each generated by dividing the actions so as to contain an equal number of movements.
4 . The partial action segment estimation model building device of claim 1 , wherein the processor is configured to augment the first supervised data by adding teacher information of the first supervised data to each item of data generated by adding noise to the first supervised data and oversampling the first supervised data.
5 . The partial action segment estimation model building device of claim 4 , wherein the noise is random noise.
6 . The partial action segment estimation model building device of claim 4 , wherein a variance of the noise becomes larger as scattering in a sample of the movement becomes larger.
7 . A partial action segment estimation model building method comprising:
by a computer,
in a hidden semi-Markov model including a plurality of second hidden Markov models each containing a plurality of first hidden Markov models using types of movement of a person as states, and the plurality of second hidden Markov models each using partial actions that are parts of actions defined by combining a plurality of the movements as states, learning observation probabilities for each of the movement types of the plurality of first hidden Markov models using unsupervised learning;
fixing the learnt observation probabilities, augmenting input first supervised data so as to give second supervised data, and learning transition probabilities of the movements of the first hidden Markov models by supervised learning in which the second supervised data is employed;
building the hidden semi-Markov model that is a model for estimating segments of the partial actions by using the learnt observation probabilities and the learnt transition probabilities;
extracting feature vectors from sensor data generated by detecting postures of a person using sensors;
taking a series of the extracted feature vectors as observation data, and estimate successive durations of respective action phases by comparison against the built HSMM;
computing successive durations for the respective actions by adding the successive durations for the action phases contained in each action; and
estimating time segments for each action from the successive durations of each action state, and the built HSMM is applied for estimating the time segments of the action that contains partial actions.
8 . The partial action segment estimation model building method of claim 7 , wherein the partial actions are each generated by dividing the actions so as to contain durations of equal lengths.
9 . The partial action segment estimation model building method of claim 7 , wherein the partial actions are each generated by dividing the actions so as to contain an equal number of movements.
10 . The partial action segment estimation model building method of claim 7 , wherein augmentation is performed on the first supervised data by adding teacher information of the first supervised data to each item of data generated by adding noise to the first supervised data and oversampling the first supervised data.
11 . The partial action segment estimation model building method of claim 10 , wherein the noise is random noise.
12 . The partial action segment estimation model building method of claim 10 , wherein a variance of the noise becomes larger as scattering in a sample of the movement becomes larger.
13 . A non-transitory recording medium storing a program that causes a computer to execute a partial action segment estimation model building processing, the processing comprising:
in a hidden semi-Markov model including a plurality of second hidden Markov models each containing a plurality of first hidden Markov models using types of movement of a person as states, and the plurality of second hidden Markov models each using partial actions that are parts of actions defined by combining a plurality of the movements as states, learning observation probabilities for each of the movement types of the plurality of first hidden Markov models using unsupervised learning;
fixing the learnt observation probabilities, augmenting input first supervised data so as to give second supervised data, and learning transition probabilities of the movements of the first hidden Markov models by supervised learning in which the second supervised data is employed;
building the hidden semi-Markov model that is a model for estimating segments of the partial actions by using the learnt observation probabilities and the learnt transition probabilities;
extracting feature vectors from sensor data generated by detecting postures of a person using sensors;
taking a series of the extracted feature vectors as observation data, and estimate successive durations of respective action phases by comparison against the built HSMM;
computing successive durations for the respective actions by adding the successive durations for the action phases contained in each action; and
estimating time segments for each action from the successive durations of each action state, and the built HSMM is applied for estimating the time segments of the action that contains partial actions.
14 . The non-transitory recording medium of claim 13 , wherein the partial actions are each generated by dividing the actions so as to contain durations of equal lengths.
15 . The non-transitory recording medium of claim 13 , wherein the partial actions are each generated by dividing the actions so as to contain an equal number of movements.
16 . The non-transitory recording medium of claim 13 , wherein augmentation is performed on the first supervised data by adding teacher information of the first supervised data to each item of data generated by adding noise to the first supervised data and oversampling the first supervised data.
17 . The non-transitory recording medium of claim 16 , wherein the noise is random noise.
18 . The non-transitory recording medium of claim 16 , wherein a variance of the noise becomes a larger as scattering in a sample of the movement becomes larger.