Skeleton recognition device, training method, and storage medium

1 . A skeleton recognition device of performing skeleton recognition using an image method, the skeleton recognition device comprising:

one or more memories; and

one or more processors coupled to the one or more memories, the one or more processors being configured to:

acquire an output result by inputting teacher data to a training model and processing forward propagation, the teacher data having pairs of skeleton information and input data, the skeleton information indicating positions of a plurality of bones of a human as a correct answer value, the input data being data generated from image data obtained by capturing the human by a camera;

acquire a value of a loss function based on a difference in bone length and a difference in angle of the plurality of bones, bone lengths and angles used for determining the difference in bone length and the difference in angle being based on the skeleton information and based on the output result; and

adjust parameters of the training model based on the value of the loss function, wherein the acquiring of the value of the loss function includes:

acquiring a difference in a joint position based on the skeleton information and the output result;

acquiring an order of the difference in the joint position, an order of the difference in the bone length, and an order of the difference in the angle;

acquiring the value of the loss function based on the difference in the joint position, the difference in the bone length, and the difference in the angle;

acquiring a first weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the bone length;

acquiring a second weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the angle; and

acquiring the value of the loss function by integrating a value of the difference in the joint position, a value of the difference in the bone length multiplied by the first weight, and a value of the difference in the angle multiplied by the second weight.

2 . A training method implemented by a computer of performing skeleton recognition using an image method, the training method comprising:

acquiring an output result by inputting teacher data to a training model and processing forward propagation, the teacher data having pairs of skeleton information and input data, the skeleton information indicating positions of a plurality of bones of a human as a correct answer value, the input data being data generated from image data obtained by capturing the human by a camera;

acquiring a value of a loss function based on a difference in bone length and a difference in angle of the plurality of bones, bone lengths and angles used for determining the difference in bone length and the difference in angle being based on the skeleton information and based on the output result; and

adjusting parameters of the training model based on the value of the loss function, wherein the acquiring of the value of the loss function includes:

acquiring a difference in a joint position based on the skeleton information and the output result;

acquiring an order of the difference in the joint position, an order of the difference in the bone length, and an order of the difference in the angle;

acquiring the value of the loss function based on the difference in the joint position, the difference in the bone length, and the difference in the angle;

acquiring a first weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the bone length;

acquiring a second weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the angle; and

acquiring the value of the loss function by integrating a value of the difference in the joint position, a value of the difference in the bone length multiplied by the first weight, and a value of the difference in the angle multiplied by the second weight.

3 . A non-transitory computer-readable storage medium storing a training program of performing skeleton recognition using an image method, the training program comprising instructions which, when executed by at least one computer, cause the at least one computer to execute processing, the processing comprising:

acquiring an output result by inputting teacher data to a training model and processing forward propagation, the teacher data having pairs of skeleton information and input data, the skeleton information indicating positions of a plurality of bones of a human as a correct answer value, the input data being data generated from image data obtained by capturing the human by a camera;

acquiring a value of a loss function based on a difference in bone length and a difference in angle of the plurality of bones, bone lengths and angles used for determining the difference in bone length and the difference in angle being based on the skeleton information and based on the output result; and

adjusting parameters of the training model based on the value of the loss function, wherein the acquiring of the value of the loss function includes:

acquiring a difference in a joint position based on the skeleton information and the output result;

acquiring an order of the difference in the joint position, an order of the difference in the bone length, and an order of the difference in the angle;

acquiring the value of the loss function based on the difference in the joint position, the difference in the bone length, and the difference in the angle;

acquiring a first weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the bone length;

acquiring a second weight based on a relative magnitude between the order of the difference in the joint position and the order of the difference in the angle; and

acquiring the value of the loss function by integrating a value of the difference in the joint position, a value of the difference in the bone length multiplied by the first weight, and a value of the difference in the angle multiplied by the second weight.