IP Library Granted Patent US 11,640,669
Granted Patent B2
US 11,640,669 · App. 17/065,020 · Granted May 2, 2023

Motion analysis system, motion analysis method, and computer-readable storage medium

Inventors: Haruya Suzuki (Tokyo, JP); Masamichi Kitagawa (Tokyo, JP); Takahisa Miyatake (Tokyo, JP); Koji Fujimoto (Tokyo, JP); Yuichi Tashiro (Tokyo, JP)
Assignee: TENSOR CONSULTING CO. LTD.
G06T7/254G06N20/00G06T7/70G06T2207/10016G06T2207/20081G06T2207/20224G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,640,669
App. No.
17/065,020
Granted
May 2, 2023
Kind
B2
Abstract

A motion analysis system analyzing a prescribed motion performed by a person using a prescribed tool, with a prescribed portion of the person or portion of the tool in the motion being taken as a measurement portion, a predictive model generated by learning based on learning data including an image frame of a motion video, wherein the person performing the motion, and an inter-frame differential frame indicating a difference in pixel values between frames of each pixel of the image and an adjacent frames, which is adjacent to the image frame, in the motion video; generates an inter-frame differential frame of a given analysis object video; and predicts by using the predictive model a position of the measurement portion in an image frame of the analysis object video on the basis of the image frame of the analysis object video and the inter-frame differential frame of the analysis object video.

Claims (37)

1. A motion analysis system analyzing a prescribed motion performed by a person using a prescribed tool, the motion analysis system comprising:

a computer configured to perform:

with a prescribed portion of the person or a prescribed portion of the tool in the motion being taken as a measurement portion, store a predictive model generated by learning based on learning data including:

a plurality of image frames of a motion video, in which the person is performing the motion, and

an inter-frame differential frame indicating a difference in pixel values between (i) each pixel in a first image frame of the plurality of image frames and (ii) each corresponding pixel in a second image frame of the plurality of image frames, the first image frame and the second image frame being temporally consecutive adjacent frames in the motion video;

generate the inter-frame differential frame for a given analysis object video based on whether an absolute value of the difference in pixel values in the analysis object video exceeds a predetermined threshold; and

predict, using the predictive model, a position of the measurement portion in the analysis object video on the basis of the first and second image frames of the analysis object video and the inter-frame differential frame of the analysis object video.

2. The motion analysis system according to claim 1 , wherein the plurality of image frames included in the learning data are each image frames in a time range during which the measurement portion in the motion is moving.

3. The motion analysis system according to claim 1 , the computer further being configured to calculate a position of the measurement portion at a specific time point at which the measurement portion satisfies a prescribed condition in the motion.

4. The motion analysis system according to claim 3 , wherein the computer is further configured to display, on a screen, a trajectory line that is a line tracing a trajectory of the measurement portion in the analysis object video and a marker that indicates, in a prescribed shape and color, a position on the trajectory line at which the measurement portion has been identified.

5. A motion analysis system analyzing a prescribed motion performed by a person using a prescribed tool, the motion analysis system comprising:

a computer configured to perform:

with a prescribed portion of the person or a prescribed portion of the tool in the motion being taken as a measurement portion, store a predictive model generated by learning based on learning data including:

an image frame of a motion video, in which the person is performing the motion, and

an inter-frame differential frame indicating a difference in pixel values between frames of each pixel of the image frame and an adjacent frame, which is adjacent to the image frame, in the motion video; and

predict, using the predictive model, a position of the measurement portion in an image frame of the analysis object video on the basis of the image frame of the analysis object video and the inter-frame differential frame of the analysis object video, wherein

the prescribed motion includes a plurality of motion phases which are temporally divided and which differ from one another,

the specific time point includes a phase boundary time point that constitutes a boundary between the motion phases, and

the specific time-point position calculating unit is configured to identify the phase boundary time point on the basis of the number of pixels, by which an absolute value of a difference in brightness values of the pixel between an image frame and an adjacent frame exceeds a prescribed threshold.

6. The motion analysis system according to claim 5 , wherein

the specific time point includes an intraphase time point at which a position of the measurement portion satisfies a prescribed condition in the motion phase, and

the computer is further configured to identify the motion phase in the analysis object video on the basis of the number of pixels, by which an absolute value of a difference in brightness values of the pixel between an image frame and an adjacent frame exceeds a prescribed threshold, and to identify the intraphase time point on the basis of the motion phase and the position of the measurement portion.

7. The motion analysis system according to claim 6 , wherein the computer is further configured to interpolate, with a spline curve, a position of the measurement portion between image frames in the analysis object video, and, on the basis of the motion phase and the spline curve, take a position that satisfies the condition on the spline curve as a position of the measurement portion at the intraphase time point.

8. The motion analysis system according to claim 7 , wherein the computer is further configured to interpolate, with the spline curve, a position of the measurement portion between the image frames and a position of the measurement portion in an image frame, in which a position of the measurement portion has not been acquired.

9. A motion analysis method for analyzing a prescribed motion performed by a person using a prescribed tool, the motion analysis method executing by a computer:

storing, with a prescribed portion of the person or a prescribed portion of the tool in the motion being taken as a measurement portion, a predictive model generated by learning based on learning data including:

a plurality of image frames of a motion video, in which the person is performing the motion, and

an inter-frame differential frame indicating a difference in pixel values between (i) each pixel in a first image frame of the plurality of image frames and (ii) each corresponding pixel in a second image frame of the plurality of image frames, the first image frame and the second image frame being temporally consecutive adjacent frames in the motion video;

generating the inter-frame differential frame for a given analysis object video based on whether an absolute value of the difference in pixel values in the analysis object video exceeds a predetermined threshold; and

predicting by using the predictive model a position of the measurement portion in the analysis object video on the basis of the first and second image frames of the analysis object video and the inter-frame differential frame of the analysis object video.

10. A non-transitory computer-readable storage medium storing a motion analysis program for analyzing a prescribed motion performed by a person using a prescribed tool,

the motion analysis program causing a computer to execute:

storing, with a prescribed portion of the person or a prescribed portion of the tool in the motion being taken as a measurement portion, a predictive model generated by learning based on learning data including:

a plurality of image frames of a motion video, in which the person is performing the motion, and

an inter-frame differential frame indicating a difference in pixel values between (i) each pixel in a first image frame of the plurality of image frames and (ii) each corresponding pixel in a second image frame of the plurality of image frames, the first image frame and the second image frame being temporally consecutive adjacent frames in the motion video;

generating the inter-frame differential frame for a given analysis object video based on whether an absolute value of the difference in pixel values in the analysis object video exceeds a predetermined threshold; and

predicting by using the predictive model a position of the measurement portion in an image frame of the analysis object video on the basis of the first and second image frames of the analysis object video and the inter-frame differential frame of the analysis object video.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 7, 2020
From: SUZUKI, HARUYA; KITAGAWA, MASAMICHI; MIYATAKE, TAKAHISA; FUJIMOTO, KOJI; TASHIRO, YUICHI
To: TENSOR CONSULTING CO. LTD.
Reel/Frame 053998/0879 →
Priority Claims (1)
JP JP2019-200458 · Nov 5, 2019 · national
Continuity (1)
Related Publication 20210133987A1 · May 6, 2021