Granted Patent B2

US 12,377,316 · App. 17/615,006 · Granted Aug 5, 2025

Inventors: Hiroshi Ueda (Tokyo, JP); Makio Kashino (Tokyo, JP)

Assignee: NIPPON TELEGRAPH AND TELEPHONE CORPORATION

A63B24/0006G06V40/23A63B2024/0015A63B2220/62

Abstract

Claims (57)

1. A physical exertion feedback device comprising a processor configured to execute operations comprising:

dynamically generating, as a modulated image, an image reflecting modulated motion time-series data, the modulated motion time-series data being obtained by modulating, in real time, data within a partial time period of input motion time-series data representing a motion of a subject using reference motion time-series data representing a motion of a teacher, the modulated motion time-series data representing a motion for correcting the motion of the subject,

the modulating further comprises modulating the input motion time-series data to correct a spatial shift between the motion of the subject and the motion of the teacher and/or to correct a temporal shift between the motion of the subject and the motion of the teacher,

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points,

the modulating the input motion time-series data further comprises, as predetermined steps:

calculating a representative value of a difference between positional information about representative points of a body of the teacher and positional information about representative points of a body of the subject at a time point t after the modulation start time point, and

spatially changing the positional information about each representative point of the body of the subject to make the difference shifted toward the representative value or shifted away from the representative value;

setting a modulation timing parameter indicating the modulation start time point and a modulation content parameter to be used in the modulating the input motion time-series data, wherein

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the spatial shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject spatially shifted toward the motion of the teacher or performed to make the motion of the subject spatially shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the spatial shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation,

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the temporal shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject temporarily shifted toward the motion of the teacher or performed to make the motion of the subject temporarily shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the temporal shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation, and

the modulating the input motion time-series data is based on the modulation timing parameter and the modulation content parameter set in the modulator; and presenting, in real time, the modulated image to the subject, wherein

the modulated motion time-series data is, at respective time points from a time point at which the subject starts the motion to a predetermined time point which is hereinafter referred to as a modulation start time point, motion time-series data that matches the input motion time-series data, and is, at respective time points after the modulation start time point, motion time-series data obtained by modulating the input motion time-series data, and the modulating the data is based on at least one of spatial modulation or temporal modulation of the data.

2. The physical exertion feedback device according to claim 1 , wherein

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points, and

the modulating the input motion time-series data further comprises, as predetermined steps:

calculating a difference between positional information about representative points of a body of the teacher at a time point t after the modulation start time point and positional information about representative points of a body of the subject at each time point included in proximity of the time point t;

calculating a representative value of the difference at each time point included in the proximity of the time point t;

determining that a time point at which an absolute value of the representative value of the difference becomes smallest is a time point corresponding to the time point t; and

temporarily changing the positional information about each representative point of the body of the subject to make the time point corresponding to the time point t shifted away from the time point t or to make the time point corresponding to the time point t shifted toward the time point t, when the time point corresponding to the time point t is earlier than the time point t.

3. A physical exertion feedback method comprising:

dynamically generating as a modulated image, an image reflecting modulated motion time-series data, the modulated motion time-series data being obtained by modulating, in real time, data within a partial time period of input motion time-series data representing a motion of a subject using reference motion time-series data representing a motion of a teacher, the modulated motion time-series data representing a motion for correcting the motion of the subject,

the modulating further comprises modulating the input motion time-series data to correct a spatial shift between the motion of the subject and the motion of the teacher and/or to correct a temporal shift between the motion of the subject and the motion of the teacher,

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points,

the modulating the input motion time-series data further comprises, as predetermined steps:

calculating a representative value of a difference between positional information about representative points of a body of the teacher and positional information about representative points of a body of the subject at a time point t after the modulation start time point, and

spatially changing the positional information about each representative point of the body of the subject to make the difference shifted toward the representative value or shifted away from the representative value;

setting a modulation timing parameter indicating the modulation start time point and a modulation content parameter to be used in the modulating the input motion time-series data, wherein

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the spatial shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject spatially shifted toward the motion of the teacher or performed to make the motion of the subject spatially shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the spatial shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation,

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the temporal shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject temporarily shifted toward the motion of the teacher or performed to make the motion of the subject temporarily shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the temporal shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation, and

the modulating the input motion time-series data is based on the modulation timing parameter and the modulation content parameter set in the modulator; and

presenting, in real time, the modulated image to the subject, wherein the modulated motion time-series data is, at respective time points from a time point at which the subject starts the motion to a predetermined time point which is hereinafter referred to as a modulation start time point, motion time-series data that matches the input motion time-series data, and is, at respective time points after the modulation start time point, motion time-series data obtained by modulating the input motion time-series data, and the modulating the data is based on at least one of spatial modulation or temporal modulation of the data.

4. The physical exertion feedback method according to claim 3 , wherein

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points, and

the modulating the input motion time-series data further comprises:

calculating a difference between positional information about representative points of a body of the teacher at a time point t after the modulation start time point and positional information about representative points of a body of the subject at each time point included in proximity of the time point t;

calculating a representative value of the difference at each time point included in the proximity of the time point t;

determining that a time point at which an absolute value of the representative value of the difference becomes smallest is a time point corresponding to the time point t; and

temporarily changing the positional information about each representative point of the body of the subject to make the time point corresponding to the time point t shifted away from the time point t or to make the time point corresponding to the time point t shifted toward the time point t, when the time point corresponding to the time point t is earlier than the time point t.

5. A computer-readable non-transitory recording medium storing computer-executable program instructions that when executed by a processor causes a computer system to:

dynamically generate as a modulated image, an image reflecting modulated motion time-series data, the modulated motion time-series data being obtained by modulating, in real time, data within a partial time period of input motion time-series data representing a motion of a subject using reference motion time-series data representing a motion of a teacher, the modulated motion time-series data representing a motion for correcting the motion of the subject

the modulating further comprises modulating the input motion time-series data to correct a spatial shift between the motion of the subject and the motion of the teacher and/or to correct a temporal shift between the motion of the subject and the motion of the teacher,

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points,

the modulating the input motion time-series data further comprises, as predetermined steps:

calculating a representative value of a difference between positional information about representative points of a body of the teacher and positional information about representative points of a body of the subject at a time point t after the modulation start time point, and

spatially changing the positional information about each representative point of the body of the subject to make the difference shifted toward the representative value or shifted away from the representative value;

set a modulation timing parameter indicating the modulation start time point and a modulation content parameter to be used in the modulating the input motion time-series data, wherein

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the spatial shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject spatially shifted toward the motion of the teacher or performed to make the motion of the subject spatially shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the spatial shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation,

when the modulating the input motion time-series data represents modulating the input motion time-series data to correct the temporal shift between the motion of the subject and the motion of the teacher, the modulation content parameter includes a parameter indicating whether the modulation is performed to make the motion of the subject temporarily shifted toward the motion of the teacher or performed to make the motion of the subject temporarily shifted away from the motion of the teacher, a parameter indicating an amount of the correction for the temporal shift between the motion of the subject and the motion of the teacher, and a parameter indicating a method of performing the modulation, and

the modulating the input motion time-series data is based on the modulation timing parameter and the modulation content parameter set in the modulator; and

present, in real time, the modulated image to the subject, wherein the modulated motion time-series data is, at respective time points from a time point at which the subject starts the motion to a predetermined time point which is hereinafter referred to as a modulation start time point, motion time-series data that matches the input motion time-series data, and is, at respective time points after the modulation start time point, motion time-series data obtained by modulating the input motion time-series data, and the modulating the data is based on at least one of spatial modulation or temporal modulation of the data.

6. The computer-readable non-transitory recording medium of claim 5 , wherein

the motion time-series data representing the motion is a list of positional information about representative points of a human body at respective time points, and

the modulating the input motion time-series data further comprises:

calculating a difference between positional information about representative points of a body of the teacher at a time point t after the modulation start time point and positional information about representative points of a body of the subject at each time point included in proximity of the time point t;

calculating a representative value of the difference at each time point included in the proximity of the time point t;

determining that a time point at which an absolute value of the representative value of the difference becomes smallest is a time point corresponding to the time point t; and

temporarily changing the positional information about each representative point of the body of the subject to make the time point corresponding to the time point t shifted away from the time point t or to make the time point corresponding to the time point t shifted toward the time point t, when the time point corresponding to the time point t is earlier than the time point t.

Assignments (2)

Continuity (1)

References Cited (19)