IP Library › Granted Patent US 11,763,464
Granted Patent B2
US 11,763,464 · App. 17/271,573 · Granted Sep 19, 2023

Estimation apparatus, learning apparatus, estimation method, learning method, and program

Inventor: Yoshinori Ohashi (Kanagawa, JP)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
G06T7/246G06F18/214G06T7/73G06V10/7784G06V10/82G06V40/23G06V40/28G06T2207/20081G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,763,464
App. No.
17/271,573
Granted
Sep 19, 2023
Kind
B2
Abstract

An estimation apparatus, a learning apparatus, an estimation method, and a learning method, and a program capable of accurate body tracking without attaching many trackers to a user are provided. A feature extraction section ( 68 ) outputs feature data indicating a feature of a time-series transition until a latest timing in response to an input of input data that contains region data indicating a position, a posture, or a motion about a region of a body at the latest timing and feature data indicating the feature of the time-series transition previously output from the feature extraction section ( 68 ) at a timing preceding the latest timing. An estimation section ( 72 ) estimates a position, a posture, or a motion of another region of a body closer to a center of the body than the region at the latest timing on the basis of the feature data indicating the feature of the time-series transition until the latest timing.

Claims (86)

1. An estimation apparatus comprising:

an input data generation section that generates input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of

the position, the posture, or the motion about

the region data;

a feature extraction section that outputs new feature data in response to an input of the input data;

an estimation section that estimates a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data,

wherein the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing, and

wherein the estimation section estimates the position, the posture, or the motion of the other region of the body closer to the center of the body than the region at a timing of the latest measurement on a basis of the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement,

until a latest timing in response to the input of the input data that contains the region data at the latest timing and the feature data previously output from the feature extraction section and indicating the feature of the time-series transition at a timing preceding the latest timing, and the estimation section estimates the position, the posture, or the motion of the other region of the body closer to the center of the body than the region at the latest timing on the basis of the feature data indicating the feature of the time-series transition until the latest timing; and

a body tracking execution section that estimates a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

2. The estimation apparatus according to claim 1 , wherein

the region data is data indicating positions, postures, or motions about a plurality of regions different from one another.

3. The estimation apparatus according to claim 2 , wherein

the region data indicates a direction of a head included in the body, an angular speed of a left hand included in the body, and an angular speed of a right hand included in the body, and

the estimation section estimates a direction of a chest or a lumbar included in the body, or an angular speed of the chest or the lumbar included in the body.

4. The estimation apparatus according to claim 1 , wherein

the region data indicates a direction of a hand included in the body, an angular speed of the hand, and a position or a speed of the hand, and

the estimation section estimates a direction or an angular speed of a wrist of the hand included in the body.

5. An estimation apparatus comprising:

an input data generation section that generates input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

a feature extraction section that outputs new feature data in response to an input of the input data;

an estimation section that estimates a position, a posture, or a motion of another region from the region on a basis of the feature data, wherein

wherein the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing, and

wherein the estimation section estimates the position, the posture, or the motion of the other region of the body at a timing of the latest measurement on the basis of the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement;

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing;

a body tracking execution section that estimates a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

6. A learning apparatus comprising:

an input data generation section that generates input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

a feature extraction section that outputs new feature data in response to an input of the input data;

an estimation section that estimates a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data;

a learning section that executes learning of at least one of the feature extraction section or the estimation section on a basis of a result of the estimation,

wherein the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing,

wherein the learning section executes the learning on the basis of the result of the estimation, by the estimation section, of the position, the posture, or the motion of the other region closer to the center of the body than the region at a timing of latest measurement based on the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

a body tracking execution section that estimates a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

7. A learning apparatus comprising:

an input data generation section that generates input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

a feature extraction section that outputs new feature data in response to an input of the input data; and

an estimation section that estimates a position, a posture, or a motion of another region of the body on a basis of the feature data; and

a learning section that executes learning of at least one of the feature extraction section or the estimation section on a basis of a result of estimation,

wherein the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing,

wherein the learning section executes the learning on the basis of the result of the estimation, by the estimation section, of the position, the posture, or the motion of the other region at a timing of latest measurement based on the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement, and

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

a body tracking execution section that estimates a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

8. An estimation method comprising:

generating input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

outputting new feature data in response to an input of the input data; and

estimating a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data,

wherein the region data is made to correspond to a measurement result of the region;

wherein the outputting includes outputting feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing,

wherein the estimating includes estimating the position, the posture, or the motion of the other region of the body closer to the center of the body than the region at a timing of the latest measurement on the basis of the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement;

outputting feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

a body tracking execution section that estimates a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

9. A learning method comprising:

generating input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

outputting, by the feature extraction section, new feature data in response to an input of the input data;

estimating, by an estimation section, a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data; and

executing learning of at least one of the feature extraction section or the estimation section on a basis of a result of estimation, wherein

the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing, and

the learning includes executing the learning on the basis of the result of the estimation, by the estimation section, of the position, the posture, or the motion of the other region closer to the center of the body than the region at a timing of latest measurement based on the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

estimating a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

10. A non-transitory computer readable medium having stored thereon a program for a computer, comprising:

generating, by an input data generation section, input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

outputting, by a feature extraction section, new feature data in response to an input of the input data; and

estimating, by an estimation section, a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data, wherein

the region data is made to correspond to a measurement result of the region,

wherein feature outputting includes outputting feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing,

wherein the estimating includes estimating the position, the posture, or the motion of the other region of the body closer to the center of the body than the region at a timing of the latest measurement on the basis of the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement, and

wherein the outputting further includes outputting feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

estimating a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

11. A non-transitory computer readable medium having stored thereon a program for a computer, comprising:

generating, by an input data generation section, input data that contains region data indicating a position, a posture, or a motion about a region of a body and feature data indicating a feature of a time-series transition of the region data;

outputting, by the feature extraction section, new feature data in response to an input of the input data;

estimating, by an estimation section, a position, a posture, or a motion of another region of the body closer to a center of the body than the region on a basis of the feature data; and

executing, by a learning section, learning of at least one of the feature extraction section or the estimation section on a basis of a result of estimation,

wherein the region data is made to correspond to a measurement result of the region,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing, and

wherein the learning includes executing the learning on the basis of the result of the estimation, by the estimation section, of the position, the posture, or the motion of the other region closer to the center of the body than the region at a timing of latest measurement based on the feature data indicating the feature of the time-series transition of the region data made to correspond to the measurement result until the latest measurement,

wherein feature extraction section outputs feature data made to correspond to the latest measurement result that is based on the feature data previously output from the feature extraction section at a previous timing; and

estimating, by a body tracking execution section, a position and a direction of still another region of the body by inverse kinematics on a basis of an estimation result, by the estimation section, of the position, the posture, or the motion of the other region.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 25, 2021
From: OHASHI, YOSHINORI
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 055723/0066 →
Priority Claims (1)
WO PCT/JP2018/033069 · Sep 6, 2018 · international
Continuity (1)
Related Publication 20210350551A1 · Nov 11, 2021