IP Library › Granted Patent US 11,232,291
Granted Patent B2
US 11,232,291 · App. 16/864,601 · Granted Jan 25, 2022

Posture estimation system, posture estimation apparatus, error correction method, and error correction program

Inventor: Akira Nishiyama (Tokyo, JP)
Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
G06K9/00335G06K9/00362G06T7/70G06T2207/30196G06T2207/30204G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,232,291
App. No.
16/864,601
Granted
Jan 25, 2022
Kind
B2
Abstract

A posture estimation system includes: a marker that is mounted on an object; a camera that captures an image of the object; an analysis section that analyzes a posture of the object on a basis of a position of the marker included in a captured image captured by the camera; an inertial measurement section that is provided on the object to detect motion of the object; a calculation section that calculates calculated posture information indicating the posture of the object on a basis of detection information indicating a detection result from the inertial measurement section; an estimation section that estimates the posture of the object on a basis of the calculated posture information and analyzed posture information indicating the posture of the object analyzed by the analysis section; and a correction section that corrects an error included in the calculated posture information on a basis of the analyzed posture information.

Claims (47)

1. A posture estimation system comprising:

a marker that is mounted on an object;

a camera that captures an image of the object;

an analysis section that analyzes a posture of the object on a basis of a position of the marker included in a captured image captured by the camera;

an inertial measurement section that is provided on the object to detect motion of the object;

a calculation section that calculates calculated posture information indicating the posture of the object on a basis of detection information indicating a detection result from the inertial measurement section;

an estimation section that estimates the posture of the object on a basis of the calculated posture information and analyzed posture information indicating the posture of the object analyzed by the analysis section; and

a correction section that corrects an error included in the calculated posture information on a basis of the analyzed posture information,

wherein the analyzed posture information includes a time stamp indicating a timing of image capturing of the captured image for which the analyzed posture information has been obtained by analysis,

wherein the detection information includes a time stamp indicating a timing of detection of motion of the object by the inertial measurement section,

wherein a sampling rate of the inertial measurement section is greater than a sampling rate of the image capturing, and

wherein the correction section corrects a current piece of the calculated posture information on a basis of a difference between the analyzed posture information and the calculated posture information calculated on a basis of the detection information including a time stamp closest to the time stamp included in the analyzed posture information.

2. The posture estimation system according to claim 1 , wherein

the correction section does not correct the calculated posture information in a case where an absolute value of the difference is larger than a predetermined threshold, and

the correction section corrects the calculated posture information in a case where the absolute value of the difference is equal to or smaller than the predetermined threshold.

3. The posture estimation system according to claim 1 , wherein

in a case of assuming

that the difference is d1,

that a degree of reflection being a value larger than 0 and equal to or smaller than 1 and being a degree to which the difference is reflected in the calculated posture information is α1,

that the number of pieces of the detection information input from an assumed reception timing until actual reception of the analyzed posture information is β,

that an input rate for the detection information is R1, and

that a reception rate for the analyzed posture information is r1,

the correction section corrects the calculated posture information by repeatedly subtracting a correction value C 1 indicated by Equation (1) from the calculated posture information calculated by the calculation section according to the number of pieces of the detection information input from a time of reception of the analyzed posture information until a reception timing for a next piece of the analyzed posture information,

[Math. 1]

Correction value C 1=( d 1×α1)/( R 1/ r 1−β)  (1)

However, in a case where R1/r1<β holds true, (R1/r1−β) is 1.

4. The posture estimation system according to claim 1 , comprising:

a display section that is mounted on the object to display an input image;

a drawing section that draws, in a virtual space, an image from a viewpoint set according to the posture of the object estimated by the estimation section; and

an output section that outputs, to the display section, the image drawn by the drawing section.

5. The posture estimation system according to claim 1 , further comprising:

an analysis apparatus that acquires the captured image captured by the camera; and

a terminal apparatus that is provided on the object to acquire the detection information from the inertial measurement section, wherein

the analysis apparatus includes the analysis section, and

the terminal apparatus includes the calculation section, the estimation section, and the correction section.

6. The posture estimation system according to claim 5 , wherein

the terminal apparatus communicates wirelessly with the analysis apparatus, and

the terminal apparatus communicates with the inertial measurement section through a cable.

7. A posture estimation apparatus comprising:

a communication section that externally receives first posture information indicating a posture of an object,

wherein the first posture information is captured at a first sampling rate and comprises a first timestamp;

an input section that receives detection information indicating detection results from an inertial measurement unit provided on the object,

wherein the detection results are captured at a second sampling rate greater than the first sampling rate, and

wherein the detection results comprise a second timestamp;

a calculation unit that calculates, on a basis of the detection information, second posture information indicating the posture of the object;

an estimation section that estimates the posture of the object on a basis of the first posture information and the second posture information; and

a correction section that corrects, on the basis of the first posture information, an error included in the second posture information on a basis of a difference between the first posture information and the second posture information having the first timestamp and the second timestamp closest to each other.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 1, 2020
From: NISHIYAMA, AKIRA
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 052549/0397 →
Priority Claims (1)
JP JP2019-092809 · May 16, 2019 · national
Continuity (1)
Related Publication 20200364447A1 · Nov 19, 2020