IP Library Granted Patent US 11,675,418
Granted Patent B2
US 11,675,418 · App. 17/045,360 · Granted Jun 13, 2023

Program, information processor, and information processing method for blending motions of a plurality of actors

Inventors: Yasutaka Fukumoto (Tokyo, JP); Nobuhiro Saijo (Tokyo, JP); Kazuma Takahashi (Kanagawa, JP); Keita Mochizuki (Chiba, JP)
Assignee: SONY CORPORATION
G06F3/011A61B5/1114A61B5/1122A61B5/1126A61B5/744A63F13/428G01B21/00G06F3/04815G06T7/251G06T13/40G06T19/003A61B5/0205A61B5/7267A61B2562/0219
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,675,418
App. No.
17/045,360
Granted
Jun 13, 2023
Kind
B2
Abstract

There is provided a program, an information processor, and an information processing method that make it possible to blend motions of a plurality of actors captured by using a motion capture technique and to reproduce the blended motions in real time in an avatar or the like on a virtual space. The program causes a computer to implement a control function of dynamically controlling a motion of an avatar in a virtual space or a robot on a real space, the control function being configured to: capture motions of a plurality of actors on the real space from respective motion sensors attached to the actors; blend the motions of the plurality of actors on the basis of a predetermined algorithm; and dynamically control the motion of the avatar or the robot on the basis of the blend result to cause the avatar or the robot to make a motion reflecting the motions of the plurality of actors.

Claims (35)

1. A non-transitory computer-readable medium having embodied thereon a program, which when executed by a computer causes the computer to execute an information processing method, the method comprising:

capturing motions of a plurality of actors on a real space from respective motion sensors attached to the plurality of actors, each motion sensor being attached to a same site of each actor of the plurality of actors;

blending the motions of the plurality of actors on a basis of a predetermined algorithm; and

dynamically controlling a motion of an avatar in a virtual space or a robot on a basis of a result of the blending to cause the avatar or the robot to make a motion reflecting the motions of the plurality of actors,

wherein the predetermined algorithm blends the motions of the plurality of actors on a basis of a weighting that is applied to the same site of each actor of the plurality of actors and combining the weighted sites of the each actor of the plurality of actors with each other, and

wherein the weighting is set in advance by operating a predetermined setting screen by an operator.

2. The non-transitory computer-readable medium according to claim 1 , wherein the method further comprises:

capturing a motion corresponding to at least one site of each actor of the plurality of actors from the motion sensors attached to the each actor of the plurality of actors;

blending the motions of the sites of the plurality of actors on the basis of the predetermined algorithm; and

dynamically controlling the motion of the avatar or the robot on the basis of the result of the blending.

3. The non-transitory computer-readable medium according to claim 2 , wherein the site comprises a part of a body of the actor.

4. The non-transitory computer-readable medium according to claim 3 , wherein the part of the body includes at least one of upper body, lower body, head, a joint, waist, a wrist, or an ankle.

5. The non-transitory computer-readable medium according to claim 2 , wherein the method further comprises:

when capturing the motion of each actor of the plurality of actors, referencing a first output obtained by performing a first process on sensor data acquired by two or more of the motion sensors attached to two or more of the sites of each actor of the plurality of actors and a second output obtained by performing a second process on the sensor data, and correcting position information for the sites to which the motion sensors are attached.

6. The non-transitory computer-readable medium according to claim 1 , wherein the plurality of actors include at least one person among performers and an audience.

7. The non-transitory computer-readable medium according to claim 1 , wherein the predetermined algorithm blends the motions of the plurality of actors on the basis of the weighting that is dynamically applied to each actor of the plurality of actors in accordance with a distance from a reference point set in advance on the real space to the each actor of the plurality of actors.

8. The non-transitory computer-readable medium according to claim 1 , wherein the predetermined algorithm blends the motions of the plurality of actors on the basis of the weighting that is dynamically applied to each actor of the plurality of actors in accordance with an acceleration, a velocity, or a displacement amount of the each actor of the plurality of actors on the real space.

9. The non-transitory computer-readable medium according to claim 1 , wherein the predetermined algorithm blends the motions of the plurality of actors on the basis of the weighting that is dynamically applied to each actor of the plurality of actors in accordance with a degree of movement synchronousness between the plurality of actors on the real space.

10. The non-transitory computer-readable medium according to claim 1 , wherein the predetermined algorithm blends the motions of the plurality of actors on the basis of the weighting that is dynamically applied to each actor of the plurality of actors in accordance with biological information of the each actor of the plurality of actors.

11. The non-transitory computer-readable medium according to claim 1 , wherein the predetermined algorithm blends the motions of the plurality of actors on the basis of the weighting that is dynamically applied to each actor of the plurality of actors in accordance with environmental information around the each actor of the plurality of actors.

12. The non-transitory computer-readable medium according to claim 1 , wherein the weighting is set by performing, by the operator, an operation of moving a cursor on a line segment connecting a first site of a first actor of the plurality of actors and a second site of a second actor of the plurality of actors on the setting screen.

13. The non-transitory computer-readable medium according to claim 1 , wherein the weighting is set by performing, by the operator, an operation of moving a cursor in a polygon that connects respective predetermined sites of the plurality of actors to each other.

14. An information processor comprising

a controller that dynamically controls a motion of an avatar in a virtual space or a robot on a real space, the controller being configured to:

capture motions of a plurality of actors on the real space from respective motion sensors attached to the plurality of actors, each motion sensor being attached to a same site of each actor of the plurality of actors;

blend the motions of the plurality of actors on a basis of a predetermined algorithm; and

dynamically control the motion of the avatar or the robot on a basis of a result of the blending to cause the avatar or the robot to make a motion reflecting the motions of the plurality of actors,

wherein the predetermined algorithm blends the motions of the plurality of actors on a basis of a weighting that is applied to the same site of each actor of the plurality of actors and combining the weighted sites of the each actor of the plurality of actors with each other, and

wherein the weighting is set in advance by operating a predetermined setting screen by an operator.

15. An information processing method comprising:

capturing motions of a plurality of actors on a real space from respective motion sensors attached to the plurality of actors, each motion sensor being attached to a same site of each actor of the plurality of actors;

blending the motions of the plurality of actors on a basis of a predetermined algorithm; and

dynamically controlling a motion of an avatar in a virtual space or a robot on the real space on a basis of a result of the blending to cause the avatar or the robot to make a motion reflecting the motions of the plurality of actors,

wherein the predetermined algorithm blends the motions of the plurality of actors on a basis of a weighting that is applied to the same site of each actor of the plurality of actors and combining the weighted sites of the each actor of the plurality of actors with each other, and

wherein the weighting is set in advance by operating a predetermined setting screen by an operator.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2020
From: FUKUMOTO, YASUTAKA; SAIJO, NOBUHIRO; TAKAHASHI, KAZUMA; MOCHIZUKI, KEITA
To: SONY CORPORATION
Reel/Frame 053975/0267 →
Priority Claims (1)
JP JP2018-079335 · Apr 17, 2018 · national
Continuity (1)
Related Publication 20210166479A1 · Jun 3, 2021
Cited By (1)
US 12,346,493