IP Library › Granted Patent US 12,307,618
Granted Patent B2
US 12,307,618 · App. 17/921,709 · Granted May 20, 2025

Information processing apparatus and information processing method for aligning a second virtual object with a first virtual object

Inventors: Goro Takaki (Tokyo, JP); Matthew Forrest (Tokyo, JP); Shusuke Eshita (Tokyo, JP); Kenzo Nakajima (Tokyo, JP); Yo Nonoyama (Tokyo, JP); Satoshi Akagawa (Tokyo, JP); Katsuji Miyazawa (Tokyo, JP); Keisuke Ito (Tokyo, JP); Shoji Watanabe (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
G06T19/20G06T5/50G06T13/40G06V10/44G06V10/761G06V40/23G06T2207/20044G06T2207/20221G06T2219/2004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,307,618
App. No.
17/921,709
Granted
May 20, 2025
Kind
B2
Abstract

Provided is an information processing apparatus and an information processing method capable of providing more effective learning content in learning motion of a body. An adjustment unit generates an adjusted second virtual object by adjusting, on the basis of feature point information of a first person included in a first virtual object reflecting a body motion of the first person, a second virtual object reflecting a body motion of a second person to be superimposed on the first virtual object. The technology according to the present disclosure can be applied to, for example, an information processing apparatus such as a smartphone.

Claims (68)

1. An information processing apparatus, comprising:

circuitry configured to:

generate an adjusted second virtual object based on an adjustment of a second virtual object, wherein

the adjusted second virtual object is generated based on first feature point information of a first person included in a first virtual object,

the first virtual object reflects a body motion of the first person,

the second virtual object reflects a body motion of a second person,

the second virtual object is superimposed on the first virtual object,

the first virtual object includes

the first feature point information of the first person, and

a three-dimensional model of the first person based on the first feature point information, and

the first feature point information of the first person and the three-dimensional model of the first person are transmitted via different network slices.

2. The information processing apparatus according to claim 1 , wherein

the circuitry is further configured to change second feature point information of the second person included in the second virtual object based on the first feature point information of the first person included in the first virtual object.

3. The information processing apparatus according to claim 2 , wherein

the first feature point information includes at least one of skeleton information, left and right information, or three-dimensional contour information of the first person, and

the second feature point information includes at least one of skeleton information, left and right information, or three-dimensional contour information of the second person.

4. The information processing apparatus according to claim 2 , wherein the circuitry is further configured to:

generate, based on the first feature point information of the first person, the first virtual object that reflects the body motion of the first person; and

generate, based on the second feature point information of the second person, the first virtual object that reflects the body motion of the second person.

5. The information processing apparatus according to claim 4 , wherein the circuitry is further configured to:

extract the first feature point information of the first person based on sensor data corresponding to the first person; and

extract the second feature point information of the second person based on sensor data corresponding to the second person.

6. The information processing apparatus according to claim 5 , wherein

each of the sensor data corresponding to the first person and the sensor data corresponding to the second person includes at least one of ToF data, RGB data, or volumetric capture data.

7. The information processing apparatus according to claim 4 , wherein the circuitry is further configured to:

extract the first feature point information of the first person based on a first video that shows the first person; and

extract the second feature point information of the second person based on a second video that shows the second person.

8. The information processing apparatus according to claim 4 , wherein the circuitry is further configured to:

generate the first virtual object of a first type based on a purpose of the body motion of the first person; and

generate the second virtual object of a second type based on a purpose of the body motion of the second person.

9. The information processing apparatus according to claim 1 , wherein the circuitry is further configured to:

generate a superimposed video in which the first virtual object is superimposed on the adjusted second virtual object.

10. The information processing apparatus according to claim 9 , wherein the circuitry is further

configured to generate an evaluation value of the first virtual object based on a comparison of the first virtual object with the adjusted second virtual object.

11. The information processing apparatus according to claim 10 , wherein

the evaluation value includes at least one of a difference in three-dimensional contour information, a difference in acceleration information, or a difference in a specific fitting point between the first virtual object and the adjusted second virtual object.

12. The information processing apparatus according to claim 10 , wherein the circuitry is further

configured to generate an effect video for the superimposed video based on the evaluation value.

13. The information processing apparatus according to claim 12 , wherein the circuitry is further

configured to control display of the superimposed video and the effect video on a display unit.

14. The information processing apparatus according to claim 13 , wherein the circuitry is further

configured to generate instruction information for the second person to provide a specific instruction to the body motion of the first person based on the evaluation value.

15. The information processing apparatus according to claim 1 , wherein

the first feature point information is transmitted via a low-latency network slice, and

the three-dimensional model of the first person is transmitted via a large-capacity network slice.

16. The information processing apparatus according to claim 1 , wherein

the second virtual object is managed based on copyright information including a person ID that specifies the second person.

17. An information processing method, comprising:

by an information processing apparatus,

generating an adjusted second virtual object based on an adjustment of a second virtual object, wherein

the adjusted second virtual object is generated based on first feature point information of a first person included in a first virtual object,

the first virtual object reflects a body motion of the first person,

the second virtual object reflects a body motion of a second person,

the second virtual object is superimposed on the first virtual object,

the first virtual object includes

the first feature point information of the first person, and

a three-dimensional model of the first person based on the first feature point information, and

the first feature point information of the first person and the three-dimensional model of the first person are transmitted via different network slices.

18. A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by a computer, cause the computer to execute operations, the operations comprising:

generating an adjusted second virtual object based on an adjustment of a second virtual object, wherein

the adjusted second virtual object is generated based on first feature point information of a first person included in a first virtual object,

the first virtual object reflects a body motion of the first person,

the second virtual object reflects a body motion of a second person,

the second virtual object is superimposed on the first virtual object,

the first virtual object includes

the first feature point information of the first person, and

a three-dimensional model of the first person based on the first feature point information, and

the first feature point information of the first person and the three-dimensional model of the first person are transmitted via different network slices.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 27, 2022
From: TAKAKI, GORO; FORREST, MATTHEW; ESHITA, SHUSUKE; NAKAJIMA, KENZO; NONOYAMA, YO; AKAGAWA, SATOSHI; MIYAZAWA, KATSUJI; ITO, KEISUKE; WATANABE, SHOJI
To: SONY GROUP CORPORATION
Reel/Frame 061560/0250 →
Priority Claims (1)
JP 2020-084483 · May 13, 2020 · national
Continuity (1)
Related Publication 20230162458A1 · May 25, 2023
References Cited (11)
US 20090146378A1 · Van Luchene · 2009 [cited by examiner]
US 20140285517A1 · Park · 2014 [cited by examiner]
US 20150227652A1 · Aonuma · 2015 [cited by examiner]
CN 109151430A · 2019 [cited by examiner]
EP 2784751A2 · 2014 [cited by applicant]
JP 2013167941A · 2013 [cited by applicant]
JP 2015146980A · 2015 [cited by applicant]
“Create your own original caricature character!”, Nintendo, WiiSports, URL: https://web.archive.org/web/20061206182618/http://www.nintendo.co.jp/wii/rspj/mii/index.html, Dec. 6, 2006, 03 pages. [cited by applicant]
“Baseball”, Nintendo, WiiSports, URL: https://web.archive.org/web/20061205103216/http://www.nintendo.co.jp/wii/rspj/5sports/baseball.html, Dec. 5, 2006, 03 pages. [cited by applicant]
“Boxing”, Nintendo, WiiSports, URL: https://web.archive.org/web/20110925152014/http://www.nintendo.co.jp/wii/rspj/5sports/boxing.html, Sep. 25, 2011, 03 pages. [cited by applicant]
International Search Report and Written Opinion of PCT Application No. PCT/JP2021/017146, issued on Jul. 27, 2021, 10 pages of ISRWO. [cited by applicant]