IP Library Granted Patent US 12,323,642
Granted Patent B2
US 12,323,642 · App. 18/443,770 · Granted Jun 3, 2025

Computer program, server device, terminal device, and method

Inventors: Akihiko Shirai (Tokyo, JP); Yousuke Yamanouchi (Tokyo, JP)
Assignee: GREE, INC.
H04N21/2343H04N21/21805H04N21/2187H04N21/232H04N21/235H04N21/8545
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,323,642
App. No.
18/443,770
Granted
Jun 3, 2025
Kind
B2
Abstract

A computer program according allows a processor to function such that a first stream of a first data type including a motion, a sound, a comment, or information according to gifting is acquired, a second stream of a second data type including a video is acquired, data according to the first stream and data according to the second stream are synthesized, and the synthesized data is provided. In addition, the first data type includes intermediate information according to the video. In addition, the intermediate information includes depth information according to the video.

Claims (41)

1. A non-transitory computer readable medium storing computer executable instructions which, when executed by a processor, cause the processor to:

retrieve a first stream of a first data type including a motion, and a second stream of a second data type including a background video, the first data type including intermediate information including depth information relating to the background video and the depth information relating to the motion, wherein the motion is a motion of an avatar;

generate synthesized data from the first stream and the second stream, the synthesized data relating to the first data type and the second data type, such that the motion of the avatar is synthesized with the background video; and

transmit the synthesized data.

2. The non-transitory computer readable medium according to claim 1 , wherein a third stream of the second data type is retrieved, and the synthesized data and the third stream are synthesized.

3. The non-transitory computer readable medium according to claim 1 , wherein a fourth stream of the first data type is retrieved, and the synthesized data and the fourth stream are synthesized.

4. The non-transitory computer readable medium according to claim 1 , wherein

the first stream includes first time information and the second stream includes second time information, and

the first time information and the second time information are adjusted to have a predetermined relationship, and the first stream and the second stream are synthesized.

5. The non-transitory computer readable medium according to claim 1 , wherein synthesis of the synthesized data is performed by a server.

6. The non-transitory computer readable medium according to claim 1 , wherein the avatar is a first avatar and the second stream of the second data type further comprises a motion of a second avatar.

7. The non-transitory computer readable medium according to claim 1 , wherein

the first stream is retrieved from a first information source terminal,

the second stream is retrieved from a second information source terminal, and

synthesis of the synthesized data is performed by the first information source terminal or the second information source terminal.

8. The non-transitory computer readable medium according to claim 7 , wherein the avatar is a first avatar and the second stream of the second data type further comprises a motion of a second avatar.

9. The non-transitory computer readable medium according to claim 8 , wherein the first information source terminal is the first avatar and the second information source terminal is the second avatar.

10. The non-transitory computer readable medium according to claim 1 , wherein synthesis of the synthesized data is performed by a user terminal, and the synthesized data is provided by the user terminal.

11. The non-transitory computer readable medium according to claim 1 , wherein the synthesized data is provided to one or a plurality of user terminals.

12. The non-transitory computer readable medium according to claim 1 , wherein information relating to a viewpoint in a game space is retrieved from one user terminal, and the viewpoint in the game space is changed.

13. The non-transitory computer readable medium according to claim 12 , wherein the viewpoint in the game space is changed to another viewpoint in the game space according to any one user of a user terminal set in advance.

14. The non-transitory computer readable medium according to claim 12 , wherein the viewpoint in the game space is changed to another viewpoint in one location in the game space other than a user terminal set in advance.

15. The non-transitory computer readable medium according to claim 14 , wherein the change to the viewpoint in the one location in the game space other than the user terminal set in advance is performed by using a provision of a predetermined valuable product as a condition.

16. The non-transitory computer readable medium according to claim 1 , wherein production based on one synthesized data item that is provided in one user terminal is different from production based on one synthesized data item that is provided in another user terminal.

17. The non-transitory computer readable medium according to claim 1 , wherein

the first data type further includes information relating to gifting,

the second data type includes a video that is subjected to moving image distribution, and

synthesis of the synthesized data includes reflection of the information relating to the gifting on the video that is subjected to the moving image distribution.

18. The non-transitory computer readable medium according to claim 1 , wherein the processor is included in a smartphone, a tablet, a mobile phone or a personal computer.

19. A server device, comprising:

a processor configured to:

retrieve a first stream of a first data type including a motion, the first data type including intermediate information including depth information relating to a background video and the depth information relating to the motion, wherein the motion is a motion of an avatar;

retrieve a second stream of a second data type including the background video;

generate synthesized data relating to the first data type and the second data type, such that the motion of the avatar is synthesized with the background video; and

transmit the synthesized data.

20. A terminal device, comprising:

a processor configured to:

retrieve a first stream of a first data type including a motion, the first data type including intermediate information including depth information relating to a background video and the depth information relating to the motion, wherein the motion is a motion of an avatar;

retrieve a second stream of a second data type including the background video;

generate synthesized data relating to the first data type and the second data type, such that the motion of the avatar is synthesized with the background video; and

provide the synthesized data.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY STREET ADDRESS AND ATTORNEY DOCKET NUMBER PREVIOUSLY RECORDED AT REEL: 71308 FRAME: 765. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Jun 10, 2025
From: GREE, INC.
To: GREE HOLDINGS, INC.
Reel/Frame 071611/0252 →
CHANGE OF NAME Recorded May 16, 2025
From: GREE, INC.
To: GREE HOLDINGS, INC.
Reel/Frame 071308/0765 →
Priority Claims (1)
JP 2019-177503 · Sep 27, 2019 · national
Continuity (2)
Continuation 17030942 · Sep 24, 2020
Related Publication 20240196025A1 · Jun 13, 2024
References Cited (24)
US 7038699B2 · Sato et al. · 2006 [cited by applicant]
US 11277585B2 · Kawakami · 2022 [cited by examiner]
US 20060140508A1 · Ohgishi et al. · 2006 [cited by applicant]
US 20170006074A1 · Oates, III · 2017 [cited by applicant]
US 20180074679A1 · Wang · 2018 [cited by examiner]
US 20180204381A1 · Kanatsu et al. · 2018 [cited by applicant]
US 20180295389A1 · Kakurai · 2018 [cited by applicant]
US 20180343442A1 · Yoshikawa et al. · 2018 [cited by applicant]
US 20190208230A1 · Fan · 2019 [cited by examiner]
US 20190364224A1 · Nishibe · 2019 [cited by examiner]
US 20210176288A1 · Ahuja et al. · 2021 [cited by applicant]
JP 2002271693A · 2002 [cited by applicant]
JP 2016187063A · 2016 [cited by applicant]
JP 2018007828A · 2018 [cited by applicant]
JP 2018074294A · 2018 [cited by applicant]
JP 2019037549A · 2019 [cited by applicant]
JP 6543403B1 · 2019 [cited by applicant]
JP 6580748B1 · 2019 [cited by applicant]
WO 2018142494A1 · 2018 [cited by applicant]
WO 2019168630A2 · 2019 [cited by applicant]
Notification of Reasons for Refusal, issued by Japanese Patent Office in corresponding with Japanese Application No. 2019-177503, mailed Dec. 26, 2022 (6 pages). [cited by applicant]
Office Action issued Oct. 8, 2024 in corresponding Japanese Application No. 2023-148974. [cited by applicant]
Office Action issued Jan. 7, 2025 in Japanese Patent Application No. 2023-148974. [cited by applicant]
Reconsideration Report by Examiner before Appeal, issued Apr. 7, 2025, in corresponding Japanese Patent Application No. 2023-148974. [cited by applicant]