IP Library Granted Patent US 11,930,228
Granted Patent B2
US 11,930,228 · App. 17/030,942 · Granted Mar 12, 2024

Computer program, server device, terminal device and method

Inventors: Akihiko Shirai (Tokyo, JP); Yousuke Yamanouchi (Tokyo, JP)
Assignee: GREE, INC.
H04N21/2343H04N21/21805H04N21/2187H04N21/232H04N21/235H04N21/8545
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,930,228
App. No.
17/030,942
Granted
Mar 12, 2024
Kind
B2
Abstract

A computer program according allows a processor to function such that a first stream of a first data type including a motion, a sound, a comment, or information according to gifting is retrieved, a second stream of a second data type including a video is retrieved, data according to the first stream and data according to the second stream are synthesized, and the synthesized data is provided. In addition, the first data type includes intermediate information according to the video. In addition, the intermediate information includes depth information according to the video.

Claims (81)

1. A non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to

retrieve a first stream of a first data type including a motion, a sound, a comment, or information relating to gifting, and a second stream of a second data type including a background video, the first data type including intermediate information including depth information relating to the background video and depth information relating to the motion, wherein the motion is a motion of an avatar;

determine whether or not to perform synthesis by using the intermediate information;

generate synthesized data from the first stream and the second stream based on the determination, the synthesized data relating to the first data type and the second data type, such that the motion of the avatar is synthesized with the background video; and

transmit the synthesized data.

2. The medium according to claim 1 ,

wherein a third stream of the second data type is retrieved, and

the synthesized data and the third stream are synthesized.

3. The medium according to claim 1 ,

wherein a fourth stream of the first data type is retrieved, and

the synthesized data and the fourth stream are synthesized.

4. The medium according to claim 1 ,

wherein the first stream includes first time information and the second stream includes second time information, and

the first time information and the second time information are adjusted to have a predetermined relationship, and the first stream and the second stream are synthesized.

5. The medium according to claim 1 ,

wherein synthesis of the synthesized data is performed by a server.

6. The medium according to claim 1 ,

wherein the first stream is retrieved from a first information source terminal,

the second stream is retrieved from a second information source terminal, and

synthesis of the synthesized data is performed by the first information source terminal or the second information source terminal.

7. The medium according to claim 1 ,

wherein synthesis of the synthesized data is performed by a user terminal, and the synthesized data is provided by the user terminal.

8. The medium according to claim 1 ,

wherein the synthesized data is provided to one or a plurality of user terminals.

9. The medium according to claim 1 ,

wherein information relating to a viewpoint in a game space is retrieved from one user terminal, and the viewpoint in the game space is changed.

10. The medium according to claim 9 ,

wherein the viewpoint in the game space is changed to a viewpoint in a game space according to any one user of a user terminal set in advance.

11. The medium according to claim 9 ,

wherein the viewpoint in the game space is changed to a viewpoint in one location in a game space other than a user terminal set in advance.

12. The medium according to claim 11 ,

wherein the change to the viewpoint in the one location in the game space other than the user terminal set in advance is performed by using a provision of a predetermined valuable product as a condition.

13. The medium according to claim 1 ,

wherein production based on one synthesized data item that is provided in one user terminal is different from production based on one synthesized data item that is provided in another user terminal.

14. The medium according to claim 1 ,

wherein the first data type includes the information relating to gifting,

the second data type includes a video that is subjected to moving image distribution, and

synthesis of the synthesized data includes reflection of the information relating to gifting on the video that is subjected to the moving image distribution.

15. The medium according to claim 1 ,

wherein the information relating to gifting, the sound, or the comment is subjected to a predetermined setting based on one or a plurality of specific information source terminals, and is not provided to another one of the one or a plurality of specific user terminals.

16. The medium according to claim 15 ,

wherein the predetermined setting is performed by control information based on one user terminal.

17. The medium according to claim 1 ,

wherein the sound or the comment is provided in accordance with a user attribute associated with one user terminal.

18. The medium according to claim 17 ,

wherein the sound or the comment is provided to an extent according to the user attribute associated with the one user terminal.

19. The medium according to claim 17 ,

wherein the user attribute includes a viewing period.

20. The medium according to claim 17 ,

wherein the user attribute includes the location in a game space.

21. The medium according to claim 17 ,

wherein the user attribute includes presence or absence of provision of a valuable product or an extent of the provision.

22. The medium according to claim 1 ,

wherein the processor is a central processing unit (CPU), a microprocessor, or a graphics processing unit (GPU).

23. The medium according to claim 1 ,

wherein the processor is mounted on a smartphone, a tablet, a mobile phone, or a personal computer.

24. A server device, comprising:

a processor,

wherein the processor is configured to:

retrieve a first stream of a first data type including a motion, a sound, a comment, or information relating to gifting, the first data type including intermediate information including depth information relating to a background video and depth information relating to the motion, wherein the motion is a motion of an avatar,

retrieve a second stream of a second data type including the background video,

determine whether or not to perform synthesis by using the intermediate information,

generate synthesized data relating to the first data type and the second data type based on the determination, such that the motion of the avatar is synthesized with the background video, and

transmit the synthesized data, by executing a computer readable command.

25. A terminal device, comprising:

a processor,

wherein the processor is configured to:

retrieve a first stream of a first data type including a motion, a sound, a comment, or information relating to gifting, the first data type including intermediate information including depth information relating to a background video and depth information relating to the motion, wherein the motion is a motion of an avatar,

retrieve a second stream of a second data type including the background video,

determine whether or not to perform synthesis by using the intermediate information,

generate synthesized data relating to the first data type and the second data type based on the determination, such that the motion of the avatar is synthesized with the background video, and

provide the synthesized data, by executing a computer readable command.

26. A method to be executed by a processor executing a computer readable command, the method comprising:

retrieving a first stream of a first data type including a motion, a sound, a comment, or information relating to gifting, the first data type including intermediate information including depth information relating to the background video and depth information relating to the motion, wherein the motion is a motion of an avatar,

retrieving a second stream of a second data type including the background video,

determining whether or not to perform synthesis by using the intermediate information,

generating synthesized data relating to the first data type and the second data type based on the determination, such that the motion of the avatar is synthesized with the background video, and

providing the synthesized data, by executing the command.

27. The medium according to claim 1 , wherein the avatar is a first avatar and wherein the second stream of the second data type further comprises a motion of a second avatar.

28. The medium according to claim 6 , wherein the avatar is a first avatar and wherein the second stream of the second data type further comprises a motion of a second avatar.

29. The medium according to claim 28 , wherein the first information source terminal is the first avatar and the second information source terminal is the second avatar.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY STREET ADDRESS AND ATTORNEY DOCKET NUMBER PREVIOUSLY RECORDED AT REEL: 71308 FRAME: 765. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Jun 10, 2025
From: GREE, INC.
To: GREE HOLDINGS, INC.
Reel/Frame 071611/0252 →
CHANGE OF NAME Recorded May 16, 2025
From: GREE, INC.
To: GREE HOLDINGS, INC.
Reel/Frame 071308/0765 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2020
From: SHIRAI, AKIHIKO; YAMANOUCHI, YOUSUKE
To: GREE, INC.
Reel/Frame 053873/0754 →
Priority Claims (1)
JP 2019-177503 · Sep 27, 2019 · national
Continuity (1)
Related Publication 20210099735A1 · Apr 1, 2021