IP Library Granted Patent US 11,465,044
Granted Patent B2
US 11,465,044 · App. 16/121,135 · Granted Oct 11, 2022

Method and system for providing game based on video call and object recognition

Inventors: Keum Ryong Kim (Seongnam-si, KR); Seeun Kim (Seongnam-si, KR); Jung Young Lee (Seongnam-si, KR)
Assignee: LINE Corporation
A63F13/35A63F13/25A63F13/87G06V20/40G06V40/113G06V40/171H04N7/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,465,044
App. No.
16/121,135
Granted
Oct 11, 2022
Kind
B2
Abstract

Provided are methods and systems for providing a game based on a video call and object recognition. A game providing method of an electronic device configured as a computer including displaying game content on a video call screen in response to a video call being established between a user of the electronic device and a counterpart through a voice over Internet protocol (VoIP) channel, recognizing an object in a call image as motion data with respect to the game content, and controlling a game element corresponding to the game content using a result of the recognizing may be provided.

Claims (60)

1. A game providing method of a first electronic device configured as a computer, the method comprising:

executing game content on a video call screen in response to a video call being established between a user of the first electronic device and a counterpart of a second electronic device, the video call screen including a user image of the user and a counterpart image of the counterpart, the game content being concurrently executed on the video call screen of each of the first electronic device and the second electronic device;

in response to a first recognition of recognizing first coordinates of a first touch of the counterpart on the user image of the video call screen of the second electronic device, generating a first synthetic image in which a first texture of a first effect corresponding to a result of recognizing a first position that matches the received first coordinates on the user image is synthesized;

controlling a first game content portion of the game content associated with the user based on the first recognition;

in response to a second recognition of recognizing second coordinates of a second touch of the user on the counterpart image of the video call screen of the first electronic, generating a second synthetic image in which a second texture of a second effect corresponding to a result of recognizing a second positon that matches the received second coordinates on the counterpart image is synthesized;

controlling a second game content portion of the game content associated with the counterpart based on the second recognition; and

displaying a result of the game content based on a first result of controlling the first game content portion and a second result of controlling the second game content portion,

wherein the first recognition and the second recognition are concurrently performed based on a progress of the game content concurrently executed on the video call screen of each of the first electronic device and the second electronic device.

2. The method of claim 1 , further comprising:

synthesizing the first texture of the first effect corresponding to a third result of recognizing at least one game element of the game content corresponding to the user, and

displaying the first synthetic image.

3. The method of claim 1 , wherein

the first recognition and the second recognition comprise a recognition of a face of the user in the user image and a recognition of a face of the counterpart in the counterpart image, respectively, using face recognition technology.

4. The method of claim 1 , wherein

the first recognition comprises receiving first gesture information of the counterpart from the second electronic device of the counterpart and recognizing a position that matches location information associated with the first gesture information in the user image.

5. The method of claim 1 , further comprising:

displaying the video call screen including the user image of the user and the counterpart image of the counterpart by exchanging the user image and the counterpart image in real time in response to the video call being established.

6. The method of claim 1 , further comprising:

sharing information associated with the progress of the game content corresponding to the first result of controlling the first game content portion and the second result of controlling the second game content portion with the first electronic device and the second electronic device,

wherein the displaying comprises determining a win and loss of the user and the counterpart based on the first result of controlling the first game content portion and the second result of controlling the second game content portion and displaying the determined win and loss on the video call screen.

7. The method of claim 1 , further comprising:

displaying the result of the game content on the video call screen during the video call in response to a selection of the game content by the user or the counterpart in a messenger chatroom, which corresponds to a communication session set between a user account of the user and a counterpart account of the counterpart.

8. The method of claim 1 , further comprising:

providing the game content selected by the user or the counterpart in response to the video call being established between the user and the counterpart.

9. The method of claim 1 , the first recognition further comprises a recognition of voice information of the user, and the second recognition further comprises a recognition of voice information of the counterpart.

10. A non-transitory computer-readable recording medium storing instructions that, when executed by a processor, cause the computer to perform the game providing method of claim 1 .

11. The method of claim 1 , further comprising:

receiving, from the second electronic device of the counterpart, data associated with a third result of recognizing a motion of an object in the counterpart image, the object based on a type of the game content in the counterpart image, and

displaying the second synthetic image of the counterpart image based on the received data, the second synthetic image being an image of one of game elements of the game content to which the second effect corresponding to a result of recognizing the object of the counterpart has been synthesized to one of the game elements corresponding to the counterpart, on the video call screen.

12. The method of claim 1 , wherein the displaying includes displaying game guide information together with the game content on the video call screen in response to the video call being established between the user of the first electronic device and the counterpart of the second electronic device.

13. The method of claim 11 , further comprising:

displaying, on the video call screen during the game content being executed, the first synthetic image and the second synthetic image,

wherein, in the second electronic device, in response to a corresponding motion of the object of the counterpart being performed, the corresponding motion of the object is recognized in the second synthetic image for controlling the game content.

14. A game providing system of a first electronic device configured as a computer, the game providing system comprising:

a memory configured to store computer-readable instructions; and

at least one processor configured to execute the computer-readable instructions, the at least one processor is configured to cause the first electronic device to,

execute game content on a video call screen in response to a video call being established between a user of the first electronic device and a counterpart of a second electronic device, the video call screen including a user image of the user and a counterpart image of the counterpart, the game content being concurrently executed on the video call screen of each of the first electronic device and the second electronic device,

in response to a first recognition of recognizing first coordinates of a first touch of the counterpart on the user image of the video call screen of the second electronic device, generating a first synthetic image in which a first texture of a first effect corresponding to a result of recognizing a first position that matches the received first coordinates on the user image is synthesized,

control a first game content portion of the game content associated with the user based on the first recognition,

in response to a second recognition of recognizing second coordinates of a second touch of the user on the counterpart image of the video call screen of the first electronic device, generate a second synthetic image in which a second texture of a second effect corresponding to a result of recognizing a second position that matches the received second coordinates on the counterpart image is synthesized,

control a second game content portion of the game content associated with the counterpart based on the second recognition, and

display a result of the game content based on a first result of controlling the first game content portion and a second result of controlling the second game content portion,

wherein the at least one processor is configured to cause the first electronic device to perform the first recognition and the second recognition concurrently based on a progress of the game content concurrently executed on the video call screen of each of the first electronic device and the second electronic device.

15. The game providing system of claim 14 , wherein the at least one processor is further configured to cause the first electronic device to,

synthesize the first texture of the first effect corresponding to a third result of recognizing at least one game element of the game content corresponding to the user, and

display the first synthetic image.

16. The game providing system of claim 14 , wherein

the at least one processor is configured to the first electronic device to display the video call screen including the user image of the user and the counterpart image of the counterpart by exchanging the user image and the counterpart image in real time in response to the video call being established.

17. The game providing system of claim 14 , wherein

the at least one processor is configured to cause the first electronic device to recognize a face of the user in the user image using face recognition technology.

18. The game providing system of claim 14 , wherein

the at least one processor is configured to cause the first electronic device to perform the first recognition by receiving first gesture information of the counterpart from the second electronic device of the counterpart and recognizes a position that matches location information associated with the first gesture information in the user image.

19. The game providing system of claim 14 , wherein the at least one processor is further configured to cause the first electronic device to,

share information associated with the progress of the game content corresponding to the first result of controlling the first game content portion and the second result of controlling the second game content portion with the first electronic device and the second electronic device,

determine a win and loss of the user and the counterpart based on the first result of controlling the first game content portion and the second result of controlling the second game content portion, and

display the determined win and loss on the video call screen.

20. The game providing system of claim 14 , wherein the at least one processor is configured to cause the first electronic device to display the result of the game content on the video call screen during the video call in response to selecting the game content by the user or the counterpart in a messenger chatroom, which corresponds to a communication session set between a user account of the user and a counterpart account of the counterpart.

21. The game providing system of claim 14 , wherein

the at least one processor is further configured to display, on the video call screen during the game content being executed, the first synthetic image and the second synthetic image in which the second effect associated with the game content is synthesized with an object of the counterpart included in the counterpart image, and

the second electronic device is configured to recognize a corresponding motion of the object in the second synthetic image for controlling the game content, in response to the corresponding motion of the object of the counterpart being performed.

Assignments (7)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2024
From: Z INTERMEDIATE GLOBAL CORPORATION
To: LY CORPORATION
Reel/Frame 067096/0431 →
CHANGE OF NAME Recorded Apr 10, 2024
From: LINE CORPORATION
To: Z INTERMEDIATE GLOBAL CORPORATION
Reel/Frame 067069/0467 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF THE ASSIGNEES CITY IN THE ADDRESS SHOULD BE TOKYO, JAPAN PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0303. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 062401/0490 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE CITY SHOULD BE SPELLED AS TOKYO PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0141. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 062401/0328 →
CHANGE OF NAME Recorded Dec 28, 2021
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 058597/0141 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 058597/0303 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2018
From: KIM, KEUM RYONG; KIM, SEEUN; LEE, JUNG YOUNG
To: LINE CORPORATION
Reel/Frame 046797/0190 →
Priority Claims (1)
KR 10-2017-0114552 · Sep 7, 2017 · national
Continuity (1)
Related Publication 20190070500A1 · Mar 7, 2019
Cited By (1)
US 12,626,432