IP Library Granted Patent US 11,322,043
Granted Patent B2
US 11,322,043 · App. 16/862,753 · Granted May 3, 2022

Remote multiplayer interactive physical gaming with mobile computing devices

Inventors: Keng Fai Lee (Cupertino, CA); Qi Zhang (Hong Kong, HK); Wing Hung Chan (Hong Kong, HK); Wang Fai Ng (Hong Kong, HK); Tin Yin Lam (Fremont, CA); Alexander Wu (San Francisco, CA); Ian Smith (Mountain View, CA); David Richard Schmierer (Campbell, CA); Bradford Lewis Farkas (New York City, NY); Kin Ming Law (Hong Kong, HK)
Assignee: NEX Team Inc.
G09B19/0038A63B24/0062A63B71/0622G06K9/00342G06K9/00724G09B5/04A63B69/0071A63B71/0669A63B2071/0694A63B2220/806G06K2009/00738
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,322,043
App. No.
16/862,753
Granted
May 3, 2022
Kind
B2
Abstract

Methods and systems for remote multiplayer gameplay are disclosed. An example method can include joining a multiplayer interactive physical game including first and second players competing in an activity; capturing a first game video of a first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area; determining a first object flow associated with the first player from the first game video, by performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player; generating a first analytic based on the first object flow; receiving a second analytic associated with the second player in a second gaming area; and generating a feedback to the first player based on the first analytic or the second analytic.

Claims (45)

1. A method for multiplayer interactive physical gaming with at least two mobile computing devices, comprising:

joining a multiplayer interactive physical game including at least a first player and a second player, wherein the first player and the second player participate in an activity;

capturing a first game video of the first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area;

determining a first object flow associated with the first player from the first game video, by (a) performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player and to determine an affinity field map, and (b) determining a pose of the first player based on the key points and the affinity field map;

generating a first analytic based on the first object flow;

receiving a second analytic associated with the second player in a second gaming area; and

generating a feedback to the first player based on at least one of the first analytic and the second analytic.

2. The method of claim 1 , wherein the second gaming area is geographically different from the first gaming area.

3. The method of claim 1 , wherein the first player and the second player of the multiplayer interactive physical game are playing asynchronously in time.

4. The method of claim 1 , further comprising:

receiving a user input from the first player to initiate the multiplayer interactive physical game, wherein the user input is selected from the group consisting of a voice input and a gesture input to the first mobile computing device.

5. The method of claim 1 , further comprising:

detecting a scoring attempt from the first game video of the multiplayer interactive physical game, and wherein the generating of the first analytic is based on an outcome of the scoring attempt.

6. The method of claim 1 , further comprising:

providing the feedback to the first player using at least one of an audio output and a video output associated with the first mobile computing device.

7. The method of claim 1 , wherein the generating of the feedback is live during the first play of the multiplayer interactive physical game.

8. The method of claim 1 , wherein the feedback to the first player comprises instructions for actions to be performed by the first player.

9. The method of claim 1 , further comprising:

determining a location of a portion of a body of the first player in the first gaming area from the first game video, wherein the generating of the feedback is further based on the location.

10. A system for remote multiplayer interactive physical gaming with mobile computing devices, comprising:

at least one processor on a mobile computing device; and

a non-transitory physical medium for storing program code accessible by the at least one processor, the program code when executed by the processor causes the processor to:

join a multiplayer interactive physical game including at least a first player and a second player, wherein the first player and the second player participate in an activity;

capture a first game video of the first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area;

determine a first object flow associated with the first player from the first game video, by (a) performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player and to determine an affinity field map, and (b) determining a pose of the first player based on the key points and the affinity field map;

generate a first analytic based on the first object flow;

receive a second analytic associated with the second player in a second gaming area; and

generate a feedback to the first player based on at least one of the first analytic and the second analytic.

11. The system of claim 10 , wherein the second mobile computing device is in a second gaming area that is geographically different from the first gaming area.

12. The system of claim 10 , wherein the first player and the second player of the multiplayer interactive physical game are playing asynchronously in time.

13. The system of claim 10 , wherein the program code when executed by the processor further causes the processor to:

receive a user input from the first player to initiate the multiplayer interactive physical game, wherein the user input is selected from the group consisting of a voice input and a gesture input to the first mobile computing device.

14. The system of claim 10 , wherein the program code when executed by the processor further causes the processor to:

detect a scoring attempt from the first game video of the multiplayer interactive physical game, and wherein the program code that causes the processor to generate the first analytic is based on an outcome of the scoring attempt.

15. The system of claim 10 , wherein the program code when executed by the processor further causes the processor to:

provide the feedback to the first player using at least one of an audio output and a video output associated with the first mobile computing device.

16. The system of claim 10 , wherein the generating the feedback is live during the first play of the multiplayer interactive physical game.

17. The system of claim 10 , wherein the feedback to the first player comprises instructions for actions to be performed by the first player.

18. A non-transitory physical storage medium for remote multiplayer interactive physical gaming with mobile computing devices, the storage medium comprising program code stored thereon, and the program code when executed by a processor causes the processor to:

join a multiplayer interactive physical game including at least a first player and a second player, wherein the first player and the second player participate in an activity;

capture a first game video of the first player in a first play of the multiplayer interactive physical game, using a first camera on a first mobile computing device in a first gaming area;

determine a first object flow associated with the first player from the first game video, by (a) performing an algorithm on one or more frames of the first game video to detect one or more key points of the first player and to determine an affinity field map, and (b) determining a pose of the first player based on the key points and the affinity field map;

generate a first analytic based on the first object flow;

receive a second analytic associated with the second player in a second gaming area; and

generate a feedback to the first player based on at least one of the first analytic and the second analytic.

Assignments (2)
SECURITY INTEREST Recorded Oct 9, 2025
From: NEX TEAM INC.
To: SILVER LAKE WATERMAN FUND III, L.P., AS AGENT
Reel/Frame 072524/0882 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2020
From: LEE, KENG FAI; ZHANG, QI; CHAN, WING HUNG; NG, WANG FAI; LAM, TIN YIN; WU, ALEXANDER; SMITH, IAN; SCHMIERER, DAVID RICHARD; FARKAS, BRADFORD LEWIS; LAW, KIN MING
To: NEX TEAM, INC.
Reel/Frame 052555/0954 →
Continuity (3)
Continuation 16445893 · Jun 19, 2019
Provisional Application 62687393 · Jun 20, 2018
Related Publication 20200258419A1 · Aug 13, 2020