IP Library Granted Patent US 12,118,639
Granted Patent B2
US 12,118,639 · App. 17/146,828 · Granted Oct 15, 2024

Method and system for interactive graphics streaming

Inventor: Adam Billyard (Sussex, GB)
Assignee: Mythical, Inc.
G06T1/20G06F3/14G06T1/60H04L65/60G06T2200/16G06T2200/24H04L67/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,118,639
App. No.
17/146,828
Granted
Oct 15, 2024
Kind
B2
Abstract

The present invention relates to a method of streaming interactive computer graphics from a server to a client device. The method includes the steps of: intercepting graphics instructions transmitted from an application destined for a graphical processing unit (GPU) at the server; processing the graphics instructions to generate graphics data at the server; generating index information for, at least, some of the graphics data at the server; transmitting the index information in place of the graphics data to a client device; extracting corresponding graphics data stored at the client device utilizing the index information; and rendering computer graphics at a graphical processing unit (GPU) at the client device using the corresponding graphics data. A system for streaming interactive computer graphics is also disclosed.

Claims (34)

1. A method of streaming interactive computer graphics from a server to a client device, wherein the client device includes local storage and a client-side graphical processing unit (GPU), the method including:

a) intercepting, at the server, graphics instructions transmitted from an application destined for a graphical processing unit (GPU) at the server, wherein the application is being executed at the server;

b) processing the graphics instructions to generate graphics data at the server, wherein the graphics data includes a vertex buffer, wherein the vertex buffer is divided into blocks, and wherein the vertex buffer is further divided into stripes corresponding to different vertex fields;

c) generating index information for at least some of the graphics data at the server, including particular index information that indexes one or more portions of the vertex buffer that have been modified due to execution of one or more of the graphics instructions, wherein the index information is generated for a run of blocks in the vertex buffer that have been modified, and wherein individual values of the index information are generated for individual ones of the stripes that have been modified;

d) transmitting the index information in place of the graphics data from the server to the client device, wherein the transmitted index information includes the particular index information that indexes the one or more portions of the vertex buffer that have been modified;

e) extracting, by the client device, corresponding graphics data that corresponds to the graphics data, wherein the corresponding graphics data is stored in the local storage, and wherein extracting the corresponding graphics data utilizes the transmitted index information; and

f) rendering computer graphics by the client-side graphical processing unit (GPU) using the corresponding graphics data and the transmitted index information.

2. The method of claim 1 , wherein the graphics data includes one or more from the set of graphics state, static resources, and dynamic resources.

3. The method of claim 1 , wherein objects within the graphics data are hashed to generate the index information.

4. The method of claim 1 , wherein the index information is generated for a trimmed run of blocks that have been modified such that the run extends from a first modified bit within a first block in the trimmed run to a last modified bit within a last block in the run.

5. The method of claim 1 , further including:

synchronizing the graphics data between the server and the client.

6. The method of claim 5 , wherein, when the graphics data includes textures, only data for the textures that are used are synchronized.

7. The method of claim 5 , wherein a profile of the client device determines the synchronization of graphics data.

8. The method of claim 7 , wherein the profile of the client device determines the synchronization of graphics data by assigning lower resolution graphics for higher resolution graphics at the client device.

9. The method of claim 1 , further comprising:

displaying the rendered computer graphics on a display at the client device.

10. The method of claim 9 , further comprising:

receiving, by the client device, user input in response to the displayed computer graphics; and

transmitting the user input back to the application that is being executed on the server.

11. The method of claim 10 , wherein the user input is transmitted, at least in part, using User Datagram Protocol (UDP).

12. The method of claim 11 , wherein State Transition Events are synthesized on the server.

13. The method of claim 1 , wherein the application is selected for execution at the server by a user from a plurality of applications.

14. A client device configured for use with the method of claim 1 .

15. A system for streaming interactive computer graphics, including:

a server configured for:

(i) executing an application,

(ii) intercepting graphics instructions transmitted from the application destined for a graphical processing unit (GPU) at the server,

(iii) processing the graphics instructions to generate graphics data, wherein the graphics data includes a vertex buffer, wherein the vertex buffer is divided into blocks, and wherein the vertex buffer is further divided into stripes corresponding to different vertex fields, wherein the different vertex fields include vertex color, vertex texture coordinate, and vertex position,

(iv) generating index information for at least some of the graphics data at the server, including particular index information that indexes one or more portions of the vertex buffer that have been modified due to execution of one or more of the graphics instructions, wherein the index information is generated for a run of blocks in the vertex buffer that are dirty, and wherein individual values of the index information are generated for individual ones of the stripes that have been modified, and

(v) transmitting the index information in place of the graphics data from the server to a client device, wherein the transmitted index information includes the particular index information that indexes the one or more portions of the vertex buffer that have been modified; and

the client device configured for:

(i) extracting corresponding graphics data that corresponds to the graphics data, wherein the corresponding graphics data is stored in local storage at the client device, and wherein extracting the corresponding graphics data utilizes the transmitted index information; and

(ii) rendering computer graphics at a client-side graphical processing unit (GPU) using the corresponding graphics data and the transmitted index information.

Assignments (5)
SECURITY INTEREST Recorded Jan 6, 2025
From: MYTHICAL, INC.; DMARKET INC.
To: NFL PROPERTIES LLC (AS COLLATERAL AGENT)
Reel/Frame 069827/0309 →
SECURITY INTEREST Recorded Jan 30, 2024
From: MYTHICAL, INC.; DMARKET INC.
To: RHENIUM SOF I-A MYTHICAL 2023, LP; GALAXY EOS VC FUND, L.P.
Reel/Frame 066381/0852 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2022
From: BILLYARD, ADAM
To: ELEKTRAGLIDE LTD
Reel/Frame 060417/0947 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 30, 2022
From: MYTHICAL GAMES UK LTD
To: MYTHICAL, INC.
Reel/Frame 060418/0493 →
CHANGE OF NAME Recorded Jun 30, 2022
From: ELEKTRAGLIDE LTD
To: MYTHICAL GAMES UK LTD
Reel/Frame 060558/0529 →
Continuity (4)
Continuation 16528862 · Aug 1, 2019
Continuation In Part 15113176
Provisional Application 61929538 · Jan 21, 2014
Related Publication 20210133916A1 · May 6, 2021