Centralised interactive graphical application server
This invention relates to a method of processing a plurality of graphical programs on a centralized computer system whereby the images produced by the programs are compressed and transmitted to remote processing devices where they are decompressed. Compression assistance data (CAD) is produced by inspecting instructions outputted by the programs and the CAD is then used in the compression step.
1. A method of generating compressed image data comprising the steps of:
a) outputting graphics instructions by a graphics generating computer program;
b) identifying at least one of the outputted graphics instructions having an influence on a co-ordinate associated with the image data;
c) generating compression assistance data responsive to the identified graphics instruction, the compression assistance data comprises information relating to the co-ordinate of at least one pixel in the plurality of pixels of step (d);
d) processing at least one of the outputted graphics instructions to generate a plurality of pixels;
e) processing at least one of the plurality of pixels using at least part of the compression assistance data to generate the compressed image data.
2. The method as claimed in claim 1 wherein step (c) occurs before step (d).
3. The method as claimed in claim 2 wherein the generation of the compressed image data utilizes motion vectors corresponding to at least one of the plurality of pixels and its relative position between two different points of time.
4. The method as claimed in claim 3 wherein the motion vectors are determined in step (c).
5. The method as claimed in claim 4 wherein the step of identifying at least one of the outputted graphics instructions comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
6. The method as claimed in claim 5 wherein the outputted graphics instructions are DirectX instructions or OpenGL instructions.
7. The method as claimed in claim 6 wherein at least one graphic instruction of the outputted graphics instructions is a shader instruction.
8. The method as claimed in claim 7 wherein the processing of step (d) occurs in a GPU.
9. The method as claimed in claim 2 wherein the outputted graphics instructions are OpenGL instructions or DirectX instructions .
10. The method as claimed in claim 1 wherein the generation of the compressed image data utilizes motion vectors corresponding to at least one of the plurality of pixels and its relative position between two different points of time.
11. The method as claimed in claim 10 wherein the motion vectors are determined in step (c).
12. The method as claimed in claim 11 wherein at least one of the outputted graphics instructions is intercepted by a wrapper before the compression assistance data is generated.
13. The method as claimed in claim 10 wherein at least one of the outputted graphics instructions is intercepted by a wrapper before the compression assistance data is generated.
14. The method as claimed in claim 1 wherein at least one of the outputted graphics instructions is intercepted by a wrapper before the compression assistance data is generated.
15. The method as claimed in claim 1 wherein the step of identifying at least one of the outputted graphics instructions comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
16. The method as claimed in claim 1 wherein at least one of the outputted graphics instructions of step (d) is modified before being processed.
17. The method as claimed in claim 16 wherein the modifying is responsive to the identified instruction.
18. A method of generating a plurality of compressed image data transmissions comprising the steps of:
a) outputting graphics instructions by a first graphics generating computer program;
b) outputting graphics instructions by a second graphics generating computer program;
c) identifying at least one of the outputted graphics instructions having an influence on a co-ordinate associated with the image data;
d) modifying at least one of the graphics instructions outputted by the first program to create a first set of modified graphics instructions wherein the modifying is responsive to the identified instruction;
e) modifying at least one of the graphics instructions outputted by the second program to create a second set of modified graphics instructions;
f) processing at least one graphics instruction of the first set of modified graphics instructions to generate compression assistance data, the compression assistance data comprises information relating to the co-ordinate of at least one pixel in the plurality of pixels in step (g) or of step (h);
g) processing at least one of the outputted graphics instructions from the first program or at least one graphics instruction of the first set of modified graphics instructions or a combination thereof to generate a first plurality of pixels;
h) processing at least one of the outputted graphics instructions from the second program or at least one graphics instruction of the second set of modified graphics instructions or a combination thereof to generate a second plurality of pixels;
i) processing at least one of the first plurality of pixels using at least part of the compression assistance data to generate a first of the compressed image data transmissions;
j) processing at least one of the second plurality of pixels to generate a second of the compressed image data transmissions;
k) transmitting the first of the compressed image data transmissions to a first remote decompression device; and
l) transmitting the second of the compressed image data transmissions to a second remote decompression device.
19. The method as claimed in claim 18 wherein step (f) occurs before step (g).
20. The method as claimed in claim 19 wherein the generation of the first of the compressed image data transmissions utilizes motion vectors corresponding to at least one of the first plurality of pixels and its relative position between two different points of time.
21. The method as claimed in claim 20 wherein the motion vectors are determined in step (f).
22. The method as claimed in claim 21 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
23. The method as claimed in claim 22 wherein the outputted graphics instructions are OpenGL instructions or DirectX instructions.
24. The method as claimed in claim 19 wherein the outputted graphics instructions are OpenGL instructions or DirectX instructions.
25. The method as claimed in claim 24 wherein the steps of modifying comprise at least one step selected from a group consisting of adding at least one graphics instruction, removing at least one of the outputted graphics instructions, replacing at least one of the outputted graphics instructions, translating at least one of the outputted graphics instructions, altering at least one of the outputted graphics instructions and any combination thereof.
26. The method as claimed in claim 25 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
27. The method as claimed in claim 24 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of indentifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
28. The method as claimed in claim 18 wherein the generation of the first of the compressed image data transmissions utilizes motion vectors corresponding to at least one of the first plurality of pixels and its relative position between two different points of time.
29. The method as claimed in claim 28 wherein the motion vectors are determined in step (f).
30. The method as claimed in claim 29 wherein at least one of the graphics instructions outputted by the first program is intercepted by a wrapper before the compression assistance data is generated.
31. The method as claimed in claim 28 wherein at least one of the graphics instructions outputted by the first program is intercepted by a wrapper before the compression assistance data is generated.
32. The method as claimed in claim 18 wherein the outputted graphics instructions are OpenGL instructions or DirectX instructions.
33. The method as claimed in claim 32 wherein the processing of step (g) is performed by a GPU.
34. The method as claimed in claim 33 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
35. The method as claimed in claim 32 wherein the generation of the first of the compressed image data transmissions utilizes motion vectors corresponding to at least one of the first plurality of pixels and its relative position between two different points of time.
36. The method as claimed in claim 32 wherein the steps of modifying comprise at least one step selected from a group consisting of adding at least one graphics instruction, removing at least one of the outputted graphics instructions, replacing at least one of the outputted graphics instructions, translating at least one of the outputted graphics instructions and any combination thereof.
37. The method as claimed in claim 36 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
38. The method as claimed in claim 37 wherein at least one of the graphics instructions outputted by the first program is a shader instruction.
39. The method as claimed in claim 36 wherein at least one of the graphics instructions outputted by the first program is a shader instruction.
40. The method as claimed in claim 32 wherein the steps of modifying comprise at least one step selected from a group consisting of adding at least one graphics instruction, removing at least one of the outputted graphics instructions, replacing at least one of the outputted graphics instructions, translating at least one of the outputted graphics instructions, altering at least one of the outputted graphics instructions and any combination thereof.
41. The method as claimed in claim 40 wherein the step of identifying an attribute comprises at least one step selected from a group consisting of identifying the type of one of the outputted graphics instructions, identifying the contents of one of the outputted graphics instructions and any combination thereof.
42. The method as claimed in claim 41 wherein at least one of the graphics instructions outputted by the first program is a shader instruction.
43. The method as claimed in claim 40 wherein at least one of the graphics instructions outputted by the first program is a shader instruction.