IP Library › Granted Patent US 12,614,345
Granted Patent B2
US 12,614,345 · App. 18/545,911 · Granted Apr 28, 2026

Streaming a compressed light field

Inventors: Michael Stengel (Cupertino, CA); Alexander Majercik (San Francisco, CA); Ben Boudaoud (Efland, NC); Morgan McGuire (Williamstown, MA)
Assignee: NVIDIA CORPORATION
G06T15/506G06T15/04G06T15/06H04L67/131H04N19/46
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,614,345
App. No.
18/545,911
Granted
Apr 28, 2026
Kind
B2
Abstract

A remote device utilizes ray tracing to compute a light field for a scene to be rendered, where the light field includes information about light reflected off surfaces within the scene. This light field is then compressed utilizing one or more video compression techniques that implement temporal reuse, such that only differences between the light field for the scene and a light field for a previous scene are compressed. The compressed light field data is then sent to a client device that decompresses the light field data and uses such data to obtain the light field for the scene at the client device. This light field is then used by the client device to compute global illumination for the scene. The global illumination may be used to accurately render the scene at the mobile device, resulting in a realistic scene that is presented by the mobile device.

Claims (44)

1 . A method comprising:

receiving compressed light field data for a scene, wherein the compressed light field data is a compressed representation of a portion of a light field computed for the scene that is changed from a prior light field obtained for a previous scene;

receiving an indication of the portion of the light field that is included in the compressed light field data;

decompressing the compressed light field data to obtain the portion of the light field for the scene;

obtaining the light field for the scene by combining the portion of the light field for the scene with the prior light field obtained for the previous scene based on the indication;

computing global illumination for the scene, using the light field for the scene.

2 . The method of claim 1 , wherein the indication is received as an index buffer that maps coordinates of the portion of the light field to the prior light field obtained for the previous scene.

3 . A method comprising:

receiving compressed light field data for a scene, wherein the compressed light field data is a compressed representation of a portion of a light field computed for the scene that is changed from a prior light field obtained for a previous scene, wherein the light field includes an array of blocks, each block in the array of blocks including color texture information and visibility texture information;

decompressing the compressed light field data to obtain the portion of the light field for the scene;

obtaining the light field for the scene by combining the portion of the light field for the scene with the prior light field obtained for the previous scene;

computing global illumination for the scene, using the light field for the scene.

4 . The method of claim 3 , wherein the compressed light field data is received from a remote device at a client device, where the remote device includes a node within a cloud-based computing environment, and the client device includes one or more of a mobile computing device, a portable gaming platform, and a virtual reality (VR) headset.

5 . The method of claim 3 , wherein the compressed light field data is received via one or more of a wireless internet connection and a cellular communications network.

6 . The method of claim 3 , wherein a client device decompresses the compressed light field data using a video decompression unit.

7 . The method of claim 3 , wherein a client device performs lookups within the light field for the scene when computing global illumination for the scene, instead of performing ray tracing.

8 . The method of claim 3 , further comprising rendering the scene at a client device, using the computed global illumination.

9 . The method of claim 8 , further comprising displaying the rendered scene by the client device.

10 . The method of claim 3 , wherein an indirect light model indicates light is reflected off of surfaces onto other surfaces within the scene.

11 . The method of claim 3 , wherein a direct light model indicates light that hits a surface directly from a light source.

12 . The method of claim 3 , wherein the color texture information includes lighting color information within the block.

13 . The method of claim 3 , wherein the visibility texture information includes a distance to a closest surface within the block.

14 . The method of claim 3 , wherein the scene and the previous scene are consecutive scenes to be rendered.

15 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, causes the processor to:

receiving compressed light field data for a scene, wherein the compressed light field data is a compressed representation of a portion of a light field computed for the scene that is changed from a prior light field obtained for a previous scene, wherein the light field includes an array of blocks, each block in the array of blocks including color texture information and visibility texture information;

decompressing the compressed light field data to obtain the portion of the light field for the scene;

obtaining the light field for the scene by combining the portion of the light field for the scene with the prior light field obtained for the previous scene; and

computing global illumination for the scene, using the light field for the scene.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the compressed light field data is received from a remote device at a client device, where the remote device includes a node within a cloud-based computing environment, and the client device includes one or more of a mobile computing device, a portable gaming platform, and a virtual reality (VR) headset.

17 . The non-transitory computer-readable storage medium of claim 15 , wherein the compressed light field data is received via one or more of a wireless internet connection and a cellular communications network.

18 . The non-transitory computer-readable storage medium of claim 15 , wherein a client device decompresses the compressed light field data using a video decompression unit.

19 . The non-transitory computer-readable storage medium of claim 15 , wherein a client device performs lookups within the light field for the scene when computing global illumination for the scene, instead of performing ray tracing.

20 . The non-transitory computer-readable storage medium of claim 15 , wherein the steps further comprise rendering the scene at a client device, using the computed global illumination.

21 . The non-transitory computer-readable storage medium of claim 20 , wherein the steps further comprise displaying the rendered scene by the client device.

22 . The non-transitory computer-readable storage medium of claim 15 , wherein an indirect light model indicates light is reflected off of surfaces onto other surfaces within the scene.

23 . The non-transitory computer-readable storage medium of claim 15 , wherein a direct light model indicates light that hits a surface directly from a light source.

24 . A system, comprising:

a non-transitory memory comprising instructions; and

one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:

receive compressed light field data for a scene, wherein the compressed light field data is a compressed representation of a portion of a light field computed for the scene that is changed from a prior light field obtained for a previous scene, wherein the light field includes an array of blocks, each block in the array of blocks including color texture information and visibility texture information;

decompress the compressed light field data to obtain the portion of the light field for the scene;

obtain the light field for the scene by combining the portion of the light field for the scene with the prior light field obtained for the previous scene; and

compute global illumination for the scene, using the light field for the scene.

25 . The system of claim 24 , wherein the non-transitory memory and the one or more processors are components of a client device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2024
From: STENGEL, MICHAEL; MAJERCIK, ALEXANDER; BOUDAOUD, BEN; MCGUIRE, MORGAN
To: NVIDIA CORPORATION
Reel/Frame 066655/0084 →
Continuity (3)
Continuation 17177011 · Feb 16, 2021
Provisional Application 63054681 · Jul 21, 2020
Related Publication 20240119664A1 · Apr 11, 2024
References Cited (222)
US 6567083B1 · Baum et al. · 2003 [cited by applicant]
US 7995059B1 · Arvo · 2011 [cited by applicant]
US 8147339B1 · Perry · 2012 [cited by applicant]
US 8587588B2 · Smyth · 2013 [cited by applicant]
US 10008034B2 · McGuire et al. · 2018 [cited by applicant]
US 10262451B1 · Chou · 2019 [cited by examiner]
US 10713838B2 · McGuire et al. · 2020 [cited by applicant]
US 11501467B2 · Stengel et al. · 2022 [cited by applicant]
US 20010037402A1 · Schneider · 2001 [cited by applicant]
US 20050041024A1 · Green et al. · 2005 [cited by applicant]
US 20060066616A1 · Sevastianov et al. · 2006 [cited by applicant]
US 20070139433A1 · Anderson et al. · 2007 [cited by applicant]
US 20070229502A1 · Tong et al. · 2007 [cited by applicant]
US 20080018647A1 · Bunnell · 2008 [cited by applicant]
US 20080143720A1 · Elmquist · 2008 [cited by applicant]
US 20080207322A1 · Mizrahi · 2008 [cited by applicant]
US 20090167763A1 · Waechter et al. · 2009 [cited by applicant]
US 20090254293A1 · Tartaglia et al. · 2009 [cited by applicant]
US 20100033493A1 · Nutter et al. · 2010 [cited by applicant]
US 20100315423A1 · Ahn et al. · 2010 [cited by applicant]
US 20110012901A1 · Kaplanyan · 2011 [cited by applicant]
US 20110043523A1 · Kim et al. · 2011 [cited by applicant]
US 20110122135A1 · Kim et al. · 2011 [cited by applicant]
US 20110175913A1 · Sloan et al. · 2011 [cited by applicant]
US 20120114037A1 · Winder et al. · 2012 [cited by applicant]
US 20120212491A1 · Hager · 2012 [cited by applicant]
US 20120256915A1 · Jenkins · 2012 [cited by applicant]
US 20120313944A1 · Kontkanen et al. · 2012 [cited by applicant]
US 20130038618A1 · Urbach · 2013 [cited by applicant]
US 20130113800A1 · McCombe et al. · 2013 [cited by applicant]
US 20130120385A1 · Krishnaswamy et al. · 2013 [cited by applicant]
US 20130137511A1 · Bae et al. · 2013 [cited by applicant]
US 20130321417A1 · Van Den Berghe et al. · 2013 [cited by applicant]
US 20130335434A1 · Wang et al. · 2013 [cited by applicant]
US 20140035900A1 · Slavin, III et al. · 2014 [cited by applicant]
US 20140139513A1 · Mammou · 2014 [cited by applicant]
US 20140173674A1 · Wolman et al. · 2014 [cited by applicant]
US 20140232719A1 · Wahrenberg · 2014 [cited by applicant]
US 20140267271A1 · Billeter et al. · 2014 [cited by applicant]
US 20140285499A1 · Iwasaki · 2014 [cited by applicant]
US 20140327690A1 · McGuire · 2014 [cited by examiner]
US 20140375659A1 · Mcguire et al. · 2014 [cited by applicant]
US 20150325035A1 · Howell · 2015 [cited by applicant]
US 20180081429A1 · Akenine-Moller · 2018 [cited by examiner]
US 20180278939A1 · Abercrombie · 2018 [cited by applicant]
US 20180322691A1 · Mitchell et al. · 2018 [cited by applicant]
US 20190166359A1 · Lapstun · 2019 [cited by applicant]
US 20190236831A1 · Akenine-Moller et al. · 2019 [cited by applicant]
US 20200275074A1 · Hamilton et al. · 2020 [cited by applicant]
US 20200312018A1 · McGuire et al. · 2020 [cited by applicant]
US 20220028158A1 · Stengel et al. · 2022 [cited by applicant]
US 20220138988A1 · Stengel et al. · 2022 [cited by applicant]
CN 101458824A · 2009 [cited by applicant]
CN 105263050A · 2016 [cited by applicant]
CN 106662749A · 2017 [cited by applicant]
CN 109564704A · 2019 [cited by applicant]
CN 110024002A · 2019 [cited by applicant]
CN 110728740A · 2020 [cited by applicant]
JP 2012004636A · 2012 [cited by applicant]
Slater M, Mortensen J, Khanna p. Yu I. A virtual light field approach to global illumination. In Proceedings Computer Graphics International, 2004. Jun. 19, 2004 (pp. 102-109). IEEE. (Year: 2004). [cited by examiner]
Hajisharif S, Miandji E, Larsson p. Tran K, Unger J. Light field video compression and real time rendering. In Computer Graphics Forum Oct. 2019 (vol. 38, No. 7, pp. 265-276). (Year: 2019). [cited by examiner]
Chan SC, Ng KT, Gan ZF, Chan KL, Shum HY. The plenoptic video. IEEE Transactions on Circuits and Systems for Video Technology. Dec. 31, 2005; 15(12):1650-9. (Year: 2005). [cited by examiner]
Liu Y, Dai Q, Xu W. A real time interactive dynamic light field transmission system. In2006 IEEE International Conference on Multimedia and Expo Jul. 9, 2006 (pp. 2173-2176). IEEE. (Year: 2006). [cited by examiner]
Han Z, Dai Q. A new scalable free viewpoint video streaming system over IP network. In2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP'07 Apr. 15, 2007 (vol. 2, pp. II-773). IEEE. (Ye… [cited by examiner]
Cai W, Cheung G, Kwon T, Lee SJ. Optimized frame structure for interactive light field streaming with cooperative caching. In2011 IEEE International Conference on Multimedia and Expo Jul. 11, 2011 (pp. 1-6). IEEE. (Year… [cited by examiner]
Kovács PT, Zare A, Balogh T, Bregovic R, Gotchev A. Architectures and codecs for real-time light field streaming. Journal of Imaging Science and Technology. Jan. 1, 2017;61(1):010403. (Year: 2017). [cited by examiner]
Hattachi et al., “NGMN 5G White Paper,” NGMN Alliance, Feb. 17, 2015, 125 pages. [cited by applicant]
Bugeja et al., “An asynchronous method for cloud-based rendering,” The Visual Computer, Dec. 2019, 15 pages. [cited by applicant]
Bierton, D., “Face-off: Gaikai vs. OnLive,” Eurogamer, Mar. 24, 2012, 17 pages, retrieved from https://www.eurogamer.net/articles/digitalfoundry-face-off-gaikai-vs-onlive. [cited by applicant]
Chen et al., “A Framework for Adaptive Residual Streaming for Single-Player Cloud Gaming,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 15, No. 2s, No. 66, Jul. 2019, 23 pages. [cited by applicant]
Collet et al., “Zstandard Compression and the application/zstd Media Type,” RFC 8478, Internet Engineering Task Force (IETF), Oct. 2018, 55 pages. [cited by applicant]
Crassin et al., “CloudLight: A System for Amortizing Indirect Lighting in Real-Time Rendering,” Journal of Computer Graphics Techniques vol. 4, No. 4, 2015, pp. 1-27. [cited by applicant]
Choy et al., “The Brewing Storm in Cloud Gaming: A Measurement Study on Cloud to End-User Latency,” NetGames 2012: The 11th ACM Annual Workshop on Network and Systems Support for Games, Nov. 2012, 8 pages. [cited by applicant]
Dayal et al., “Adaptive Frameless Rendering,” Eurographics Symposium on Rendering, Jan. 2005, 12 pages. [cited by applicant]
Enet, “Enet Reliable UDP networking library,” Enet, v1.3.17, 2020, 1 page, retrieved from http://enet.bespin.org/. [cited by applicant]
Geutskens, Y., “The HDR Ecosystem Tracker (mid-2019),” flatpanelshd, Jun. 4, 2019, 18 pages, retrieved from https://www.flatpanelshd.com/focus.php?subaction=showfull&id=1559638820. [cited by applicant]
Google, “Bandwidth, data usage, and stream quality,” Google, Stadia Help, 2021, 3 pages, retrieved from https://support.google.com/stadia/answer/9607891?hl=en. [cited by applicant]
GPD, “GPD WIN 2,” GPD GamePod Digital, 2018, 15 pages, retrieved from https://www.gpd.hk/gdpwin2. [cited by applicant]
Greger et al., “The irradiance volume,” IEEE Computer Graphics and Applications, vol. 18, 1998, 21 pages, retrieved from https://www.researchgate.net/publication/3208677_The_Irradiance_Volume. [cited by applicant]
Jansen et al., “5G vs. 4G: How will the newest network improve on the last?” digitaltrends, Feb. 26, 2021, 11 pages, retrieved from https://www.digitaltrends.com/mobile/5g-vs-4g/. [cited by applicant]
Hladky et al., “The Camera Offset Space: Real-time Potentially Visible Set Computations for Streaming Rendering,” ACM Transactions on Graphics, vol. 38, No. 6, Nov. 2019, pp. 231:1-231:14. [cited by applicant]
Hladky et al., “Tessellated Shading Streaming,” Eurographics Symposium on Rendering, vol. 38, No. 4, 2019, 13 pages. [cited by applicant]
HDR10+, “HDR10+ System Whitepaper,” HDR10+ Technologies, LLC, Sep. 4, 2019, 14 pages. [cited by applicant]
Hillesland et al., “Texel shading,” Eurographics Proceedings, 2016, 4 pages, retrieved from https://gpuopen.com/learn/texel-shading/. [cited by applicant]
Koroneos et al., “Customers in Chicago and Minneapolis are first in the world to get 5g-enabled smartphones connected to a 5g network,” Verizon, Apr. 3, 2019, 4 pages, retrieved from https://www.verizon.com/about/news/c… [cited by applicant]
Liu et al., “Hybrid Lossless-Lossy Compression for Real-Time Depth-Sensor Streams in 3D Telepresence Applications,” Pacific Rim Conference on Multimedia, 2015, 10 pages, retrieved from https://www.semanticscholar.org/pa… [cited by applicant]
Levoy, M., “Polygon-Assisted JPEG and MPEG Compression of Synthetic Images,” Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, 1995, pp. 21-28. [cited by applicant]
Lindstrom, P., “Fixed-Rate Compressed Floating-Point Arrays,” IEEE Transactions on Visualization and Computer Graphics, Aug. 2014, 11 pages. [cited by applicant]
Mcgarry, C., “5G speed: 5G vs 4G performance compared,” tom's guide, 2021, 20 pages, retrieved from https://www.tomsguide.com/features/5g-vs-4g. [cited by applicant]
Majercik et al., “Dynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields,” Journal of Computer Graphics Techniques vol. 8, No. 2, 2019, pp. 1-30. [cited by applicant]
Morgan, T., “Doom Eternal on Stadia looks great—but the lag is just too high,” Eurogamer, Mar. 2020, 9 pages, retrieved from https://www.eurogamer.net/articles/digitalfoundry-2020-doom-eternal-stadia-looks-the-part-but-… [cited by applicant]
Mueller et al., “Shading Atlas Streaming,” ACM Transactions on Graphics, vol. 37, No. 6, Nov. 2018, pp. 199:1-199:16. [cited by applicant]
Nystad et al., “Adaptive Scalable Texture Compression,” High Performance Graphics, 2012, pp. 105-114. [cited by applicant]
NVIDIA, “Video Encode and Decode GPU Support Matrix,” NVIDIA Developer, 2021, 1 page, retrieved from https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new. [cited by applicant]
NVIDIA, “Geforce Now System Requirements,” NVIDIA, 2021, 6 pages, retrieved from https://www.nvidia.com/en-us/geforce-now/system-reqs/. [cited by applicant]
NVIDIA, “Jetson Xavier Nx,” NVIDIA Developer, 2021, 4 pages, retrieved from https://developer.nvidia.com/embedded/jetson-xavier-nx. [cited by applicant]
Speedtest, “Speedtest Global Index,” Speedtest, 2021, 13 pages, retrieved https://www.speedtest.net/reports/united-states/#mobile. [cited by applicant]
Pajak et al., “Scalable Remote Rendering with Depth and Motion-flow Augmented Streaming,” Eurographics, vol. 30, No. 2, 2011, 10 pages. [cited by applicant]
Qualcomm, “Snapdragon 865 5G Mobile Platform,” Qualcomm, 2021, 12 pages, retrieved from https://www.qualcomm.com/products/snapdragon-865-5g-mobile-platform. [cited by applicant]
Richardson et al., “The Remote Framebuffer Protocol,” Internet Engineering Task Force, RFC 6143, Mar. 2011, pp. 1-40. [cited by applicant]
Scherzer et al., “Pre-convolved Radiance Caching,” Eurographics, vol. 31, No. 4, 2012, 7 pages. [cited by applicant]
Sullivan et al., “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, No. 12, Dec. 2012, pp. 1649-1668. [cited by applicant]
Schwarz et al., “Emerging MPEG Standards for Point Cloud Compression,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, No. 1, Mar. 2019, pp. 133-148. [cited by applicant]
Tom's Guide Staff, “Fastest wireless network 2020: It's not even close,” tom's guide, Dec. 6, 2019, pp. 1-19. [cited by applicant]
Stereolabs, “H.264/H.265 Video Encoding Support Matrix for Nvidia Jetson,” stereolabs.com, 2020, 5 pages, retrieved from https://www.stereolabs.com/blog/h-264-h-265-video-encoding-support-matrix-for-nvidia-jetson/. [cited by applicant]
Suvorov, A., “Crunch compression of ETC textures,” Unity Blog, Dec. 15, 2017, 31 pages, retrieved from https://blogs.unity3d.com/2017/12/15/crunch-compression-of-etc-textures/. [cited by applicant]
Van Waveren et al., “Real-Time YCoCg-DXT Compression,” NVIDIA, Sep. 14, 2007, 53 pages, retrieved from https://developer.download.nvidia.com/whitepapers/2007/Real-Time-YCoCg-DXT-Compression/Real-Time%20YCoCg-DXT%20Compr… [cited by applicant]
Wilson, A., “Fast Lossless Depth Image Compression,” ACM International Conference on Interactive Surfaces, 2017, pp. 100-105. [cited by applicant]
Zabriskie, N., “NetLight: Cloud Baked Indirect Illumination,” Thesis, Brigham Young University, 37 pages, retrieved from https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=8031&context=etd. [cited by applicant]
Magro et al., “Interactive Cloud-based Global Illumination for Shared Virtual Environments,” IEEE 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), 2019, 8 pages. [cited by applicant]
Schelkens et al., “JPEG2000 Part 10—Volumetric Data Encoding,” IEEE ISCAS, 2006, pp. 3874-3877. [cited by applicant]
Leaf et al., “In Situ Video Encoding of Floating-Point Volume Data Using Special-Purpose Hardware for a Posteriori Rendering and Analysis,” IEEE Symposium on Large Data Analysis and Visualization, 2017, pp. 64-73. [cited by applicant]
Gamito et al., “Lossless Coding of Floating Point Data with JPEG 2000 Part 10,” Proceedings of SPIE, The International Society for Optical Engineering, Nov. 2004, 13 pages. [cited by applicant]
Stengel et al., U.S. Appl. No. 17/177,011, filed Feb. 16, 2021. [cited by applicant]
Stengel et al., U.S. Appl. No. 17/308,893, filed May 5, 2021. [cited by applicant]
Mcguire et al., U.S. Appl. No. 14/270,252, filed May 5, 2014. [cited by applicant]
Ziv et al., “Compression of Individual Sequences via Variable-Rate Coding,” IEEE Transactions on Information Theory, vol. IT-24, No. 5, Sep. 1978, pp. 530-536. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 17/308,893, dated May 23, 2022. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 17/177,011, dated Sep. 20, 2023. [cited by applicant]
Whitted, T., “An improved illumination model for shaded display,” ACM SIGGRAPH Computer Graphics, vol. 13, Issue 2, Aug. 1979, pp. 1-6. [cited by applicant]
Akeley et al., “Minimum triangle Separation for Correct Z-Buffer Occlusion,” Graphics Hardware, Eurographics Association, 2006, pp. 27-30. [cited by applicant]
Akenine-Moller et al., “Real-time rendering,” 3rd Edition, A. K. Peters, Ltd., 2008, pp. 1-1027. [cited by applicant]
Arikan et al., “Fast and detailed approximate global illumination by irradiance decomposition,” Computer Graphics Proceedings, Annual Conference Series, 2005, pp. 1108-1114. [cited by applicant]
Bavoil et al., “Multi-Layer Dual-Resolution Screen-Space Ambient Occlusion,” SIGGRAPH 2009 Talks, Article 45, Aug. 2009, p. 1. [cited by applicant]
Blinn et al., “Texture and reflection in computer generated images,” Graphics Image Processing, Communications of the ACM, vol. 19, No. 10, Oct. 1976, pp. 542-547. [cited by applicant]
Filion et al., “Effects & Techniques,” STARCRAFT, Advances in Real-Time Rendering in 3D Graphics and Games Course, SIGGRAPH 2008 Games, Aug. 2008, Chapter 5, pp. 133-164. [cited by applicant]
Green et al., “Efficient Self-Shadowed Radiosity Normal Mapping,” Vlave, 2007, pp. 1-4. [cited by applicant]
Greene, N., “Environment Mapping and Other Applications of World Projections,” Computer Graphics and Applications, IEEE, vol. 6, No. 11, Nov. 1986, pp. 21-29. [cited by applicant]
Hoang et al., “Efficient screen-space approach to high-quality multiscale ambient occlusion,” The Visual Computer: International Journal of Computer Graphics, vol. 28, Issue 3, Mar. 2012, pp. 289-304. [cited by applicant]
Immel, et al., “A Radiosity Method For Non-Diffuse Environments,” ACM SIGGRAPH Computer Graphics, vol. 20, No. 4, Aug. 18-22, 1986, pp. 133-142. [cited by applicant]
Jensen, H., “Global Illumination Using Photon Maps,” Proceedings of the Seventh Eurographics Workshop on Rendering, 1996, pp. 21-30, pp. 1-17. [cited by applicant]
Kajiya, J.T., “The Rendering Equation,” ACM SIGGRAPH Computer Graphics, vol. 20, No. 4, Aug. 18-22, 1986, pp. 143-150. [cited by applicant]
Kaplanyan, A., “CryENGINE 3: Reaching the speed of light,” SIGGRAPH 2010 Courses, Los Angeles, CA, ACM, Sep. 17, 2010, 99 slides. [cited by applicant]
King, G., “Real-Time Computation of Dynamic Irradiance Environment Maps,” GPU Gems 2, Apr. 2005, pp. 1-14, retrieved from http.developer.nvidia.com/GPUGems2/gpugems2_chapter10.html. [cited by applicant]
Lafortune, E., “Mathematical Models and Monte Carlo Algorithms for Physically Based Rendering,” Department of Computer Science, Faculty of Engineering, Katholieke Universiteit Leuven, 1996, pp. 1-142. [cited by applicant]
Lapidous et al., “Optimal Depth Buffer for Low-Cost Graphics Hardware,” Eurographics, 1999, pp. 67-73. [cited by applicant]
Levoy, M., “Polygon-Assisted JPEG and MPEG Compression of Synthetic Images,” Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '95, Sep. 1995, pp. 21-28. [cited by applicant]
Loos et al., “Volumetric Obscurance,” Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Feb. 2010, pp. 151-156. [cited by applicant]
Mcguire, M., “Ambient Occlusion Volumes,” Proceedings of the Conference on High Performance Graphics, Eurographics Association, Jun. 2010, 9 pages. [cited by applicant]
Mcguire et al., “The Alchemy Screen-Space Ambient Obscurance Algorithm,” Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics, Aug. 2011, pp. 25-32. [cited by applicant]
Mctaggart, G., “Half-life 2 / Valve Source Shading,” Direct3D Tutorial Day, Game Developers Conference, Valve Corporation, Mar. 22, 2004, 97 slides, retrieved from http://www.valvesoftware.com/publications/2004/GDC2004_… [cited by applicant]
Miller et al., “Illumination and Reflection Maps: Simulated Objects in Simulated and Real Environments,” SIGGRAPH 84, Course Notes for Advanced Computer Graphics Animation, Jul. 23, 1984, pp. 1-12. [cited by applicant]
Mittring, M., “Finding next gen: CryEngine 2,” Advanced Real-Time Rendering in 3D Graphics and Games Course—SIGGRAPH 2007, Aug. 2007, CRYTEK, Chapter 8, pp. 97-121. [cited by applicant]
Mcguire et al., “Scalable Ambient Obscurance,” Proceedings of the Fourth ACM SIGGRAPH/ Eurographics Conference on High-Performance Graphics, Eurographics Association, Jun. 2012, pp. 97-103. [cited by applicant]
Nicodemus et al., “Geometrical Considerations and Nomenclature for Reflectance,” Institute for Basic Standards, National Bureau of Standards, U.S. Dept of Commerce, Oct. 1977, 67 pages. [cited by applicant]
Noguera et al., “Navigating large terrains using commodity mobile devices,” Computers and Geosciences, vol. 37, No. 9, Sep. 2011, pp. 1218-1233. [cited by applicant]
NVIDIA, “GPU Gems 2,” NVIDIA Developer, Apr. 2005, 25 pages, retrieved from https://developer.nvidia.com/gpugems/gpugems2/copyright. [cited by applicant]
Ramamoorthi et al., “An Efficient Representation for Irradiance Environment Maps,” ACM SIGGRAPH, Aug. 12-17, 2001, pp. 497-500. [cited by applicant]
Rose, M., “OnLive Works With Game Devs On Newly-Launched Tablet, Smartphone Initiative,” Dec. 8, 2011, p. 1, retrieved at http://www.gamasutra.com/view/news/128588/OnLive_Works_With_Game_Devs_On_NewlyLaunched_Tablet_Sma… [cited by applicant]
Shanmugam et al., “Hardware Accelerated Ambient Occlusion Techniques on GPUs,” Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games I3D, Apr. 2007, pp. 73-80. [cited by applicant]
Sloan, P.P., “Stupid Spherical Harmonics (SH) Tricks,” Game Developers Conference, vol. 9, Feb. 2008, 42 pages. [cited by applicant]
Smith, A. R., “The viewing transformation,” Technical Memo 84, Jun. 24, 1983, 27 pages. [cited by applicant]
Laszlo et al., “Volumetric Ambient Occlusion for Real-Time Rendering and Games,” IEEE Computer Graphics and Applications, Jan. 2010, pp. 70-79. [cited by applicant]
Scheuermann et al., “Cubemap Filtering with CubeMapGen,” ATI: 3D Application Research Group, Game Developers Conference Talk, 2005, 27 slides. [cited by applicant]
Upchurch et al., “Tightening the Precision of Perspective Rendering,” preprint for the Journal of Graphics Tools, vol. 16, Issue 1, 2012, 6 pages. [cited by applicant]
Veach et al., “Metropolis Light Transport,” Aug. 1997, pp. 65-76. [cited by applicant]
Williams, L., “Casting Curved Shadows On Curved Surfaces,” ACM SIGGRAPH Computer Graphics, vol. 12, Issue 3, Aug. 1978, pp. 270-274. [cited by applicant]
Foley, J. D. et al., “A Tutorial on Satellite Graphics Systems,” Computer, Aug. 1976, pp. 14-21. [cited by applicant]
Broxton et al., “Immersive Light Field Video with a Layered Mesh Representation,” ACM Transactions on Graphics, vol. 39, No. 4, Jul. 2020, pp. 86:1-86:15. [cited by applicant]
Cai et al., “Toward Gaming as a Service,” IEEE Internet Computing, May/Jun. 2014, pp. 12-18. [cited by applicant]
Illahi et al., “Cloud Gaming with Foveated Video Encoding,” ACM Transactions on Multimedia Computing Communications and Applications, Mar. 2020, pp. 7:1-7:24. [cited by applicant]
Kamarainen et al., “CloudVR: Cloud Accelerated Interactive Mobile Virtual Reality,” Proceedings of the 26th ACM international conference on Multimedia, Oct. 2018, pp. 1181-1189. [cited by applicant]
Shea et al., “Cloud Gaming: Architecture and Performance,” IEEE Network, Jul./Aug. 2013, pp. 16-21. [cited by applicant]
Slivar et al., “Empirical QoE Study of In-Home Streaming of Online Games,” IEEE 13th Annual Workshop on Network and Systems Support for Games, 2014, 6 pages. [cited by applicant]
Tolia et al., “Quantifying Interactive User Experience on Thin Clients,” Computer, IEEE, 2006, pp. 46-52. [cited by applicant]
Liu et al., “Lightweight WebSIM Rendering Framework Based on Cloud-Baking,” Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, 2017, pp. 221-229. [cited by applicant]
Liu et al., “Cloud Baking: Collaborative Scene Illumination for DynamicWeb3D Scenes,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 14, No. 3s, Article 59, Jun. 2018, pp. 59:1-59:20. [cited by applicant]
Wijnants et al., “Standards-compliant HTTP Adaptive Streaming of Static Light Fields,” Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, 2017, 12 pages. [cited by applicant]
Matela, J., “Designing Image Compression Algorithms for Massively Parallel Processors,” Masaryk University, Ph. D. Thesis, 2016, 131 pages. [cited by applicant]
Office Action from Chinese Patent Application No. 202110818935.0, dated Mar. 25, 2025, 9 pages. [cited by applicant]
Ziv et al., “A Universal Algorithm for Sequential Data Compression,” IEEE Transactions on Information Theory, vol. IT-23, No. 3, May 1977, pp. 337-343. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 14/270,252, dated Sep. 24, 2015. [cited by applicant]
Final Office Action from U.S. Appl. No. 14/270,252, dated May 26, 2016. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 14/270,252, dated May 3, 2017. [cited by applicant]
Final Office Action from U.S. Appl. No. 14/270,252, dated Jan. 23, 2018. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 14/270,252, dated Sep. 20, 2018. [cited by applicant]
Final Office Action from U.S. Appl. No. 14/270,252, dated Feb. 7, 2019. [cited by applicant]
Advisory Action from U.S. Appl. No. 14/270,252, dated Jul. 3, 2019. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 14/270,252, dated Sep. 17, 2019. [cited by applicant]
Notice of Allowance from U.S. Appl. No. 14/270,252, dated Mar. 11, 2020. [cited by applicant]
Corrected Notice of Allowance from U.S. Appl. No. 14/270,252, dated Mar. 25, 2020. [cited by applicant]
Christensen et al., “An Irradiance Atlas for Global Illumination in Complex Production Scenes,” Eurographics Symposium on Rendering, 2004, pp. 1-10. [cited by applicant]
Pacanowski et al., “Efficient Streaming of 3D Scenes with Complex Geometry and Complex Lighting,” 2008, pp. 1-7. [cited by applicant]
Damez et al., “State of the Art in Global Illumination for Interactive Applications and High-quality Animations,” Computer Graphics Forum, vol. 21, No. 4, 2003, pp. 55-77. [cited by applicant]
Waters, Z., “Photon Mapping,” 2006, 8 pages, retrieved from https://web.cs.wpi.edu/˜emmanuel/courses/cs563/write_ups/zackw/photon_mapping/PhotonMapping.html. [cited by applicant]
Xu et al., “Real-time Rendering of Dynamic Objects in Dynamic, Low-frequency Lighting Environments,” Semantic Scholar, 2005, 7 pages, retrieved from https://www.semanticscholar.org/paper/Real-time-Rendering-of-Dynamic-O… [cited by applicant]
Klionsky, Z., “A New Architecture for Cloud Rendering and Amortized Graphics,” Thesis, Aug. 2011, 47 pages. [cited by applicant]
Wyman et al., “Interactive Display of Isosurfaces with Global Illumination,” IEEE Transactions on Visualization and Computer Graphics, vol. 12, No. 2, 2006, pp. 186-196. [cited by applicant]
Meneveaux et al., “Photon Streaming for Interactive Global Illumination in Dynamic Scenes,” HAL archives-ouvertes.fr, Oct. 2008, 19 pages. [cited by applicant]
Armbrust et al., “A view of cloud computing,” Communications of the ACM, vol. 53, No. 4, Apr. 2010, pp. 50-58. [cited by applicant]
Boukerche et al., “Remote rendering and streaming of progressive panoramas for mobile devices,” Proceedings of the 14th Annual ACM International Conference on Multimedia, Oct. 23-27, 2006, pp. 691-694. [cited by applicant]
Boulton, M., “Static lighting tricks in Halo 4,” GDC Presentation, 2013, 17 slides. [cited by applicant]
Brodlie, et al., “Distributed and Collaborative Visualization,” Eurographics, 2003, pp. 1-28. [cited by applicant]
Chalmers et al., “Practical Parallel Rendering,” A K Peters, 2002, 392 pages. [cited by applicant]
Chang et al., “Practical Strategies for Power-Efficient Computing Technologies,” Proceedings of the IEEE, vol. 98, No. 2, Feb. 2010, pp. 215-236. [cited by applicant]
Chen et al., “Measuring The Latency of Cloud Gaming Systems,” In Proceedings of the 19th ACM International Conference on Multimedia, ACM, Nov. 28-Dec. 1, 2011, pp. 1269-1272. [cited by applicant]
Choy et al., “The Brewing Storm in Cloud Gaming: A Measurement Study on Cloud to End-User Latency,” NetGames 2012: The 11th ACM Annual Workshop on Network and Systems Support for Games, Nov. 2012, pp. 1-7. [cited by applicant]
Cohen et al., “A progressive refinement approach to fast radiosity image generation,” In ACM SIGGRAPH Computer Graphics, vol. 22, No. 4, Aug. 1988, pp. 75-84. [cited by applicant]
Crassin et al., “Interactive Indirect Illumination Using Voxel Cone Tracing,” Computer Graphics Forum, Wiley-Blackwell, vol. 30, No. 7, Sep. 2011, pp. 1921-1930. [cited by applicant]
Crassin et al., “CloudLight: A system for amortizing indirect lighting in real-time rendering,” NVIDIA Technical Report NVR-2013-001, Jul. 2013, pp. 1-10. [cited by applicant]
Dutre et al., “Advanced Global Illumination,” SIGGRAPH 2002 Course 2, 2002, pp. 1-25. [cited by applicant]
Hu et al., “Visual Cues for Imminent Object Contact in Realistic Virtual Environment,” In Proceedings of the Conference on Visualization '00, IEEE Computer Society Press, Oct. 2000, pp. 179-185. [cited by applicant]
Jensen, H. W., “Realistic Image Synthesis Using Photon Mapping,” A. K. Peters, Ltd., 2001, pp. 1-181. [cited by applicant]
Karapantazis et al., “VolP: A comprehensive survey on a promising technology,” Computer Networks, vol. 53, 2009, pp. 2050-2090. [cited by applicant]
Koller et al., “Protected interactive 3D graphics via remote rendering,” ACM Transactions on Graphics (TOG), Aug. 2004, vol. 23, No. 3, pp. 695-703. [cited by applicant]
Lawton, G., “Cloud Streaming Brings Video to Mobile Devices,” Computer, vol. 45, Issue 2, Feb. 2012, pp. 14-16. [cited by applicant]
Loos et al., “Modular radiance transfer,” ACM Transactions on Graphics (TOG), Proceedings of ACM SIGGRAPH Asia 2011, vol. 30, No. 6, Dec. 2011, Article No. 178, pp. 1-10. [cited by applicant]
Luke et al., “Semotus Visum: A Flexible Remote Visualization Framework,” IEEE Visualization, Oct. 27-Nov. 1, 2002, pp. 61-68. [cited by applicant]
Manzano et al., “An empirical study of Cloud Gaming,” Annual Workshop on Network and Systems Support for Games, Nov. 2012, 3 pages. [cited by applicant]
Mara, M., “Cloudlight: A Distributed Global Illumination System For Real-Time Rendering,” Thesis, May 13, 2012, pp. 1-62. [cited by applicant]
Mara et al., “Toward practical real-time photon mapping: efficient GPU density estimation,” Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Mar. 2013, 9 pages, retrieved from https://casu… [cited by applicant]
Martin et al., “A Real-Time Radiosity Architecture for Video Games,” Geomerics, Electronic Arts (EA) DICE, SIGGRAPH 2010 Courses, Slideshow, Aug. 1, 2010, 38 pages. [cited by applicant]
Mitchell et al., “Shading in Valve's Source Engine,” Advanced Real-Time Rendering in 3D Graphics and Games Course—ACM SIGGRAPH 2006, Chapter 8, pp. 8-1-8-14. [cited by applicant]
Parker et al., “OptiX: A General Purpose Ray Tracing Engine,” ACM Transactions on Graphics, vol. 29, No. 4, Article No. 66, Jul. 2010, 66:1-66:13. [cited by applicant]
Paul et al., “Chromium renderserver: Scalable and open remote rendering infrastructure,” Visualization and Computer Graphics, IEEE Transactions, vol. 14, Issue 3, Jun. 3, 2008, pp. 627-639. [cited by applicant]
Ritschel et al., “The State Of The Art In Interactive Global Illumination,” Computer Graphics forum, Blackwell Publishing Ltd., vol. 0, No. 0, 1981, pp. 1-27. [cited by applicant]
Rao et al., “Network characteristics of video streaming traffic,” CoNEXT '11, Dec. 6-9, 2011, pp. 1-12. [cited by applicant]
Shi et al., “A Real-Time Remote Rendering System for Interactive Mobile Graphics,” ACM Transactions on Multimedia Computing, Communications and Applications, vol. 8, No. 3s, Article 46, Sep. 2012, pp. 46:1-46:20. [cited by applicant]
Sinha et al., “Recovering reflectance and illumination in a world of painted polyhedra,” Fourth International Conference on Computer Vision, May 11-14, 1993, pp. 156-163. [cited by applicant]
Suznjevic et al., “MMORPG player actions: Network performance, session patterns and latency requirements analysis,” Multimedia Tools and Applications, vol. 45, No. 1-3, Oct. 2009, pp. 191-214. [cited by applicant]
Tamm et al., “Hybrid Distributed Rendering,” IEEE LDAV Symposium 2012, pp. 1-2. [cited by applicant]
Second Office Action from Chinese Patent Application No. 202110818935.0, dated Oct. 29, 2025. [cited by applicant]