IP Library Granted Patent US 12,205,194
Granted Patent B2
US 12,205,194 · App. 18/375,134 · Granted Jan 21, 2025

Multi-dimensional 3D engine computing and virtualization-based dynamic load balancing of virtual or real worlds

Inventors: Cevat Yerli (Dubai, AE); Prashanth Hirematada (Frankfurt am Main, DE)
Assignee: TMRW FOUNDATION IP S.ÀR.L.
G06T1/20G06T1/60G06T17/005H04L67/01H04L67/10G06T2210/36
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,205,194
App. No.
18/375,134
Granted
Jan 21, 2025
Kind
B2
Abstract

A system enabling a distributed 3D engine for performing dynamic load balancing through virtual worlds are provided. The system comprises one or more server computers comprising memory and at least one processor, the memory storing a data structure representing at least one portion of a virtual or real world as a plurality of cells storing virtual objects. The memory further stores a distributed 3D engine comprising a resource manager implemented in a distributed deployment and a plurality of individual software engines. Resources are dynamically allocated via the distributed deployment to one or more cells based on a current load and a corresponding computed and ranked demand. In some embodiments, the demand further considers one or more of an amount of virtual objects and level of interactions within the portion of the persistent virtual world system visible to a user avatar. Methods thereof are also provided.

Claims (47)

1. A system comprising:

one or more server computers comprising memory and at least one processor, the memory storing:

a data structure virtualizing at least one portion of a virtual or real world into a plurality of cells storing virtual objects forming a persistent virtual world system; and

a distributed three-dimensional (3D) engine, the distributed 3D engine comprising a plurality of distributed software engines,

wherein resources are dynamically allocated via a distributed deployment to the plurality of cells based on a current load of the plurality of cells;

wherein an original number of the plurality of cells representing one or more areas of the at least one portion of the virtual or real world are partitioned into a greater number of cells,

wherein the resources are restored based on one or more requests, and

wherein each cell of the plurality of cells comprises one or more streams, each stream comprising a plurality of stream-specific virtual objects being programmed to be enabled or disabled for viewing on and interacting with client devices, and wherein each stream is associated with one or more applications.

2. The system of claim 1 , wherein the at least one portion of the virtual or real world is consolidated back into the original number of cells.

3. The system of claim 1 , wherein the persistent virtual world system comprises a low fidelity simulation used for demand assessment and load balancing, and a high fidelity simulation used for improving user experience.

4. The system of claim 1 , wherein at least one of the virtual objects of the persistent virtual world system comprise self-computing capabilities and autonomous behavior.

5. The system of claim 1 , wherein the distributed software engines are used sequentially or in parallel, through the distributed deployment, to complement engine services of each other for realization of at least one task.

6. The system of claim 1 , wherein the data structure is an octree data structure,

wherein at least one of the plurality cells is represented as a voxel within the octree data structure, and

wherein the voxel representing the at least one of the plurality of cells is selected from among a sparse voxel and a dense voxel based on a size of the at least one of the plurality of cells.

7. The system of claim 1 , wherein the data structure comprises at least one of BSP trees, sparse voxel octrees, 3D arrays, kD trees, point clouds, wire-frames, boundary representations (B-Rep), constructive solid geometry trees (CSG Trees), bintrees, or hexagonal structures, or combinations thereof.

8. The system of claim 1 , wherein a resource manager of a distributed 3D engine performs the allocation of the resources through a distributed message exchange platform,

wherein the distributed message exchange platform utilizes a publish-subscribe model, and

wherein at least one of the virtual objects subscribe to at least one of the plurality of cells where the resources are published.

9. The system of claim 8 , wherein the distributed message exchange platform shares a dynamically updated state of the at least one portion of the virtual or real world stored in the memory with one or more client devices or servers,

wherein the dynamically updated state is modified through data obtained by one or more of a plurality of connected devices including sensors providing sensor data to the persistent virtual world system, by user input, by server computations, or combinations thereof.

10. A method comprising:

providing, in memory of at least one server computer, a data structure virtualizing at least one portion of a virtual or real world into a plurality of cells storing virtual objects forming a persistent virtual world system, wherein at least one of the virtual objects are virtual replicas of corresponding real world elements;

providing, in the memory of the at least one server computer, a distributed three-dimensional (3D) engine, the distributed 3D engine comprising a plurality of distributed software engines; and

dynamically allocating resources to the individual cells based on a current load of the plurality of cells,

wherein an original number of the plurality of cells representing one or more areas of the at least one portion of the virtual or real world are partitioned into a greater number of cells,

herein the resources are restored based on one or more requests, and

wherein each of the plurality of cells comprises at least one stream, the at least one stream comprising a plurality of stream-specific virtual objects and being configured to be enabled or disabled for viewing on and interacting with client devices, and wherein the at least one stream is associated with one or more user applications.

11. The method of claim 10 , wherein the at least one portion of the virtual or real world is consolidated back into the original number of cells.

12. The method of claim 10 , wherein the persistent virtual world system comprises a low fidelity simulation used for demand assessment and load balancing, and

wherein a high fidelity simulation used for improving user experience.

13. The method of claim 10 , wherein the distributed software engines are used sequentially or in parallel, through the distributed deployment, to complement engine services of each other for realization of at least one task.

14. The method of claim 10 , further comprising providing an octree data structure as the data structure representing the virtual or real world, and

wherein at least one of the plurality of cells is represented as a voxel, and wherein the voxel representing the at least one of the plurality of cells is selected from among a sparse voxel and a dense voxel based on size of the at least one of the plurality of cells.

15. The method of claim 10 , further comprising:

partitioning, based on a demand of the plurality of cells, the one or more areas of the at least one portion of the virtual or real world into additional cells; and

allocating corresponding resources to the additional cells.

16. The method of claim 10 , wherein the allocation of the resources is performed by:

publishing the resources to corresponding cells through a distributed message exchange platform of a resource manager using a publish-subscribe model; and

subscribing, by the one or more virtual objects, to at least one of the plurality of cells order to obtain required resources.

17. One or more non-transitory computer-readable media having stored thereon instructions configured to, when executed by one or more computers, cause the one or more computers to perform steps comprising:

providing, in memory of at least one server computer, a data structure virtualizing at least one portion of a virtual or real world into a plurality of cells storing virtual objects forming a persistent virtual world system, wherein at least one of the virtual objects are virtual replicas of corresponding real world elements;

providing, in the memory of the at least one server computer, a distributed three-dimensional (3D) engine, the distributed 3D engine comprising a plurality of distributed software engines; and

dynamically allocating resources to the individual cells based on a current load of the plurality of cells,

wherein an original number of the plurality of cells representing one or more areas of the at least one portion of the virtual or real world are partitioned into a greater number of cells, and

wherein the resources are restored based on one or more requests, and

wherein each of the plurality of cells comprises at least one stream, the at least one stream comprising a plurality of stream-specific virtual objects and being configured to be enabled or disabled for viewing on and interacting with client devices, and wherein the at least one stream is associated with one or more user applications.

Assignments (2)
CHANGE OF NAME Recorded Apr 18, 2025
From: TMRW FOUNDATION IP S.À R.L.
To: TMRW GROUP IP
Reel/Frame 070891/0012 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 20, 2023
From: YERLI, CEVAT; HIREMATADA, PRASHANTH
To: TMRW FOUNDATION IP SÀRL
Reel/Frame 065624/0068 →
Continuity (3)
Continuation 17138680 · Dec 30, 2020
Provisional Application 62955247 · Dec 30, 2019
Related Publication 20240029194A1 · Jan 25, 2024
References Cited (43)
US 7702784B2 · Berstis et al. · 2010 [cited by applicant]
US 8572207B2 · Shuster et al. · 2013 [cited by applicant]
US 8751668B2 · Kulkami et al. · 2014 [cited by applicant]
US 9498711B2 · Assa · 2016 [cited by applicant]
US 10003640B2 · Bhogal et al. · 2018 [cited by applicant]
US 20100113158A1 · Chapman et al. · 2010 [cited by applicant]
US 20100113159A1 · Chapman et al. · 2010 [cited by applicant]
US 20140176574A1 · Bakalash · 2014 [cited by applicant]
US 20150321101A1 · Rosedale et al. · 2015 [cited by applicant]
US 20180060138A1 · Whitehead et al. · 2018 [cited by applicant]
US 20190176037A1 · Leung et al. · 2019 [cited by applicant]
JP H0877365A · 1996 [cited by applicant]
JP H0877387A · 1996 [cited by applicant]
JP H1145348A · 1999 [cited by applicant]
JP H11154048A · 1999 [cited by applicant]
JP 2002269588A · 2002 [cited by applicant]
JP 2023271985A · 2003 [cited by applicant]
JP 2004348702A · 2004 [cited by applicant]
JP 2006031358A · 2006 [cited by applicant]
JP 2007156796A · 2007 [cited by applicant]
JP 2009187115A · 2009 [cited by applicant]
JP 2010131409A · 2010 [cited by applicant]
JP 2015087892A · 2015 [cited by applicant]
JP 2018032112A · 2018 [cited by applicant]
JP 2018136664A · 2018 [cited by applicant]
KR 1020100051014A · 2010 [cited by applicant]
WO 2007069724A1 · 2007 [cited by applicant]
WO 2019207104A1 · 2019 [cited by applicant]
Office Action mailed on Sep. 20, 2022, Japanese Patent Application No. 2020-214349 (Japanese version), 21 pages. [cited by applicant]
Office Action mailed on Oct. 1, 2021, Indian Patent Application No. 202014056483, filed on Dec. 25, 2020, 7 pages. [cited by applicant]
Office Action mailed on Mar. 8, 2022, Japanese Patent Application No. 2020214349 (Japanese version), 11 pages. [cited by applicant]
Ali, A.E.E., et al., “An allocation management algorithm for DVE System,” Computer Engineering & Systems International Conference, Piscataway, NJ, USA, Dec. 14, 2009, p. 489-494. [cited by applicant]
Bouras, C., et al., “Performance improvement of Distributed Virtual Environments by Exploiting objects attributes,” Virtual Reality vol. 16, No. 3, Aug. 23, 2011, p. 187-203. [cited by applicant]
Extended European Search Report mailed May 7, 2021 issued in corresponding EP 20217104, 11 pages. [cited by applicant]
Tanin, E., et al., “A Serverless 3D World,” Proceedings of the 12th Annual ACM International Workshop on Geographic Information Systems, Washington, D.C., Nov. 2004, 9 pages. [cited by applicant]
Fiedler, S., et al., A Communciation Architecture for Massive Multiplayer Games, Net Games '02: Proceedings of the 1st Workshop on Network and System Support for Games, Braunschweig, Germany, Apr. 2002, 9 pages. [cited by applicant]
Mullender, S.J., et al., “Amoeba A Distributed Operating System for the 1990s″ Computer: 23(5)”, pp. 44-53, May 1990. [cited by applicant]
Munro, J. et al., “Architecture Studies of Games Engines—The Quake Series, ”2009 International IEEE Consumer Electronics Society's Games Innovations Conference, London, UK, Aug. 25-28, 2009, pp. 1-10. [cited by applicant]
Kazem, I., et al., “A Visibility-Driven Approach to Managing Internet in Distributed Simulations with Dynamic Load Balancing”, 11th IEEE International Symposium on Distributed Simulation and Real-Time Applications (DS-R… [cited by applicant]
Hildebrand, D., “An Architectural Overview of QNX”, Proceedings of the Usenix Workshop on Micro-Kennels and Other Kennel Achitectures, Seattle, WA, Apr. 1992, p. 1-16. [cited by applicant]
Maggiorini, D., et al., “Smash: A Distributed Game Engine Architecture,” 2016 IEEE Symposium on Computers and Communication (ISCC), Messino, Italy, Jun. 27-30, 2016, 6 pages. [cited by applicant]
Search Report by Registered Search Organization dated Feb. 22, 2024. [cited by applicant]
Tanaka et al. “A Case Study on Linux Grid using Virtual Machine,” Information Processing Society of Japan Technical Report, vol. 2007 Nov. 26, 2007, pp. 91-96. [cited by applicant]