IP Library Granted Patent US 12,368,821
Granted Patent B2
US 12,368,821 · App. 17/875,581 · Granted Jul 22, 2025

Optimizing physics for static objects in a three-dimensional virtual environment

Inventors: Gerard Cornelis Krol (Leiden, NL); Petr Polyakov (Tampa, FL)
Assignee: Katmai Tech Inc.
H04N7/157G06F3/04815H04L12/1827H04N21/44012
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,368,821
App. No.
17/875,581
Granted
Jul 22, 2025
Kind
B2
Abstract

Disclosed herein is a web-based videoconference system that allows for video avatars to navigate within a virtual environment. Various methods for efficient modeling, rendering, and shading are disclosed herein.

Claims (62)

1. A computer-implemented method for optimizing physics simulation in a three-dimensional (“3D”) virtual environment including a plurality of objects displayed in a scene of a video conferencing application to a user, comprising:

evaluating each object in the plurality of objects to determine that a particular object's position, rotation, or scale has changed, wherein each object in the plurality of objects is represented by a node in a tree hierarchy, wherein the plurality of objects in the scene are specified in a virtual reality framework using a hypertext markup language, and wherein the plurality of objects comprise an avatar controlled by a second user of the video conferencing application;

in response to the determining, transforming the particular object with respect to at least one of position, rotation, or scale;

in response to the transformation, determining how the plurality of objects interact with one another, comprising:

for each object in the plurality of objects, determining whether the respective object is fixed or dynamic by referencing the tree hierarchy, wherein a fixed object cannot move, transform, or rotate within the 3D virtual environment;

for each pair of objects in the plurality of objects:

determining whether both objects in the respective pair are fixed; and

when both objects in the respective pair are determined to be fixed, disabling a simulation of physical interaction between the two objects; and

rendering the scene using a rendering engine comprising a cross-browser Javascript library that creates a graphical processing unit (GPU)-accelerated 3D animation in a web browser based on the scene to represent a virtual camera of the user in the video conferencing application, wherein a fixed object is rendered only once for each position of the virtual camera.

2. The method of claim 1 , further comprising:

in response to determining that at least one object in the respective pair is dynamic, conducting the simulation of physical interaction between the two objects.

3. The method of claim 2 , wherein the conducting the simulation of physical interaction comprises:

determining whether a collision occurs between the objects in the respective pair; and

in response to determining that the collusion occurs, preventing the objects in the respective pair from penetrating one another.

4. The method of claim 2 , wherein the plurality of objects comprise fixed objects representing an arena.

5. The method of claim 1 , wherein the avatar represents a virtual camera controlled by the second user.

6. The method of claim 5 , further comprising:

receiving a video stream captured of the second user; and

mapping the video stream onto the avatar.

7. The method of claim 1 , further comprising:

in response to determining that none of the plurality of objects are dynamic, skipping physics simulation for the plurality of objects.

8. The method of claim 1 , wherein the avatar is represented in the tree hierarchy and has a back child node and a video child node in the tree hierarchy.

9. A computer-readable non-transitory storage medium comprising instructions which, when executed by a computer, cause the computer to carry out operations for optimizing physics simulation of a three-dimensional (“3D”) virtual environment, the operations comprising:

evaluating each object in a plurality of objects displayed in a scene of a video conferencing application to a user to determine that a particular object's position, rotation, or scale has changed, wherein each object in the plurality of objects is represented by a node in a tree hierarchy, wherein the plurality of objects in the scene are specified in a virtual reality framework using a hypertext markup language, wherein the plurality of objects comprise an avatar controlled by a second user of the video conferencing application;

in response to the determining, transforming the particular object with respect to at least one of position, rotation, or scale;

in response to the transformation, determining how the plurality of objects interact with one another, comprising:

for each object in the plurality of objects, determining whether the respective object is fixed or dynamic by referencing the tree hierarchy, wherein a fixed object cannot move, transform, or rotate within the 3D virtual environment;

for each pair of objects in the plurality of objects:

determining whether both objects in the respective pair are fixed; and

when both objects in the respective pair are determined to be fixed, disabling a simulation of physical interaction between the two objects; and

rendering the scene using a rendering engine comprising a cross-browser Javascript library that creates a graphical processing unit (GPU)-accelerated 3D animation in a web browser based on the scene to represent a virtual camera of the user in the video conferencing application, wherein a fixed object is rendered only once for each position of the virtual camera.

10. The computer-readable non-transitory storage medium of claim 9 , the operations further comprising:

in response to determining that at least one object in the respective pair is dynamic, conducting the simulation of physical interaction between the two objects.

11. The computer-readable non-transitory storage medium of claim 10 , wherein the conducting the simulation comprises:

determining whether a collision occurs between the objects in the respective pair; and

in response to determining that the collision occurs, preventing the objects in the respective pair from penetrating one another.

12. The computer-readable non-transitory storage medium of claim 10 , wherein the plurality of objects comprise fixed objects representing an arena.

13. The computer-readable non-transitory storage medium of claim 9 , wherein the avatar represents a virtual camera controlled by the second user.

14. The computer-readable non-transitory storage medium of claim 13 , the operations further comprising:

receiving a video stream captured of the second user; and

mapping the video stream onto the avatar.

15. The computer-readable non-transitory storage medium of claim 9 , wherein the avatar is represented in the tree hierarchy and has a back child node and a video child node in the tree hierarchy.

16. A system for optimizing physics simulation in a three-dimensional (“3D”) virtual environment including a plurality of objects displayed in a scene of a video conferencing application to a user, comprising:

a memory;

at least one processor coupled to the memory and configured to:

evaluate each object in the plurality of objects to determine that a particular object's position, rotation, or scale has changed, wherein each object in the plurality of objects is represented by a node in a tree hierarchy, wherein the plurality of objects in the scene are specified in a virtual reality framework using a hypertext markup language, wherein the plurality of objects comprise an avatar controlled by a second user of the video conferencing application;

in response to the determining, transform the particular object with respect to at least one of position, rotation, or scale;

in response to the transformation, determine how the plurality of objects interact with one another, comprising:

(i) for each object in the plurality of objects, determine whether the respective object is fixed or dynamic by referencing the tree hierarchy, wherein a fixed object cannot move, transform, or rotate within the 3D virtual environment,

(ii) for each pair of objects in the plurality of objects, determine whether both objects in the respective pair are fixed, and

(iii) when both objects in the respective pair are determined to be fixed, disable a simulation of physical interaction between the two objects; and

render the scene using a rendering engine comprising a cross-browser Javascript library that creates a graphical processing unit (GPU)-accelerated 3D animation in a web browser based on the scene to represent a virtual camera of the user in the video conferencing application, wherein a fixed object is rendered only once for each position of the virtual camera.

17. The system of claim 16 , the at least one processor further configured to:

in response to determining that at least one object in the respective pair is dynamic, conduct the simulation of physical interaction between the two objects.

18. The system of claim 16 , the at least one processor further configured to:

determine whether a collision occurs between the objects in the respective pair; and

in response to determining that the collusion occurs, prevent the objects in the respective pair from penetrating one another.

19. The system of claim 16 , wherein the avatar represents a virtual camera controlled by the second user.

20. The system of claim 19 , the at least one processor further configured to:

receive a video stream captured of the second user; and

map the video stream onto the avatar.

21. The system of claim 16 , wherein the avatar is represented in the tree hierarchy and has a back child node and a video child node in the tree hierarchy.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 6, 2022
From: KROL, GERARD CORNELIS; POLYAKOV, PETR
To: KATMAI TECH INC.
Reel/Frame 061337/0427 →
Continuity (1)
Related Publication 20240040085A1 · Feb 1, 2024
References Cited (121)
US 6215498B1 · Filo et al. · 2001 [cited by applicant]
US 6348927B1 · Lipkin · 2002 [cited by examiner]
US 6559863B1 · Megiddo · 2003 [cited by applicant]
US 6583808B2 · Boulanger et al. · 2003 [cited by applicant]
US 6853398B2 · Malzbender et al. · 2005 [cited by applicant]
US 7197126B2 · Kanada · 2007 [cited by applicant]
US 7346654B1 · Weiss · 2008 [cited by applicant]
US 7634073B2 · Kanada · 2009 [cited by applicant]
US 7840668B1 · Sylvain et al. · 2010 [cited by applicant]
US 8072479B2 · Valliath et al. · 2011 [cited by applicant]
US 8279254B2 · Goose et al. · 2012 [cited by applicant]
US 8403751B2 · Boustead et al. · 2013 [cited by applicant]
US 8520872B2 · Jang et al. · 2013 [cited by applicant]
US 8620009B2 · Zhang et al. · 2013 [cited by applicant]
US 9041764B2 · Wang et al. · 2015 [cited by applicant]
US 9218685B2 · Piemonte et al. · 2015 [cited by applicant]
US 9305319B2 · Maor et al. · 2016 [cited by applicant]
US 9384594B2 · Maciocci et al. · 2016 [cited by applicant]
US 9420229B2 · Pourashraf et al. · 2016 [cited by applicant]
US 9565316B2 · Gleim · 2017 [cited by applicant]
US 9607428B2 · Li · 2017 [cited by applicant]
US 9656168B1 · Bear et al. · 2017 [cited by applicant]
US 9661274B2 · Kubota et al. · 2017 [cited by applicant]
US 9743044B2 · Safaei et al. · 2017 [cited by applicant]
US 9910509B2 · Markovic et al. · 2018 [cited by applicant]
US 10013805B2 · Barzuza et al. · 2018 [cited by applicant]
US 10101585B2 · Ueno · 2018 [cited by examiner]
US 10155164B2 · Boustead et al. · 2018 [cited by applicant]
US 10304238B2 · Cooper et al. · 2019 [cited by applicant]
US 10304239B2 · Gorur Sheshagiri et al. · 2019 [cited by applicant]
US 10334384B2 · Sun et al. · 2019 [cited by applicant]
US 10356216B2 · Khalid et al. · 2019 [cited by applicant]
US 10573071B2 · Sun et al. · 2020 [cited by applicant]
US 10609334B2 · Li · 2020 [cited by applicant]
US 10679411B2 · Ziman · 2020 [cited by applicant]
US 10979672B1 · Krol · 2021 [cited by examiner]
US 11057351B1 · Simanel et al. · 2021 [cited by applicant]
US 11711494B1 · Krol · 2023 [cited by examiner]
US 11741674B1 · Swanson · 2023 [cited by examiner]
US 11924442B2 · Hindriks · 2024 [cited by examiner]
US 11956571B2 · Krol · 2024 [cited by examiner]
US 20020158873A1 · Williamson · 2002 [cited by applicant]
US 20050062738A1 · Handley · 2005 [cited by examiner]
US 20070274528A1 · Nakamoto et al. · 2007 [cited by applicant]
US 20090033737A1 · Goose · 2009 [cited by examiner]
US 20090113314A1 · Dawson et al. · 2009 [cited by applicant]
US 20110072367A1 · Bauer · 2011 [cited by applicant]
US 20110077923A1 · Bond · 2011 [cited by examiner]
US 20110199377A1 · Jang · 2011 [cited by examiner]
US 20130321564A1 · Smith et al. · 2013 [cited by applicant]
US 20140058807A1 · Altberg et al. · 2014 [cited by applicant]
US 20140108915A1 · Lu · 2014 [cited by examiner]
US 20140267230A1 · Osuna · 2014 [cited by examiner]
US 20140267564A1 · Pourashraf · 2014 [cited by examiner]
US 20150302661A1 · Miller · 2015 [cited by applicant]
US 20160227172A1 · Safaei · 2016 [cited by examiner]
US 20160307353A1 · Ligenza · 2016 [cited by examiner]
US 20170351476A1 · Yoakum · 2017 [cited by applicant]
US 20180095636A1 · Valdivia · 2018 [cited by examiner]
US 20180253884A1 · Burnett, III · 2018 [cited by examiner]
US 20190199993A1 · Babu J D et al. · 2019 [cited by applicant]
US 20190310761A1 · Agarawala et al. · 2019 [cited by applicant]
US 20190320144A1 · Tong et al. · 2019 [cited by applicant]
US 20190354170A1 · Rosedale · 2019 [cited by applicant]
US 20190371060A1 · Energin et al. · 2019 [cited by applicant]
US 20200008003A1 · Thompson et al. · 2020 [cited by applicant]
US 20200037091A1 · Jeon et al. · 2020 [cited by applicant]
US 20200098191A1 · McCall · 2020 [cited by applicant]
US 20200221247A1 · Latypov et al. · 2020 [cited by applicant]
US 20200302681A1 · Totty · 2020 [cited by examiner]
US 20210365277A1 · Sproch · 2021 [cited by examiner]
US 20220086203A1 · Morris · 2022 [cited by examiner]
US 20230141027A1 · Cerrato · 2023 [cited by examiner]
US 20230149811A1 · Myers · 2023 [cited by examiner]
US 20240040086A1 · Krol · 2024 [cited by examiner]
CN 103580881A · 2014 [cited by applicant]
CN 105487657A · 2016 [cited by applicant]
CN 103888714B · 2017 [cited by applicant]
CN 106648528A · 2017 [cited by applicant]
CN 108319439A · 2018 [cited by applicant]
CN 106528038B · 2019 [cited by applicant]
EP 3282669B1 · 2020 [cited by applicant]
EP 3627860A1 · 2020 [cited by applicant]
EP 3684083A1 · 2020 [cited by applicant]
GB 2351216B · 2002 [cited by applicant]
JP H07288791A · 1995 [cited by applicant]
JP 2003006132A · 2003 [cited by applicant]
JP 4426484B2 · 2010 [cited by applicant]
KR 20200028871A · 2020 [cited by applicant]
WO WO2008125593A2 · 2008 [cited by applicant]
WO WO2008141596A1 · 2008 [cited by applicant]
WO WO2010083119A2 · 2010 [cited by applicant]
WO WO2010143359A1 · 2010 [cited by applicant]
WO WO2018005235A1 · 2018 [cited by applicant]
WO WO2019008320A1 · 2019 [cited by applicant]
WO WO2019046597A1 · 2019 [cited by applicant]
WO WO2020041652A1 · 2020 [cited by applicant]
Merriam-Webster. (n.d.). Fixed. In Merriam-Webster.com dictionary. Retrieved Jan. 18, 2023 from https://www.merriam-webster.com/dictionary/fixed (Year: 2023). [cited by examiner]
Merriam-Webster. (n.d.). Static. In Merriam-Webster.com dictionary. Retrieved Jan. 18, 2023 from https://www.merriam-webster.com/dictionary/static (Year: 2023). [cited by examiner]
Dipaola, S. and Collins, D., “A 3D Virtual Environment for Social Telepresence”, Mar. 2002; 6 pages. [cited by applicant]
EngageVR, “Next Generation Education, Training, Meetings & Events,” accessed at https://engagevr.io/, accessed on Oct. 28, 2020; 6 pages. [cited by applicant]
Event Farm, “Bring Your Events Into the Virtual World,” accessed at https://eventfarm.com/the-echo, accessed on Oct. 28, 2020; 6 pages. [cited by applicant]
Hongtongsak, K., “Accurately Pinpointing 3D Sound in Virtual Environments,” Jun. 2016, 31 pages. [cited by applicant]
Johansson, N., “The 10 Best VR Meeting Apps—Productive Remote Collaboration,” accessed at https://immersive.ly/best-vr-apps-productive-remote-meetings/, accessed on Oct. 28, 2020; 10 pages. [cited by applicant]
Leung, W. and Chen, T., “Networked Collaborative Environment With Animated 3D Avatars, Dept. of Electrical and Computer Engineering Carnegie Mellon University,” Aug. 6, 2002; 3 pages. [cited by applicant]
Leung, W et al.. , “Networked Intelligent Collaborative Environment (NetICE), Multimedia and Expo 2000,” Aug. 6, 2002; 4 pages. [cited by applicant]
Leung, W. and Chen, T., “A Multi-User 3-D Virtual Environment With Interactive Collaboration and Shared Whiteboard Technologies, Kluwer Academic Publishers,” Jul. 23, 2003, 17 pages. [cited by applicant]
Lin, J. et al., “A Virtual Reality Platform for Dynamic Human-Scene Interaction,” Dec. 2016; 4 pages. [cited by applicant]
Nishihara, R. and Okubo, M., “A Study on Personal Space in Virtual Space Based on Personality,” 2015; 8 pages. [cited by applicant]
Sandre, A., “Is Virtual Reality the Future of Video Conferencing?” May 15, 2020; 7 pages. [cited by applicant]
De Sousa, A., “Remote Proxemics for Collaborative Virtual Environments,” Nov. 2014; 10 pages. [cited by applicant]
Takahashi, D., “MeetinVR launches online VR meetings with ‘superpowers’,” venturebeat.com, accessed at https://venturebeat.com/2020/05/27/meetinvr-launches-online-vr-meetings-with-superpowers, published May 27, 2020; 7 … [cited by applicant]
Teooh, “Virtual Spaces for Real Communities,” accessed at https://teooh.com/, accessed on Oct. 28, 2020; 3 pages. [cited by applicant]
VirBELA Media, “Team Suites,” accessed at https://www.virbela.com/teamsuites, accessed on Oct. 1, 2020; 26 pages. [cited by applicant]
Virtway Events, “Your 3D Space for Events,” accessed at https://www.virtwayevents.com/, accessed on Oct. 28, 2020; 5 pages. [cited by applicant]
SaganWorks product website, “A personal metaverse of knowledge: Create stunning spaces to organize and store your information,” available at https://saganworks.com, accessed Aug. 4, 2022; 4 pages. [cited by applicant]
Spatial, “Frequently Asked Questions,” product website available at https://support.spatial.io/hc/en-us, accessed Aug. 4, 2022; 3 pages. [cited by applicant]
VSpatial product website, Virtual Reality Offices, “The workspace of the Future is Here.” available at https://vspatial.com, accessed Aug. 4, 2022; 5 pages. [cited by applicant]
LearnOpenGL—CSM Online Resource, “Cascaded Shadow Mapping,” available at https://learnopengl.com/Guest-Articles/2021/CSM, accessed Aug. 2, 2022; 10 pages. [cited by applicant]
Wikipedia, online resource, “Shadow mapping,” available at https://en.wikipedia.org/wiki/Shadow_mapping, accessed Aug. 2, 2022; 6 pages. [cited by applicant]
Wikipedia, online resource, “Volumetric lighting,” available at https://en.wikipedia.org/wiki/Volumetric_lighting, accessed Aug. 2, 2022; 2 pages. [cited by applicant]