IP Library Granted Patent US 12,634,413
Granted Patent B2
US 12,634,413 · App. 17/813,708 · Granted May 19, 2026

Two-dimensional view of a presentation in a three-dimensional videoconferencing environment

Inventors: Gerard Cornelis Krol (Leiden, NL); Erik Stuart Braund (Saugerties, NY); James Donahower (Harvey Cedars, NJ); Petr Polyakov (Tampa, FL)
Assignee: KATMAI TECH INC.
H04N7/157G06F3/1454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,634,413
App. No.
17/813,708
Granted
May 19, 2026
Kind
B2
Abstract

Disclosed herein is a web-based videoconference system that allows for two-dimensional screen sharing within the virtual environment. In some embodiments, data specifying a three-dimensional virtual space. The three-dimensional virtual space comprises a plurality of participants and an avatar representing each of the plurality of participants. A presentation stream is received from a first client device of a first participant. The presentation stream is mapped onto a three-dimensional model of a presentation screen in the three-dimensional virtual space. A selection of the presentation screen is received from a second participant. A two-dimensional view of the presentation stream is rendered to the second participant.

Claims (34)

1 . A computer-implemented method for presenting in a virtual conference including a plurality of participants, comprising:

receiving data specifying a three-dimensional virtual space, wherein the three-dimensional virtual space comprises the plurality of participants and an avatar representing each of the plurality of participants;

receiving a presentation stream from a first client device of a first participant of the plurality of participants;

mapping the presentation stream onto a three-dimensional model of a presentation screen in the three-dimensional virtual space;

receiving a selection of the presentation screen from a second participant of the plurality of participants; and

rendering for display to the second participant a two-dimensional view of the presentation stream overlaid on a view of the three-dimensional virtual space rendered from a perspective of a virtual camera of the second participant as the perspective of the second participant is continuously updated.

2 . The computer-implemented method of claim 1 , further comprising rendering the presentation stream onto on the three-dimensional model of the presentation screen behind the two-dimensional view of the presentation stream in the three-dimensional virtual space.

3 . The computer-implemented method of claim 1 , wherein the selection is an input received by a second client device of the second participant.

4 . The computer-implemented method of claim 1 , wherein a communication server informs a second client device of the second participant of the presentation stream and the second client device requests the presentation stream from the communication server.

5 . The computer-implemented method of claim 1 , wherein the presentation stream comprises a first participant's view of the three-dimensional virtual space.

6 . The computer-implemented method of claim 1 , wherein the presentation screen comprises audio and video data captured by the first client device.

7 . The computer-implemented method of claim 1 , further comprising removing the presentation stream from the three-dimensional model of the presentation screen based on a position of the avatar of the second participant in the three-dimensional virtual space with respect to the three-dimensional model of the presentation screen or an input received by the first client device.

8 . A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations for presenting in a virtual conference including a plurality of participants, the operations comprising:

receiving data specifying a three-dimensional virtual space, wherein the three-dimensional virtual space comprises the plurality of participants and an avatar representing each of the plurality of participants;

receiving a presentation stream from a first client device of a first participant of the plurality of participants;

mapping the presentation stream onto a three-dimensional model of a presentation screen in the three-dimensional virtual space;

receiving a selection of the presentation screen from a second participant of the plurality of participants; and

rendering for display to the second participant a two-dimensional view of the presentation stream overlaid on a view of the three-dimensional virtual space rendered from a perspective of a virtual camera of the second participant as the perspective of the second participant is continuously updated.

9 . The device of claim 8 , wherein the operations further comprise rendering the presentation stream onto on the three-dimensional model of the presentation screen behind the two-dimensional view of the presentation stream in the three-dimensional virtual space.

10 . The device of claim 8 , wherein the selection is an input received by a second client device of the second participant.

11 . The device of claim 8 , wherein a communication server informs a second client device of the second participant of the presentation stream and the second client device requests the presentation stream from the communication server.

12 . The device of claim 8 , wherein the presentation stream comprises a first participant's view of the three-dimensional virtual space.

13 . The device of claim 8 , wherein the presentation screen comprises audio and video data captured by the first client device.

14 . The device of claim 8 , wherein the operations further comprise removing the presentation stream from the three-dimensional model of the presentation screen based on a position of the avatar of the second participant in the three-dimensional virtual space with respect to the three-dimensional model of the presentation screen or an input received by the first client device.

15 . A system for presenting in a virtual conference including a plurality of participants, comprising:

a processor coupled to a memory;

a network interface configured to (i) receive data specifying a three-dimensional virtual space, wherein the three-dimensional virtual space comprises the plurality of participants and an avatar representing each of the plurality of participants, (ii) receive a presentation stream from a first client device of a first participant of the plurality of participants, and (iii) receive a selection of a presentation screen from a second participant of the plurality of participants;

a texture mapper, implemented on the processor, configured to texture map the presentation stream onto a three-dimensional model of the presentation screen; and

a renderer, implemented on the processor, configured to render for display to a second participant of the plurality of participants, the three-dimensional virtual space including a two-dimensional view of the presentation stream overlaid on a view of the three-dimensional virtual space rendered from a perspective of a virtual camera of the second participant as the perspective of the second participant is continuously updated.

16 . The system of claim 15 , wherein the renderer is further configured to render the presentation stream onto on the three-dimensional model of the presentation screen behind the two-dimensional view of the presentation stream in the three-dimensional virtual space.

17 . The system of claim 15 , wherein the selection is an input received by a second client device of the second participant.

18 . The system of claim 15 , wherein a communication server informs a second client device of the second participant of the presentation stream and the second client device requests the presentation stream from the communication server.

19 . The system of claim 15 , wherein the presentation stream comprises a first participant's view of the three-dimensional virtual space.

20 . The system of claim 15 , wherein the presentation screen comprises audio and video data captured by the first client device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 1, 2022
From: KROL, GERARD CORNELIS; BRAUND, ERIK STUART; DONAHOWER, JAMES; POLYAKOV, PETR
To: KATMAI TECH INC.
Reel/Frame 060685/0081 →
Continuity (1)
Related Publication 20240031531A1 · Jan 25, 2024
References Cited (116)
US 6215498B1 · Filo et al. · 2001 [cited by applicant]
US 6559863B1 · Megiddo · 2003 [cited by applicant]
US 6583808B2 · Boulanger et al. · 2003 [cited by applicant]
US 6853398B2 · Malzbender et al. · 2005 [cited by applicant]
US 7197126B2 · Kanada · 2007 [cited by applicant]
US 7346654B1 · Weiss · 2008 [cited by applicant]
US 7634073B2 · Kanada · 2009 [cited by applicant]
US 7840668B1 · Sylvain et al. · 2010 [cited by applicant]
US 8072479B2 · Valliath et al. · 2011 [cited by applicant]
US 8279254B2 · Goose et al. · 2012 [cited by applicant]
US 8403751B2 · Boustead et al. · 2013 [cited by applicant]
US 8520872B2 · Jang et al. · 2013 [cited by applicant]
US 8620009B2 · Zhang et al. · 2013 [cited by applicant]
US 9041764B2 · Wang et al. · 2015 [cited by applicant]
US 9218685B2 · Piemonte et al. · 2015 [cited by applicant]
US 9305319B2 · Maor et al. · 2016 [cited by applicant]
US 9384594B2 · Maciocci et al. · 2016 [cited by applicant]
US 9420229B2 · Pourashraf et al. · 2016 [cited by applicant]
US 9565316B2 · Gleim · 2017 [cited by applicant]
US 9607428B2 · Li · 2017 [cited by applicant]
US 9656168B1 · Bear et al. · 2017 [cited by applicant]
US 9661274B2 · Kubota et al. · 2017 [cited by applicant]
US 9743044B2 · Safaei et al. · 2017 [cited by applicant]
US 9910509B2 · Markovic et al. · 2018 [cited by applicant]
US 10013805B2 · Barzuza et al. · 2018 [cited by applicant]
US 10155164B2 · Boustead et al. · 2018 [cited by applicant]
US 10304238B2 · Cooper et al. · 2019 [cited by applicant]
US 10304239B2 · Gorur Sheshagiri et al. · 2019 [cited by applicant]
US 10334384B2 · Sun et al. · 2019 [cited by applicant]
US 10356216B2 · Khalid et al. · 2019 [cited by applicant]
US 10491946B2 · Tillman et al. · 2019 [cited by applicant]
US 10573071B2 · Sun et al. · 2020 [cited by applicant]
US 10609334B2 · Li · 2020 [cited by applicant]
US 10679411B2 · Ziman · 2020 [cited by applicant]
US 10979672B1 · Krol et al. · 2021 [cited by applicant]
US 11057351B1 · Simanel et al. · 2021 [cited by applicant]
US 11095857B1 · Krol · 2021 [cited by examiner]
US 11457178B2 · Krol · 2022 [cited by examiner]
US 20020158873A1 · Williamson · 2002 [cited by applicant]
US 20070274528A1 · Nakamoto et al. · 2007 [cited by applicant]
US 20090113314A1 · Dawson et al. · 2009 [cited by applicant]
US 20090172557A1 · Muta · 2009 [cited by examiner]
US 20090254843A1 · Van Wie · 2009 [cited by examiner]
US 20110072367A1 · Bauer · 2011 [cited by applicant]
US 20130321564A1 · Smith et al. · 2013 [cited by applicant]
US 20140033078A1 · Goldman · 2014 [cited by examiner]
US 20140040783A1 · Goldman et al. · 2014 [cited by applicant]
US 20140058807A1 · Altberg et al. · 2014 [cited by applicant]
US 20150302661A1 · Miller · 2015 [cited by applicant]
US 20160300387A1 · Ziman · 2016 [cited by examiner]
US 20170351476A1 · Yoakum · 2017 [cited by applicant]
US 20190199993A1 · Babu J D et al. · 2019 [cited by applicant]
US 20190310761A1 · Agarawala et al. · 2019 [cited by applicant]
US 20190320144A1 · Tong et al. · 2019 [cited by applicant]
US 20190354170A1 · Rosedale · 2019 [cited by applicant]
US 20190371060A1 · Energin et al. · 2019 [cited by applicant]
US 20200008003A1 · Thompson et al. · 2020 [cited by applicant]
US 20200037091A1 · Jeon et al. · 2020 [cited by applicant]
US 20200098191A1 · McCall · 2020 [cited by applicant]
US 20200221247A1 · Latypov et al. · 2020 [cited by applicant]
US 20200294312A1 · Ziman · 2020 [cited by applicant]
US 20210248803A1 · Kojima et al. · 2021 [cited by applicant]
US 20220070232A1 · Young · 2022 [cited by examiner]
US 20220086203A1 · Morris · 2022 [cited by examiner]
US 20220124130A1 · Punwani · 2022 [cited by examiner]
US 20220124140A1 · Okina · 2022 [cited by examiner]
US 20220277528A1 · Funazukuri · 2022 [cited by examiner]
US 20230128659A1 · Krol · 2023 [cited by examiner]
CN 103580881A · 2014 [cited by applicant]
CN 105487657A · 2016 [cited by applicant]
CN 103888714B · 2017 [cited by applicant]
CN 106648528A · 2017 [cited by applicant]
CN 108319439A · 2018 [cited by applicant]
CN 106528038B · 2019 [cited by applicant]
EP 3282669B1 · 2020 [cited by applicant]
EP 3627860A1 · 2020 [cited by applicant]
EP 3684083A1 · 2020 [cited by applicant]
GB 2351216B · 2002 [cited by applicant]
JP H07288791A · 1995 [cited by applicant]
JP 2003006132A · 2003 [cited by applicant]
JP 4426484B2 · 2010 [cited by applicant]
KR 20200028871A · 2020 [cited by applicant]
WO WO2008125593A2 · 2008 [cited by applicant]
WO WO2008141596A1 · 2008 [cited by applicant]
WO WO2010083119A2 · 2010 [cited by applicant]
WO WO2010143359A1 · 2010 [cited by applicant]
WO WO2018005235A1 · 2018 [cited by applicant]
WO WO2019008320A1 · 2019 [cited by applicant]
WO WO2019046597A1 · 2019 [cited by applicant]
WO WO2020041652A1 · 2020 [cited by applicant]
WO 2022087147A1 · 2022 [cited by applicant]
Roach, “Mesh for Microsoft Teams aims to make collaboration in the ‘metaverse’ personal and fun,” Nov. 2, 2021, https://news.microsoft.com/source/features/innovation/mesh-for-microsoft-teams/. [cited by examiner]
Purdy, “How the Metaverse Could Change Work,” Apr. 5, 2022, https://hbr.org/2022/04/how-the-metaverse-could-change-work. [cited by examiner]
Dipaola, S. and Collins, D., “A 3D Virtual Environment for Social Telepresence”, Mar. 2002; 6 pages. [cited by applicant]
EngageVR, “Next Generation Education, Training, Meetings & Events,” accessed at https://engagevr.io/, accessed on Oct. 28, 2020; 6 pages. [cited by applicant]
Event Farm, “Bring Your Events Into the Virtual World,” accessed at https://eventfarm.com/the-echo, accessed on Oct. 28, 2020; 6 pages. [cited by applicant]
Hongtongsak, K., “Accurately Pinpointing 3D Sound in Virtual Environments,” Jun. 2016, 31 pages. [cited by applicant]
Johansson, N., “The 10 Best VR Meeting Apps—Productive Remote Collaboration,” accessed at https://immersive.ly/best-vr-apps-productive-remote-meetings/, accessed on Oct. 28, 2020; 10 pages. [cited by applicant]
Leung, W. and Chen, T., “Networked Collaborative Environment With Animated 3D Avatars, Dept. of Electrical and Computer Engineering Carnegie Mellon University,” Aug. 6, 2002; 3 pages. [cited by applicant]
Leung, W. et al.., “Networked Intelligent Collaborative Environment (NetICE), Multimedia and Expo 2000,” Aug. 6, 2002; 4 pages. [cited by applicant]
Leung, W. and Chen, T., “A Multi-User 3-D Virtual Environment With Interactive Collaboration and Shared Whiteboard Technologies, Kluwer Academic Publishers,” Jul. 23, 2003, 17 pages. [cited by applicant]
Lin, J. et al., “A Virtual Reality Platform for Dynamic Human-Scene Interaction,” Dec. 2016; 4 pages. [cited by applicant]
Nishihara, R. and Okubo, M., “A Study on Personal Space in Virtual Space Based on Personality,” 2015; 8 pages. [cited by applicant]
Sandre, A., “Is Virtual Reality the Future of Video Conferencing?” May 15, 2020; 7 pages. [cited by applicant]
De Sousa, A., “Remote Proxemics for Collaborative Virtual Environments,” Nov. 2014; 10 pages. [cited by applicant]
Takahashi, D., “MeetinVR launches online VR meetings with ‘superpowers’,” venturebeat.com, accessed at https://venturebeat.com/2020/05/27/meetinvr-launches-online-vr-meetings-with-superpowers, published May 27, 2020; 7 … [cited by applicant]
Teooh, “Virtual Spaces for Real Communities,” accessed at https://teooh.com/, accessed on Oct. 28, 2020; 3 pages. [cited by applicant]
VirBELA Media, “Team Suites,” accessed at https://www.virbela.com/teamsuites, accessed on Oct. 1, 2020; 26 pages. [cited by applicant]
Virtway Events, “Your 3D Space for Events,” accessed at https://www.virtwayevents.com/, accessed on Oct. 28, 2020; 5 pages. [cited by applicant]
SaganWorks product website, “A personal metaverse of knowledge: Create stunning spaces to organize and store your information,” available at https://saganworks.com, accessed Aug. 4, 2022; 4 pages. [cited by applicant]
Spatial, “Frequently Asked Questions,” product website available at https://support.spatial.io/hc/en-us, accessed Aug. 4, 2022; 3 pages. [cited by applicant]
VSpatial product website, Virtual Reality Offices, “The workspace of the Future is Here.” available at https://vspatial.com, accessed Aug. 4, 2022; 5 pages. [cited by applicant]
LearnOpenGL—CSM Online Resource, “Cascaded Shadow Mapping,” available at https://learnopengl.com/Guest-Articles/2021/CSM, accessed Aug. 2, 2022; 10 pages. [cited by applicant]
Wikipedia, online resource, “Shadow mapping,” available at https://en.wikipedia.org/wiki/Shadow_mapping, accessed Aug. 2, 2022; 6 pages. [cited by applicant]
Wikipedia, online resource, “Volumetric lighting,” available at https://en.wikipedia.org/wiki/Volumetric_lighting, accessed Aug. 2, 2022; 2 pages. [cited by applicant]
Search Report and Written Opinion issued in PCT/US2023/070509, Nov. 3, 2023, 11 pages. [cited by applicant]