IP Library Granted Patent US 12,593,008
Granted Patent B2
US 12,593,008 · App. 18/456,994 · Granted Mar 31, 2026

User interface tile arrangement based on relative locations

Inventors: Anthony Bao Quoc Hoang (San Francisco, CA); Cynthia Eshiuan Lee (Austin, TX); Jeffrey William Smith (Layton, UT); Chi-chian Yu (San Ramon, CA)
Assignee: Zoom Communications, Inc.
H04N7/15G06F3/04845H04L12/1822H04L65/1083H04L65/403
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,593,008
App. No.
18/456,994
Granted
Mar 31, 2026
Kind
B2
Abstract

User interface (UI) tiles of conference participants are arranged in a UI of conferencing software according to relative locations of those conference participants within a conference room. Positional information and video data are obtained from one or more video capture devices located within a conference room. Relative locations of conference participants within the conference room are determined based on the positional information, such as by defining coordinates for the conference participants within a coordinate system based on the positional information and the video data and determining the relative locations based on the coordinates. Output configured to cause a client application to arrange UI tiles associated with the conference participants is generated according to the relative locations. The output is then transmitted to one or more client devices to cause a UI of conferencing software at each of those client devices to display the UI tiles in the specified arrangement.

Claims (50)

1 . A method, comprising:

obtaining positional information from multiple cameras within a room;

determining, using a common coordinate system associated with the multiple cameras, relative locations of conference participants within the room based on the positional information and overlapping fields of view of the multiple cameras, wherein a first subset of the conference participants are within a first field of view of a first camera of the multiple cameras, a second subset of the conference participants are within a second field of view of a second camera of the multiple cameras, and the overlapping fields of view include the first field of view and the second field of view; and

causing a client application to arrange user interface tiles associated with the conference participants according to the relative locations.

2 . The method of claim 1 , wherein determining the relative locations of the conference participants within the room comprises:

defining coordinates for each of the conference participants within the common coordinate system based on the positional information and video data obtained from the multiple cameras; and

determining the relative locations based on the coordinates.

3 . The method of claim 1 , further comprising:

causing the client application to group the user interface tiles together and specifying, to the client application, an order in which to present the user interface tiles within a user interface of conferencing software.

4 . The method of claim 1 , wherein a participant list presented within conferencing software identifies the room as a participant and each of the conference participants as sub-participants of the room.

5 . The method of claim 1 , wherein information represented for a conference participant within a participant list indicates whether a companion device associated with the conference participant is available for messaging.

6 . The method of claim 1 , further comprising:

determining at least two regions of interest within the room based on a video stream obtained from one of the multiple cameras, wherein each of the at least two regions of interest corresponds to one or more of the conference participants; and

producing an output video stream for rendering within the user interface tiles for each of the at least two regions of interest.

7 . The method of claim 1 , further comprising:

causing the client application to rearrange the user interface tiles based on a change in one or both of a number of the conference participants or locations of one or more of the conference participants within the room.

8 . The method of claim 7 , wherein the change in the one or both of the number of the conference participants or the locations of the one or more of the conference participants within the room is detected in response to a triggering event.

9 . The method of claim 1 , wherein the positional information obtained from a camera of the multiple cameras indicates whether the camera is a leftmost camera, a rightmost camera, or a central camera.

10 . An apparatus, comprising:

a memory; and

a processor configured to execute instructions stored in the memory to:

obtain positional information from multiple cameras within a room;

determine, using a common coordinate system associated with the multiple cameras, relative locations of conference participants within the room based on the positional information and overlapping fields of view of the multiple cameras, wherein a first subset of the conference participants are within a first field of view of a first camera of the multiple cameras, a second subset of the conference participants are within a second field of view of a second camera of the multiple cameras, and the overlapping fields of view include the first field of view and the second field of view; and

cause a client application to arrange user interface tiles associated with the conference participants according to the relative locations.

11 . The apparatus of claim 10 , wherein, to determine the relative locations of the conference participants within the room, the processor is configured to execute the instructions to:

define coordinates for each of the conference participants within the common coordinate system based on the positional information and video data obtained from the multiple cameras; and

determine the relative locations based on the coordinates.

12 . The apparatus of claim 10 , wherein the processor is configured to execute the instructions to:

cause the client application to group the user interface tiles together; and

specify, to the client application, an order in which to present the user interface tiles within a user interface of conferencing software.

13 . The apparatus of claim 10 , wherein names of the conference participants presented in the user interface of conferencing software are determined by processing one or both of video data obtained from the multiple cameras and audio data obtained from one or more audio capture devices within the room.

14 . The apparatus of claim 10 , wherein names of the conference participants are presented within a participant list of conferencing software.

15 . The apparatus of claim 10 , wherein the processor is configured to execute the instructions to:

determine at least two regions of interest within the room based on a video stream obtained from one of the multiple cameras, wherein each of the at least two regions of interest corresponds to one or more of the conference participants; and

produce an output video stream for rendering within the user interface tiles for each of the at least two regions of interest.

16 . The apparatus of claim 10 , wherein the processor is configured to execute the instructions to:

cause the client application to rearrange the user interface tiles based on a change in one or both of a number of the conference participants or locations of one or more of the conference participants within the room.

17 . A non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:

obtaining positional information from multiple cameras within a room;

determining, using a common coordinate system associated with the multiple cameras, relative locations of conference participants within the room based on the positional information and overlapping fields of view of the multiple cameras, wherein a first subset of the conference participants are within a first field of view of a first camera of the multiple cameras, a second subset of the conference participants are within a second field of view of a second camera of the multiple cameras, and the overlapping fields of view include the first field of view and the second field of view; and

causing a client application to arrange user interface tiles associated with the conference participants according to the relative locations.

18 . The non-transitory computer readable medium of claim 17 , wherein the operations further comprise:

causing the client application to group the user interface tiles together; and

specifying, to the client application, an order in which to present the user interface tiles within a user interface of conferencing software.

19 . The non-transitory computer readable medium of claim 17 , wherein the operations for determining the relative locations of the conference participants based on the positional information comprise:

defining coordinates for each of the conference participants within the common coordinate system based on the positional information and video data obtained from the multiple cameras; and

determining the relative locations based on the coordinates.

20 . The non-transitory computer readable medium of claim 17 , wherein the operations further comprise:

determining at least two regions of interest within the room based on a video stream obtained from one of the multiple cameras, wherein each of the at least two regions of interest corresponds to one or more of the conference participants; and

producing an output video stream for rendering within the user interface tiles for each of the at least two regions of interest.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2023
From: HOANG, ANTHONY BAO QUOC; LEE, CYNTHIA ESHIUAN; SMITH, JEFFREY WILLIAM; YU, CHI-CHIAN
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 064725/0067 →
Continuity (2)
Continuation 17472534 · Sep 10, 2021
Related Publication 20230412763A1 · Dec 21, 2023
References Cited (132)
US 7428000B2 · Cutler et al. · 2008 [cited by applicant]
US 7668907B1 · Janakiraman et al. · 2010 [cited by applicant]
US 8150155B2 · El-Maleh et al. · 2012 [cited by applicant]
US 8624955B2 · Watson · 2014 [cited by examiner]
US 8797376B2 · Saleh · 2014 [cited by examiner]
US 8842161B2 · Feng et al. · 2014 [cited by applicant]
US 8878893B2 · Andresen · 2014 [cited by examiner]
US 9055189B2 · Su · 2015 [cited by applicant]
US 9100540B1 · Gates et al. · 2015 [cited by applicant]
US 9237307B1 · Vendrow · 2016 [cited by applicant]
US 9294726B2 · Decker et al. · 2016 [cited by applicant]
US 9591479B1 · Leavy et al. · 2017 [cited by applicant]
US 9706171B1 · Riley et al. · 2017 [cited by applicant]
US 9769424B2 · Michot · 2017 [cited by applicant]
US 9774823B1 · Gadnir et al. · 2017 [cited by applicant]
US 9858936B2 · Cartwright et al. · 2018 [cited by applicant]
US 9942518B1 · Tangeland et al. · 2018 [cited by applicant]
US 10057707B2 · Cartwright et al. · 2018 [cited by applicant]
US 10104338B2 · Goesnar · 2018 [cited by applicant]
US 10187579B1 · Wang et al. · 2019 [cited by applicant]
US 10440325B1 · Boxwell et al. · 2019 [cited by applicant]
US 10516852B2 · Theien et al. · 2019 [cited by applicant]
US 10522151B2 · Cartwright et al. · 2019 [cited by applicant]
US 10574899B2 · Wang et al. · 2020 [cited by applicant]
US 10778941B1 · Childress, Jr. et al. · 2020 [cited by applicant]
US 10904485B1 · Childress, Jr. et al. · 2021 [cited by applicant]
US 10939045B2 · Wang et al. · 2021 [cited by applicant]
US 10991108B2 · Schnittman et al. · 2021 [cited by applicant]
US 11064256B1 · Voss et al. · 2021 [cited by applicant]
US 11076127B1 · Schaefer et al. · 2021 [cited by applicant]
US 11082661B1 · Pollefeys · 2021 [cited by applicant]
US 11165597B1 · Matsuguma et al. · 2021 [cited by applicant]
US 11212129B1 · Sipcic et al. · 2021 [cited by applicant]
US 11350029B1 · Ostap et al. · 2022 [cited by applicant]
US 11677907B2 · Goggins · 2023 [cited by examiner]
US 11722639B2 · Jiang · 2023 [cited by examiner]
US 11736660B2 · Smith et al. · 2023 [cited by applicant]
US 11765318B2 · Kies · 2023 [cited by examiner]
US 11843898B2 · Hoang · 2023 [cited by examiner]
US 20020113862A1 · Center et al. · 2002 [cited by applicant]
US 20050078171A1 · Firestone · 2005 [cited by examiner]
US 20050081160A1 · Wee et al. · 2005 [cited by applicant]
US 20060092269A1 · Baird · 2006 [cited by examiner]
US 20060244817A1 · Harville · 2006 [cited by examiner]
US 20070285504A1 · Hesse · 2007 [cited by examiner]
US 20080184115A1 · Back et al. · 2008 [cited by applicant]
US 20080273078A1 · Grasley · 2008 [cited by examiner]
US 20080297586A1 · Kurtz · 2008 [cited by examiner]
US 20080297587A1 · Kurtz · 2008 [cited by examiner]
US 20080297588A1 · Kurtz · 2008 [cited by examiner]
US 20080297589A1 · Kurtz · 2008 [cited by examiner]
US 20090015658A1 · Enstad · 2009 [cited by examiner]
US 20090147070A1 · Marathe · 2009 [cited by examiner]
US 20090207233A1 · Mauchly · 2009 [cited by examiner]
US 20090210491A1 · Thakkar · 2009 [cited by examiner]
US 20100118112A1 · Nimri · 2010 [cited by examiner]
US 20100123770A1 · Friel et al. · 2010 [cited by applicant]
US 20100238262A1 · Kurtz et al. · 2010 [cited by applicant]
US 20110085017A1 · Robinson · 2011 [cited by examiner]
US 20110279631A1 · Ranganath · 2011 [cited by examiner]
US 20110285808A1 · Feng et al. · 2011 [cited by applicant]
US 20120093365A1 · Aragane · 2012 [cited by examiner]
US 20120212668A1 · Schultz et al. · 2012 [cited by applicant]
US 20120293606A1 · Watson et al. · 2012 [cited by applicant]
US 20130027505A1 · Ranganath · 2013 [cited by examiner]
US 20130076853A1 · Diao · 2013 [cited by examiner]
US 20130083153A1 · Lindbergh · 2013 [cited by examiner]
US 20130088565A1 · Buckler · 2013 [cited by applicant]
US 20130194374A1 · Kieft · 2013 [cited by examiner]
US 20130198629A1 · Tandon et al. · 2013 [cited by applicant]
US 20130321564A1 · Smith · 2013 [cited by examiner]
US 20130342636A1 · Tian · 2013 [cited by examiner]
US 20140002582A1 · Bear · 2014 [cited by examiner]
US 20140267561A1 · Ahuja · 2014 [cited by examiner]
US 20140267562A1 · Lemmey · 2014 [cited by examiner]
US 20140267563A1 · Baca · 2014 [cited by examiner]
US 20140282166A1 · Temkin · 2014 [cited by examiner]
US 20150085065A1 · Sun · 2015 [cited by examiner]
US 20150181164A1 · Yang · 2015 [cited by examiner]
US 20150195489A1 · Sobti · 2015 [cited by examiner]
US 20160057385A1 · Burenius · 2016 [cited by applicant]
US 20160073055A1 · Marsh · 2016 [cited by applicant]
US 20160105642A1 · Nagase · 2016 [cited by examiner]
US 20160173823A1 · Duckworth et al. · 2016 [cited by applicant]
US 20160277712A1 · Michot · 2016 [cited by applicant]
US 20170094222A1 · Tangeland et al. · 2017 [cited by applicant]
US 20170099461A1 · Nimri et al. · 2017 [cited by applicant]
US 20170140791A1 · Das et al. · 2017 [cited by applicant]
US 20180063206A1 · Faulkner et al. · 2018 [cited by applicant]
US 20180063479A1 · Nimri et al. · 2018 [cited by applicant]
US 20180063480A1 · Luks et al. · 2018 [cited by applicant]
US 20180098026A1 · Gadnir et al. · 2018 [cited by applicant]
US 20180124136A1 · Faulkner et al. · 2018 [cited by applicant]
US 20180225852A1 · Um et al. · 2018 [cited by applicant]
US 20180232920A1 · Faulkner et al. · 2018 [cited by applicant]
US 20190215464A1 · Kumar · 2019 [cited by examiner]
US 20190222892A1 · Faulkner et al. · 2019 [cited by applicant]
US 20190250968A1 · Bemanian · 2019 [cited by examiner]
US 20190324986A1 · Schwartz · 2019 [cited by examiner]
US 20190341050A1 · Diamant et al. · 2019 [cited by applicant]
US 20200099890A1 · Tanaka et al. · 2020 [cited by applicant]
US 20200126513A1 · Lau et al. · 2020 [cited by applicant]
US 20200260049A1 · Erna · 2020 [cited by applicant]
US 20200267427A1 · Rogers et al. · 2020 [cited by applicant]
US 20200275058A1 · Graham et al. · 2020 [cited by applicant]
US 20200403817A1 · Daredia et al. · 2020 [cited by applicant]
US 20210120208A1 · Morris et al. · 2021 [cited by applicant]
US 20210181930A1 · Gurule · 2021 [cited by examiner]
US 20210235040A1 · Childress, Jr. et al. · 2021 [cited by applicant]
US 20210405865A1 · Faulkner · 2021 [cited by applicant]
US 20210409893A1 · Sommer · 2021 [cited by applicant]
US 20220060660A1 · Chou et al. · 2022 [cited by applicant]
US 20220086390A1 · Erna et al. · 2022 [cited by applicant]
US 20220141259A1 · Copley · 2022 [cited by examiner]
US 20220253025A1 · Piaskowski · 2022 [cited by examiner]
US 20220319032A1 · Fan et al. · 2022 [cited by applicant]
US 20220319034A1 · Fan et al. · 2022 [cited by applicant]
US 20220353465A1 · Smith · 2022 [cited by applicant]
US 20220374136A1 · Chang · 2022 [cited by examiner]
US 20220400216A1 · Wang et al. · 2022 [cited by applicant]
US 20230081717A1 · Hoang · 2023 [cited by examiner]
US 20230412763A1 · Hoang · 2023 [cited by examiner]
GB 2594761B · 2022 [cited by applicant]
WO 2021243633A1 · 2021 [cited by applicant]
WO 2022046810A2 · 2022 [cited by applicant]
WO 2022078656A1 · 2022 [cited by applicant]
International Search Report and Written Opinion mailed on Jul. 4, 2022 in corresponding PCT Application No. PCT/US2022/024820. [cited by applicant]
International Search Report and Written Opinion mailed on Jul. 25, 2022 in corresponding PCT Application Numbber PCT/US2022/024822. [cited by applicant]
International Search Report and Written Opinion mailed on Dec. 22, 2022 in corresponding PCT Application No. PCT/US2022/042860. [cited by applicant]
International Search Report and Written Opinion mailed on May 8, 2023 in corresponding PCT Application No. PCT/US2023/011368. [cited by applicant]
Paul Chippendale and Oswald Lanz: “Optimised Meeting Recording and Annotation Using Real-Time Video Analysis”, in: Andrej Popescu-Selis: “Machine Learning for Multimodal Interaction”, Proc. of 5th MLMI workshop, Sep. 10… [cited by applicant]
Rasmussen Troels Ammitsbol et al: “SceneCam: Improving Multti-camera Remote Collaboration using Augmented Reality”, 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (!SMAR-ADJUNCT), IEEE, Oct. 10… [cited by applicant]