IP Library Granted Patent US 11,558,209
Granted Patent B1
US 11,558,209 · App. 17/390,236 · Granted Jan 17, 2023

Automatic spotlight in video conferencing

Inventor: Samuel Lum (Sunnyvale, CA)
Assignee: Zoom Video Communications, Inc.
H04L12/1818G06K9/628H04L12/1822
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,209
App. No.
17/390,236
Granted
Jan 17, 2023
Kind
B1
Abstract

Automatic spotlighting of participant video feeds in a video conference are based on one or more triggers detected in the participant video feeds. Participant video feeds are added to a spotlight video queue. The participant video feeds are elevated to an active spotlight status based on certain criteria. The participant video feeds that are elevated to the active spotlight status are displayed adjacent to a host video feed on a display.

Claims (90)

1. A method, comprising:

detecting activity in a participant video feed of a video conference; determining an activity relevance score corresponding to the detected activity, wherein the activity relevance score is based on a correlation of a participant activity in the participant video feed relative to a host activity in a host video feed;

adding the participant video feed to a spotlight queue when the activity relevance score is above a relevance threshold;

elevating the participant video feed from an inactive spotlight status to an active spotlight status; and displaying the participant video feed adjacent to the host video feed on a display.

2. The method of claim 1 , comprising: initiating a timer when the participant video feed is displayed adjacent to the host video feed; demoting the participant video feed from the active spotlight status when the timer expires; and removing the participant video feed from being displayed.

3. The method of claim 1 , comprising:

initiating a timer when the participant video feed is displayed adjacent to the host video feed;

demoting the participant video feed from the active spotlight status when the timer expires;

removing the participant video feed from being displayed;

detecting a second activity in a second participant video feed of the video conference;

determining a second activity relevance score corresponding to the second detected activity;

adding the second participant video feed to the spotlight queue when the second activity relevance score is above the relevance threshold;

elevating the second participant video feed from the inactive spotlight status to the active spotlight status when the timer expires; and

displaying the second participant video feed adjacent to the host video feed on the display.

4. The method of claim 1 , comprising:

initiating a timer when the participant video feed is displayed adjacent to the host video feed;

demoting the participant video feed from the active spotlight status when the timer expires;

removing the participant video feed from being displayed;

detecting a third activity in the first participant video feed prior to the timer expiring;

determining a third activity relevance score corresponding to the third detected activity; and

restarting the timer when the third activity relevance score is above the relevance threshold.

5. The method of claim 1 , comprising: detecting a second activity in a second participant video feed of the video conference; determining a second activity relevance score corresponding to the second detected activity; comparing the activity relevance score to the second activity relevance score, wherein the activity relevance score is above the second activity relevance score; and adding the second participant video feed to the spotlight queue below the participant video feed.

6. The method of claim 1 , wherein the spotlight queue comprises multiple participant video feeds, wherein the two participant video feeds that have been in the spotlight queue the longest are elevated to active spotlight status and removed from the spotlight queue, and comprising displaying the two participant video feeds adjacent to the host video feed on the display.

7. The method of claim 1 , wherein the spotlight queue comprises multiple participant video feeds, wherein the two participant video feeds that have the highest activity relevance scores are elevated to active spotlight status and removed from the spotlight queue, and comprising displaying the two participant video feeds adjacent to the host video feed on the display.

8. The method of claim 1 , wherein the detected activity is audio corresponding to the participant video feed, and comprising automatically unmuting the audio when the participant video feed is displayed.

9. The method of claim 1 , wherein the activity relevance score is based on a correlation of a first object activity in the participant video feed relative to a second object activity in the host video feed.

10. A video conference system, comprising:

a server comprising a processor and a memory configured to store instructions;

a host device configured to transmit a host video feed to the server; and

a participant device configured to transmit a participant video feed to the server;

wherein the processor of the server is configured to execute the instructions stored in the memory to:

detect activity in the participant video feed;

determine an activity relevance score corresponding to the detected activity, wherein the activity relevance score is based on a correlation of a participant activity in the participant video feed relative to a host activity in the host video feed;

add the participant video feed to a spotlight queue when the activity relevance score is above a relevance threshold;

elevate the participant video feed from an inactive spotlight status to an active spotlight status; and

display the participant video feed adjacent to the host video feed.

11. The video conference system of claim 10 , wherein the server is further configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires; and

remove the participant video feed from being displayed.

12. The video conference system of claim 10 , wherein the server is further configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires;

remove the participant video feed from being displayed; detect a second activity in a second participant video feed of the video conference;

determine a second activity relevance score corresponding to the second detected activity;

add the second participant video feed to the spotlight queue when the second activity relevance score is above the relevance threshold;

elevate the second participant video feed from the inactive spotlight status to the active spotlight status when the timer expires; and

display the second participant video feed adjacent to the host video feed on the display.

13. The video conference system of claim 10 , wherein the server is further configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires;

remove the participant video feed from being displayed; detect a third activity in the first participant video feed prior to the timer expiring;

determine a third activity relevance score corresponding to the third detected activity; and

restart the timer when the third activity relevance score is above the relevance threshold.

14. The video conference system of claim 10 , wherein the server is further configured to:

detect a second activity in a second participant video feed of the video conference;

determine a second activity relevance score corresponding to the second detected activity;

compare the activity relevance score to the second activity relevance score, wherein the activity relevance score is above the second activity relevance score; and

add the second participant video feed to the spotlight queue below the participant video feed.

15. A non-transitory computer-readable medium configured to store machine-readable instructions that when executed by a processor, cause the processor to:

detect activity in a participant video feed of a video conference;

determine an activity relevance score corresponding to the detected activity, wherein the activity relevance score is based on a correlation of a participant activity in the participant video feed relative to a host activity in a host video feed;

add the participant video feed to a spotlight queue when the activity relevance score is above a relevance threshold;

elevate the participant video feed from an inactive spotlight status to an active spotlight status; and

display the participant video feed adjacent to f—the host video feed on a display.

16. The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires; and

remove the participant video feed from being displayed.

17. The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires;

remove the participant video feed from being displayed;

detect a second activity in a second participant video feed of the video conference;

determine a second activity relevance score corresponding to the second detected activity;

add the second participant video feed to the spotlight queue when the second activity relevance score is above the relevance threshold;

elevate the second participant video feed from the inactive spotlight status to the active spotlight status when the timer expires; and

display the second participant video feed adjacent to the host video feed on the display.

18. The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to:

initiate a timer when the participant video feed is displayed adjacent to the host video feed;

demote the participant video feed from the active spotlight status when the timer expires;

remove the participant video feed from being displayed; detect a third activity in the first participant video feed prior to the timer expiring;

determine a third activity relevance score corresponding to the third detected activity; and

restart the timer when the third activity relevance score is above the relevance threshold.

19. The non-transitory computer-readable medium of claim 15 , wherein the processor is configured to:

detect a second activity in a second participant video feed of the video conference;

determine a second activity relevance score corresponding to the second detected activity;

compare the activity relevance score to the second activity relevance score, wherein the activity relevance score is above the second activity relevance score; and

add the second participant video feed to the spotlight queue below the participant video feed.

20. The non-transitory computer-readable medium of claim 15 , wherein the activity relevance score is based on a correlation of a first object activity in the participant video feed relative to a second object activity in the host video feed.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2021
From: LUM, SAMUEL
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 057041/0398 →
Cited By (2)
US 1,124,018 US 12,439,156