IP Library Granted Patent US 11,750,680
Granted Patent B2
US 11,750,680 · App. 16/909,437 · Granted Sep 5, 2023

Systems and methods to selectively combine video streams

Inventor: Oliver Pell (London, GB)
Assignee: Meta Platforms, Inc.
H04L67/01H04L12/1813H04L12/1827H04L51/046H04L51/52H04L65/75H04L67/02H04L67/104H04L51/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,750,680
App. No.
16/909,437
Granted
Sep 5, 2023
Kind
B2
Abstract

Systems, methods, and non-transitory computer readable media are configured for determining that a first peer involved in a video conference is a primary peer; determining that a second peer involved in the video conference is a secondary peer in response to non-satisfaction by the second peer of a threshold associated with a selected number of objects in the second peer; determining a viewing peer involved in the video conference, wherein the primary peer, the secondary peer, and the viewing peer are different; creating a modified video stream of the second peer in response to the determining that the second peer is the secondary peer based on the non-satisfaction by the second peer of the threshold associated with the selected number of objects in the second peer; detecting a static area and a dynamic area in a video stream of the first peer; superimposing the modified video stream of the second peer onto the static area in the video stream of the first peer; and providing a composite video including the video stream of the first peer, the modified video stream of the second peer, and a video stream of the viewing peer for presentation to the viewing peer.

Claims (57)

1. A computer-implemented method comprising:

determining, by a computing system, that a first peer involved in a video conference is a primary peer;

determining, by the computing system, that a second peer involved in the video conference is a secondary peer in response to non-satisfaction by the second peer of a threshold associated with a selected number of objects in the second peer;

determining, by the computing system, a viewing peer involved in the video conference, wherein the primary peer, the secondary peer, and the viewing peer are different;

creating, by the computing system, a modified video stream of the second peer in response to the determining that the second peer is the secondary peer based on the non-satisfaction by the second peer of the threshold associated with the selected number of objects in the second peer;

detecting, by the computing system, a static area and a dynamic area in a video stream of the first peer;

superimposing, by the computing system, the modified video stream of the second peer onto the static area in the video stream of the first peer; and

providing, by the computing system, a composite video including the video stream of the first peer, the modified video stream of the second peer, and a video stream of the viewing peer for presentation to the viewing peer.

2. The computer-implemented method of claim 1 , wherein determination of the primary peer is based on satisfaction of the threshold and determination of the secondary peer is based on non-satisfaction of the threshold.

3. The computer-implemented method of claim 1 , wherein the detecting the static area in the video stream of the first peer further comprises:

performing image recognition on the video stream of the first peer; and

identifying an area in the video stream of the first peer that does not reflect at least one of motion or change as the static area.

4. The computer-implemented method of claim 1 , further comprising:

recognizing objects in a video stream of at least one of the first peer or the second peer.

5. The computer-implemented method of claim 4 , further comprising:

determining a probability that at least one object in the video stream is a user;

recognizing the at least one object as a user if the probability satisfies a threshold that reflects a desired confidence level; and

determining a number of recognized users in the video stream.

6. The computer-implemented method of claim 5 , wherein recognition of the at least one object as a user is based on a classifier trained under a supervised learning technique, the supervised learning technique based on multiple images of users that have been tagged with identifying information.

7. The computer-implemented method of claim 1 , wherein the modified video stream of the second peer is superimposed onto the video stream of the first peer based on a determination that a difference in location between the first peer and a single user associated with the second peer satisfies a distance threshold.

8. The computer-implemented method of claim 1 , further comprising:

receiving an input by a user associated with the viewing peer to change presentation of the composite video; and

changing the presentation of the composite video in response to the input.

9. The method of claim 1 ,

the creating of the modified video stream removes at least some subject matter in a video stream of the second peer.

10. A system comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, cause the system to perform:

determining that a first peer involved in a video conference is a primary peer;

determining that a second peer involved in the video conference is a secondary peer in response to non-satisfaction by the second peer of a threshold associated with a selected number of objects in the second peer;

determining a viewing peer involved in the video conference, wherein the primary peer, the secondary peer, and the viewing peer are different;

creating a modified video stream of the second peer in response to the determining that the second peer is the secondary peer based on the non-satisfaction by the second peer of the threshold associated with the selected number of objects in the second peer;

detecting a static area and a dynamic area in a video stream of the first peer;

superimposing the modified video stream of the second peer onto the static area in the video stream of the first peer; and

providing a composite video including the video stream of the first peer, the modified video stream of the second peer, and a video stream of the viewing peer for presentation to the viewing peer.

11. The system of claim 10 , wherein determination of the primary peer is based on satisfaction of the threshold and determination of the secondary peer is based on non-satisfaction of the threshold.

12. The system of claim 11 , wherein the threshold is a selected number of users associated with a peer.

13. The system of claim 10 , wherein the detecting the static area in the video stream of the first peer further comprises:

performing image recognition on the video stream of the first peer; and

identifying an area in the video stream of the first peer that does not reflect at least one of motion or change as the static area.

14. The system of claim 10 , further comprising:

recognizing objects in a video stream of at least one of the first peer or the second peer.

15. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:

determining that a first peer involved in a video conference is a primary peer;

determining that a second peer involved in the video conference is a secondary peer in response to non-satisfaction by the second peer of a threshold associated with a selected number of objects in the second peer;

determining a viewing peer involved in the video conference, wherein the primary peer, the secondary peer, and the viewing peer are different;

creating a modified video stream of the second peer in response to the determining that the second peer is the secondary peer based on the non-satisfaction by the second peer of the threshold associated with the selected number of objects in the second peer;

detecting a static area and a dynamic area in a video stream of the first peer;

superimposing the modified video stream of the second peer onto the static area in the video stream of the first peer; and

providing a composite video including the video stream of the first peer, the modified video stream of the second peer, and a video stream of the viewing peer for presentation to the viewing peer.

16. The non-transitory computer-readable storage medium of claim 15 , wherein determination of the primary peer is based on satisfaction of the threshold and determination of the secondary peer is based on non-satisfaction of the threshold.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the threshold is a selected number of users associated with a peer.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the detecting the static area in the video stream of the first peer further comprises:

performing image recognition on the video stream of the first peer; and

identifying an area in the video stream of the first peer that does not reflect at least one of motion or change as the static area.

19. The non-transitory computer-readable storage medium of claim 15 , further comprising:

recognizing objects in a video stream of at least one of the first peer or the second peer.

Assignments (1)
CHANGE OF NAME Recorded Nov 24, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058238/0740 →
Continuity (2)
Continuation 14981840 · Dec 28, 2015
Related Publication 20200322295A1 · Oct 8, 2020