IP Library › Granted Patent US 11,611,600
Granted Patent B1
US 11,611,600 · App. 17/411,905 · Granted Mar 21, 2023

Streaming data processing for hybrid online meetings

Inventors: Shai Ifrach (Yavne, IL); Haim Somech (Ramat Gan, IL)
Assignee: Microsoft Technology Licensing, LLC
H04L65/403G06V40/172G10L17/02G10L25/78H04L65/60H04N5/23299H04N5/2628
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,611,600
App. No.
17/411,905
Granted
Mar 21, 2023
Kind
B1
Abstract

Techniques of streaming data processing for hybrid online meetings are disclosed herein. In one example, a method includes receiving, at the remote server, a video stream captured by a camera in the conference room. The video stream captures images of multiple local participants of an online meeting. The method also includes determining identities of the captured images of the multiple local participants in the received video stream using meeting information of the online meeting and generating a set of individual video streams each corresponding to one of the multiple local participants. The set of individual video streams can then be transmitted to the second computing device corresponding to a remote participant of the online meeting as if the multiple local participants are virtually joining the online meeting.

Claims (79)

1. A method of streaming data processing in an online meeting platform having a remote server, a first computing device located in a conference room, and a second plurality of remote computing devices remote from the conference room interconnected via a computer network, the method comprising:

receiving, at the remote server, a video stream captured by a camera of the first computing device located in the conference room, the video stream capturing images of multiple local participants of an online meeting that also has a plurality of remote participants joining the online meeting via the computer network using the second plurality of remote computing devices;

receiving, at the remote server, a plurality of video streams captured by cameras of the second plurality of remote computing devices, the plurality of video streams including a first remote video stream associated with a first remote participant and a second remote video stream associated with a second remote participant;

upon receiving the video stream captured by the camera of the first computing device, at the remote server,

determining identities of the captured images of the multiple local participants in the received video stream using meeting information of the online meeting; and

editing the received video stream from the first computing device to generate a set of individual video streams each corresponding to one of the multiple local participants and associating the generated set of individual video streams with the determined identities of the multiple local participants in the conference room;

transmitting, to a first remote computing device associated with the first remote participant, the generated set of individual video streams and the second remote video stream; and

transmitting, to a second remote computing device associated with the second remote participant, the generated set of individual video streams and the first remote video stream.

2. The method of claim 1 , further comprising:

retrieving, from a network storage, data representing the meeting information of the online meeting, the meeting information containing data representing names and identification photos of the multiple local participants; and

wherein determining the identities of the captured images of the multiple local participants includes determining the identities of the multiple local participants using the identification photos via facial recognition.

3. The method of claim 1 wherein determining the identities of the multiple participants includes:

determining that the first remote participant and the second remote participant are joining the online meeting remotely; and

in response to determining that the first remote participant and the second remote participant are joining the online meeting remotely, skipping determining an identity of the first remote participant and the second remote participant who are joining the online meeting remotely.

4. The method of claim 1 , further comprising:

retrieving, from a network storage, data representing meeting information of the online meeting, the meeting information containing data representing identities of all participants; and

wherein determining the identities of the multiple participants includes:

determining that the first remote participant and the second remote participant are joining the online meeting remotely; and

in response to determining that the first remote participant and the second remote participant are joining the online meeting remotely, retrieving identification photos of the first remote participant and the second remote participant from a contact list and using the retrieved identification photos to determine identities of the first remote participant and the second remote participant captured in the first remote video stream and the second remote video stream.

5. The method of claim 1 , further comprising:

retrieving, from a network storage, data representing meeting information of the online meeting, the meeting information containing data representing voice samples of all participants; and

wherein determining the identities of the multiple participants includes:

determining that the first remote participant and the second remote participant are joining the online meeting remotely; and

in response to determining that the first remote participant and the second remote participant are joining the online meeting remotely, using the voice samples in the retrieved data representing the meeting information to determine identities of the first remote participant and the second remote participant captured in the first remote video stream and the second remote video stream via voice matching.

6. The method of claim 1 wherein editing the received video stream includes:

for one of the multiple local participants,

cropping the received video stream to remove all other local participants from the received video stream; and

temporarily saving the cropped video stream as one of the individual video streams in the generated set.

7. The method of claim 1 wherein editing the received video stream includes:

for one of the local participants,

cropping the received video stream to remove all other local participants from the received video stream;

digitally enlarging the cropped video stream into a preset size; and

temporarily saving the digitally enlarged video stream as one of the individual video streams in the generated set.

8. The method of claim 1 ,

wherein transmitting the generated set of individual video streams to the first remote computing device includes transmitting the generated set of individual video streams with the associated identities of the multiple local participants in the conference room to the first remote computing device instead of the received video stream received from the first computing device, and

wherein transmitting the generated set of individual video streams to the second remote computing device includes transmitting the generated set of individual video streams with the associated identities of the multiple local participants in the conference room to the second remote computing device instead of the received video stream received from the first device.

9. A server in an online meeting platform having a first computing device located in a conference room and a second plurality of remote computing devices remote from the conference room interconnected to the server via a computer network, the server comprising:

a processor; and

a memory operatively coupled to the processor, the memory containing instructions executable by the processor to cause the computing device to:

receive a video stream captured by a camera of the first computing device located in the conference room, the video stream capturing images of multiple local participants of an online meeting that also has a plurality of remote participants joining the online meeting via the computer network using the second plurality of remote computing devices;

receive a plurality of video streams captured by cameras of the second plurality of remote computing devices, the plurality of video streams including a first remote video stream associated with a first remote participant and a second remote video stream associated with a second remote participant;

upon receiving the video stream captured by the camera of the first computing device located in the conference room,

determine identities of the captured images of the multiple local participants in the received video stream using meeting information of the online meeting; and

generate, based on the received video stream from the first computing device, a set of individual video streams each corresponding to one of the multiple local participants and associate the generated set of individual video streams with the determined identities of the multiple local participants;

transmit, to a first remote computing device associated with the first remote participant, the generated set of individual video streams and the second remote video stream; and

transmit, to a second remote computing device associated with the second remote participant, the generated set of individual video streams and the first remote video stream.

10. The computing device of claim 9 wherein the memory contains additional instructions executable by the processor to cause the computing device to:

retrieve, from a network storage, data representing the meeting information of the online meeting, the meeting information containing data representing names and identification photos of the multiple local participants; and

wherein to determine the identities of the captured images of the multiple local participants includes to determine the identities of the multiple local participants using the identification photos via facial recognition.

11. The computing device of claim 9 wherein the memory contains additional instructions executable by the processor to cause the computing device to:

retrieve, from a network storage, data representing meeting information of the online meeting, the meeting information containing data representing voice samples of all participants; and

wherein to determine the identities of the multiple participants includes to determine the identities of the multiple local participants using the voice samples in the meeting information via voice matching.

12. The computing device of claim 9 wherein to generate the set of individual video streams includes to:

for one of the multiple local participants,

crop the received video stream to remove all other local participants from the received video stream; and

temporarily save the cropped video stream as one of the individual video streams in the generated set.

13. The computing device of claim 9 wherein to generate the set of individual video streams includes to:

for one of the local participants,

crop the received video stream to remove all other local participants from the received video stream;

digitally format the cropped video stream into a preset size; and

temporarily save the digitally enlarged video stream as one of the individual video streams in the generated set.

14. A method of streaming data processing in an online meeting platform having a remote server, a first computing device located in a conference room, and a second plurality of remote computing devices remote from the conference room interconnected via a computer network, the method comprising:

receiving, at the remote server, a video stream captured by a camera of the first computing device located in the conference room, the video stream selectively capturing images of a first local participant from a plurality of local participants based on detected speech from the local participant;

receiving, at the remote server, a plurality of video streams captured by cameras of the second plurality of remote computing devices, the plurality of video streams including a first remote video stream associated with a first remote participant and a second remote video stream associated with a second remote participant;

transmitting, to a first remote computing device associated with the first remote participant, the video stream captured by the camera of the first computing device and the second remote video stream; and

transmitting, to a second remote computing device associated with the second remote participant, to a second remote computing device associated with the second remote participant, the generated set of individual video streams and the first remote video stream.

15. The method of claim 14 , further comprising:

retrieving, from a network storage, data representing the meeting information of the online meeting, the meeting information containing data representing the identity of the first local participant; and

wherein associating the captured video stream with the identity of the first local participant includes associating the captured video stream with the identity of the local participant in the retrieved data representing the meeting information of the online meeting.

16. The method of claim 14 , further comprising:

retrieving, from a network storage, data representing the meeting information of the online meeting, the meeting information containing data representing an identity and an identification photo of the first local participant; and

wherein associating the captured video stream with the identity of the local participant includes:

identifying the first local participant using the identification photo via facial recognition; and

associating the captured video stream with the identity of the local participant in the retrieved data representing the meeting information of the online meeting.

17. The method of claim 14 , further comprising:

retrieving, from a network storage, data representing meeting information of the online meeting, the meeting information containing data representing a voice sample of the first local participant; and

wherein associating the captured video stream with the identity of the local participant includes:

identifying the first local participant using the voice sample via voice matching; and

associating the captured video stream with the identity of the first local participant in the retrieved data representing the meeting information of the online meeting.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: IFRACH, SHAI; SOMECH, HAIM
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 058487/0717 →
Cited By (2)
US 12,192,669 US 12,407,790