IP Library Granted Patent US 12,450,928
Granted Patent B2
US 12,450,928 · App. 17/832,640 · Granted Oct 21, 2025

Resolution-based extraction of textual content from video of a communication session

Inventor: Renjie Tao (Sunnyvale, CA)
Assignee: Zoom Communications, Inc.
G06V30/153G06V20/46G06V20/49G06V30/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,450,928
App. No.
17/832,640
Granted
Oct 21, 2025
Kind
B2
Abstract

Methods and systems provide for resolution-based extraction of textual content. In one embodiment, the system receives video content of a communication session with participants. The system then extracts high-resolution versions and low-resolution versions of frames from the video content, and classifies the low-resolution frames of the video content based on identifying text within the low-resolution frames. The system identifies one or more low-resolution distinguishing frames containing text. For each low-resolution distinguishing frame containing text, the system detects a title within the frame, crops a title area with the title within the frame, and extracts, via optical character recognition (“OCR”), the title from the cropped title area of the high-resolution version of the frame. The system extracts, via OCR, textual content from the high-resolution versions of the low-resolution distinguishing frames containing text, and then transmits the extracted textual content and extracted titles to one or more client devices.

Claims (64)

1. A method, comprising:

receiving video content of a communication session comprising a plurality of participants;

extracting high-resolution versions and low-resolution versions of frames from the video content;

classifying the low-resolution frames of the video content;

identifying one or more low-resolution distinguishing frames comprising text;

for each low-resolution distinguishing frame comprising text:

detecting a title within the frame,

cropping a title area with the title within the frame, and

extracting, via optical character recognition (OCR), the title from the cropped title area of the high-resolution version of the frame;

extracting, via OCR, textual content from the high-resolution versions and the low-resolution distinguishing frames comprising text; and

transmitting, to one or more client devices, the extracted textual content and the extracted titles.

2. The method of claim 1 , wherein extracting the title from the cropped title area of the high-resolution version of the frame comprises:

identifying a timestamp corresponding to the low-resolution frame;

locating the high-resolution version of the low-resolution frame via the timestamp; and

extracting, via OCR, the title from the high-resolution version of the frame.

3. The method of claim 1 , wherein extracting the textual content from the high-resolution versions of the low-resolution distinguishing frames comprising text comprises:

identifying timestamps corresponding to each of the low-resolution distinguishing frames comprising text;

locating the high-resolution versions of the low-resolution frames via the corresponding timestamps; and

extracting, via OCR, the textual content from the high-resolution versions of the low-resolution distinguishing frames.

4. The method of claim 1 , wherein the low-resolution frames of the video content may be classified as one or more of: a black frame, a face frame, a slide frame, and a demo frame.

5. The method of claim 4 , wherein identifying one or more low-resolution distinguishing frames comprising text comprises:

filtering out low-resolution frames which are classified as a black frame or a face frame.

6. The method of claim 1 , wherein identifying one or more low-resolution distinguishing frames comprising text comprises:

filtering out low-resolution frames which do not contain text.

7. The method of claim 1 , wherein identifying one or more low-resolution distinguishing frames comprising text comprises:

filtering out low-resolution frames which cannot be distinguished from neighboring frames based on the content of the frame.

8. The method of claim 7 , wherein detecting the title within the frame further comprises:

generating one or more segregated bounding boxes within grids of residual blocks.

9. The method of claim 8 , wherein detecting the title within the frame further comprises:

determining, via intersection of union (IOU) techniques, a top bounding box with highest prediction confidence for the title from the segregated bounding boxes.

10. The method of claim 1 , wherein detecting the title within the frame is based on one or more title detection rules.

11. The method of claim 1 , wherein detecting the title within the frame comprises one or more machine learning algorithms.

12. The method of claim 1 , further comprising:

determining a layout analysis of each low-resolution distinguishing frame comprising text.

13. The method of claim 12 , wherein determining the layout analysis of each low-resolution distinguishing frame comprising text comprises classifying a plurality of areas of the frame into one or more of: text, title, table, image, and list arcas.

14. The method of claim 12 , wherein determining the layout analysis of each low-resolution distinguishing frame comprising text comprises one or more deep neural network techniques.

15. The method of claim 12 , wherein determining the layout analysis of each low-resolution distinguishing frame comprising text comprises one or more image processing techniques.

16. The method of claim 12 , wherein determining the layout analysis of each low-resolution distinguishing frame comprising text comprises one or more image processing techniques.

17. A communication system comprising:

one or more processors configured to:

receive video content of a communication session comprising a plurality of participants;

extract high-resolution versions and low-resolution versions of frames from the video content;

classify the low-resolution frames of the video content;

identify one or more low-resolution distinguishing frames comprising text;

for each low-resolution distinguishing frame comprising text:

detect a title within the frame,

crop a title area with the title within the frame, and

extract, via optical character recognition (OCR), the title from the cropped title area of the high-resolution version of the frame;

extract, via OCR, textual content from the high-resolution versions and the low-resolution distinguishing frames comprising text; and

transmit, to one or more client devices, the extracted textual content and the extracted titles.

18. The communication system of claim 17 , wherein the low-resolution frames of the video content may be classified as one or more of: a black frame, a face frame, a slide frame, and a demo frame.

19. The communication system of claim 17 , wherein the one or more processors are further configured to:

determine a semantic understanding of each low-resolution distinguishing frame comprising text.

20. A non-transitory computer-readable medium containing instructions, that when executed by a processor, cause the processor to perform comprising:

receiving video content of a communication session comprising a plurality of participants;

extracting high-resolution versions and low-resolution versions of frames from the video content;

classifying the low-resolution frames of the video content;

identifying one or more low-resolution distinguishing frames comprising text;

for each low-resolution distinguishing frame comprising text:

detecting a title within the frame,

cropping a title area with the title within the frame, and

extracting, via optical character recognition (OCR), the title from the cropped title area of the high-resolution version of the frame;

extracting, via OCR, textual content from the high-resolution versions of the low-resolution distinguishing frames comprising text; and

transmitting, to one or more client devices, the extracted textual content and the extracted titles.

Assignments (2)
CHANGE OF NAME Recorded Jan 7, 2025
From: ZOOM VIDEO COMMUNICATIONS, INC.
To: ZOOM COMMUNICATIONS, INC.
Reel/Frame 069839/0593 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 15, 2022
From: TAO, RENJIE
To: ZOOM VIDEO COMMUNICATIONS, INC.
Reel/Frame 060811/0758 →