IP Library › Granted Patent US 11,412,303
Granted Patent B2
US 11,412,303 · App. 16/115,198 · Granted Aug 9, 2022

Filtering images of live stream content

Inventors: Jeremy R. Fox (Georgetown, TX); Jonathan Dunne (Dungarvan, IE); Liam S. Harpur (Dublin, IE); Kelley Anders (East New Market, MD)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
H04N21/4542H04N21/2187H04N21/251H04N21/45455
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,412,303
App. No.
16/115,198
Filed
Aug 28, 2018
Granted
Aug 9, 2022
Kind
B2
Art Unit
2421
USPC
725/25
Abstract

A method of filtering images of live stream content may include defining a prohibited frame content template; analyzing live stream content at a frame level to determine content within each frame of the live stream content; and comparing a frame of the live stream content against the prohibited frame content template to detect prohibited content in the frame that matches prohibited frame content as defined by the prohibited frame content template.

Claims (51)

1. A method of filtering images of live stream content, comprising:

defining a prohibited frame content template, wherein a prohibited frame content of the prohibited frame content template comprises images;

receiving a subscription of categories of prohibited frame content templates from a user, wherein a subscription indicates content to be prohibited;

analyzing portions of frames of live stream content in real time to determine content within each frame of the live stream content, wherein portions that are analyzed comprise portions of frames that change between frames;

comparing the content within each frame of the live stream content against a subscribed to prohibited frame content template to detect prohibited content in the frame that matches prohibited frame content as defined by the subscribed to prohibited frame content template; and

identifying, within the frame and based on the subscription, an outline of the prohibited content and blurring a portion of the frame within the outline.

2. The method of claim 1 , wherein analyzing the live stream content comprises using a computer vision process to identify prohibited content as defined by the subscribed to prohibited frame content template.

3. The method of claim 1 , wherein comparing the frame of the live stream content against the subscribed to prohibited frame content template comprises comparing an I-frame of the live stream content, the I-frame content comprising frame content for a plurality of subsequent frames in the live stream content.

4. The method of claim 3 , wherein the subsequent frames comprise P-frames, B-frames, or combinations thereof.

5. The method of claim 1 , wherein the prohibited content further comprises: sexually explicit images, weapon images, and images of people using profanity.

6. The method of claim 1 , wherein blurring a portion of the frame comprises:

identifying an extreme left pixel, an extreme right pixel, an extreme top pixel, and an extreme bottom pixel of the prohibited content; and

generating a box bound by the extreme left pixel, the extreme right pixel, the extreme top pixel, and the extreme bottom pixel to enclose the prohibited content; and

blurring a region defined by the box.

7. The method of claim 1 , wherein analyzing portions of frames comprises simultaneously analyzing I-frames, B-frames, and P-frames.

8. A computer program product for filtering images of live stream content, the computer program product comprising:

a computer readable storage medium comprising computer usable program code embodied therewith, the computer usable program code to, when executed by a processor:

define a prohibited frame content template, wherein a prohibited frame content in the prohibited frame content template comprises images;

categorize the prohibited frame content template based on types of images to be prohibited;

receive a subscription of categories of prohibited frame content templates from a user, wherein a subscription indicates a type of image to be prohibited;

analyze portions of frames of live stream content frame-by-frame in real time to determine content within each frame of the live stream content, wherein portions that are analyzed comprise portions of frames that change between frames;

compare a first frame of the live stream content against a subscribed to prohibited frame content template to detect prohibited content comprising content in the first frame that matches prohibited frame content as defined by the subscribed to prohibited frame content;

simultaneously compare a second frame of the live stream content against the subscribed to prohibited frame template to detect prohibited content in the second frame; and

in response to detection of prohibited content, take remedial action to filter the prohibited content in both the first frame and the second frame.

9. The computer program product of claim 8 , wherein analyzing the live stream content comprises using a computer vision process to identify prohibited content as defined by the subscribed to prohibited frame content template.

10. The computer program product of claim 9 , wherein the computer vision process comprises using machine learning processes.

11. The computer program product of claim 8 , comprising computer usable program code to, when executed by the processor, store the prohibited frame content template in a database.

12. The computer program product of claim 8 , wherein the remedial action comprises alerting a user to the existence of prohibited frame content within the live stream content, obfuscating a portion of a frame containing the prohibited frame content, automatically performing take down process where at least a portion of the live stream content is removed from live streaming, blocking users from accessing the live stream content, or combinations thereof.

13. The computer program product of claim 8 , wherein comparing the frame of the live stream content against the subscribed to prohibited frame content template comprises comparing an I-frame of the live stream content, the I-frame content comprising frame content for a plurality of subsequent frames in the live stream content.

14. The computer program product of claim 13 , wherein the subsequent frames comprise P-frames, B-frames, or combinations thereof.

15. The computer program product of claim 8 , wherein analyzing the live stream content comprises using a computer vision process to identify prohibited content as defined by the subscribed to prohibited frame content template, wherein the computer vision process comprises using machine learning processes;

wherein comparing the frame of the live stream content against the prohibited frame content template comprises comparing an I-frame of the live stream content, the I-frame content comprising frame content for a plurality of subsequent frames in the live stream content, and wherein the subsequent frames comprise P-frames, B-frames, or combinations thereof; and

wherein the prohibited content further comprises: sexually explicit images, weapon images, and images of people using profanity.

16. A system for filtering images of live stream content, comprising:

a computing device, comprising:

a processor;

a data storage device communicatively coupled to the processor, the data storage device comprising:

a template definition module to, when executed by the processor;

define a prohibited frame content template, wherein the prohibited frame content template comprises an image;

categorize the prohibited frame content template based on types of images to be prohibited; and

receive a subscription of categories of prohibited frame content templates from a user, wherein a subscription indicates a type of image to be prohibited;

an analysis module to, when executed by the processor, analyze portions of frames of live stream content to determine content within each frame of the live stream content, wherein:

portions that are analyzed comprise portions of frames that change between frames; and

a subsequent frame is weighted as including prohibited content based on a previous frame comprising prohibited content; and

a comparison module to, when executed by the processor, compare a frame of the live stream content against a subscribed to prohibited frame content template to detect prohibited content comprising an image of confidential information in the frame that matches prohibited frame content as defined by the subscribed to prohibited frame content template, and propagate detection of prohibited frame content detected in an I-frame into a subsequent P-frame or B-frame.

17. The system of claim 16 , comprising a remediation module to, when executed by the processor, take remedial action to filter the prohibited frame content in response to the detection of prohibited frame content.

18. The system of claim 17 , wherein the remedial action comprises alerting a user to the existence of prohibited frame content within the live stream content, obfuscating a portion of a frame containing the prohibited frame content, automatically performing take down process where at least a portion of the live stream content is removed from live streaming, blocking users from accessing the live stream content, or combinations thereof.

19. The system of claim 16 , wherein:

analyzing the live stream content comprises using a computer vision process to identify prohibited content as defined by the subscribed to prohibited frame content template;

wherein the computer vision process comprises using machine learning processes.

20. The system of claim 16 , comprising a database coupled to the computing device to store the prohibited frame content template.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2018
From: FOX, JEREMY R.; DUNNE, JONATHAN; HARPUR, LIAM S.; ANDERS, KELLEY
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 046729/0901 →
Continuity (1)
Related Publication 20200077150A1 · Mar 5, 2020