IP Library › Granted Patent US 12,289,482
Granted Patent B2
US 12,289,482 · App. 18/610,920 · Granted Apr 29, 2025

Methods, systems, and media for identifying videos containing objectionable content

Inventors: Bartlomiej Wolowiec (Zürich, CH); Andreea-Alexandra Ganciulescu (Oxford, GB)
Assignee: Google LLC
H04N21/23418G06F18/213G06F18/24147G06N3/04H04N21/25875H04N21/2743H04N21/454
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,289,482
App. No.
18/610,920
Granted
Apr 29, 2025
Kind
B2
Abstract

Techniques are disclosed for identifying videos containing objectionable content. An example method comprises identifying, by a computing system, a video uploaded to a video sharing service, generating an embedding for the video using a neural network, wherein the embedding specifies a location of the video in a multi-dimensional space where a plurality of videos are located based on content of the videos, identifying from the videos a plurality of associated videos that each have an associated embedding that is located within a predetermined distance of the embedding, determining whether the video is likely to include a particular type of objectionable content by at least determining at least a predetermined amount of the associated videos that contain the particular type of objectionable content, and responsive to determining that the video is likely to include the particular type of objectionable content, causing the video to be blocked from the video sharing service.

Claims (43)

1. A method comprising:

identifying, by a computing system using a hardware processor, a video that has been uploaded to a video sharing service;

generating, by the computing system, an embedding for the video using a neural network, wherein the embedding specifies a location of the video in a multi-dimensional space where a plurality of videos are located based on content of the plurality of videos;

identifying, by the computing system, from the plurality of videos a plurality of associated videos that each have an associated embedding that is located within a predetermined distance of the embedding of the video in the multi-dimensional space;

determining, by the computing system, whether the video is likely to include a particular type of objectionable content by at least determining at least a predetermined amount of the plurality of associated videos that contain the particular type of objectionable content; and

responsive to determining that the video is likely to include the particular type of objectionable content, causing, by the computing system, the video to be blocked from the video sharing service.

2. The method of claim 1 , further comprising transmitting, by the computing system, an indication of the video to a user account associated with a verifier prior to causing the video to be blocked from the video sharing service, wherein causing the video to be blocked is responsive to receiving, by the computing system, a verification from the user account associated with the verifier that the video includes the objectionable content.

3. The method of claim 1 , further comprising determining, by the computing system, a confidence level that the video includes the particular type of objectionable content based on the predetermined amount.

4. The method of claim 1 , wherein identifying the plurality of associated videos that each have the associated embedding that is located within the predetermined distance of the embedding of the video comprises transmitting, by the computing system, a query to a remote server that includes the embedding for the video.

5. The method of claim 1 , wherein causing the video to be blocked from the video sharing service comprises assigning, by the computing system, a viewer restriction to the video.

6. The method of claim 1 , wherein the predetermined amount is selected from the group consisting of a predetermined amount and a predetermined percentage.

7. The method of claim 2 , wherein transmitting the indication of the video to the user account associated with the verifier responsive to determining the at least the predetermined amount of the plurality of associated videos contain the particular type of objectionable content.

8. The method of claim 3 , further comprising:

determining, by the computing system, that the confidence level is below a predetermined threshold; and

responsive to determining that the confidence level is below the predetermined threshold, transmitting, by the computing system, an indication of the video to a user account associated with a verifier prior to causing the video to be blocked from the video sharing service, wherein causing the video to be blocked is responsive to receiving, by the computing system, a verification from the user account associated with the verifier that the video includes the objectionable content.

9. A computing system comprising:

a memory that stores instructions; and

processing circuitry that executes the instructions to:

identify a video that has been uploaded to a video sharing service;

generate an embedding for the video using a neural network, wherein the embedding specifies a location of the video in a multi-dimensional space where a plurality of videos are located based on content of the plurality of videos;

identify from the plurality of videos a plurality of associated videos that each have an associated embedding that is located within a predetermined distance of the embedding of the video in the multi-dimensional space;

determine whether the video is likely to include a particular type of objectionable content by at least determining at least a predetermined amount of the plurality of associated videos that contain the particular type of objectionable content; and

responsive to determining that the video is likely to include the particular type of objectionable content, cause the video to be blocked from the video sharing service.

10. The computing system of claim 9 , wherein the processing circuitry further executes the instructions to transmit an indication of the video to a user account associated with a verifier prior to causing the video to be blocked from the video sharing service, wherein causing the video to be blocked is responsive to receiving a verification from the user account associated with the verifier that the video includes the objectionable content.

11. The computing system of claim 9 , wherein the processing circuitry further executes the instructions to determine a confidence level that the video includes the particular type of objectionable content based on the predetermined amount.

12. The computing system of claim 9 , wherein to identify the plurality of associated videos that each have the associated embedding that is located within the predetermined distance of the embedding of the video the processing circuitry further executes the instructions to transmit a query to a remote server that includes the embedding for the video.

13. The computing system of claim 9 , wherein to cause the video to be blocked from the video sharing service the processing circuitry further executes the instructions to assign a viewer restriction to the video.

14. The computing system of claim 9 , wherein the predetermined amount is selected from the group consisting of a predetermined amount and a predetermined percentage.

15. The computing system of claim 10 , wherein transmitting the indication of the video to the user account associated with the verifier is responsive to determining the at least the predetermined amount of the plurality of associated videos contain the particular type of objectionable content.

16. The computing system of claim 11 , wherein the processing circuitry further executes the instructions to:

determine that the confidence level is below a predetermined threshold; and

responsive to determining that the confidence level is below the predetermined threshold, transmit an indication of the video to a user account associated with a verifier prior to causing the video to be blocked from the video sharing service, wherein causing the video to be blocked is responsive to receiving a verification from the user account associated with the verifier that the video includes the objectionable content.

17. A non-transitory computer-readable storage medium comprising instructions, that when executed by processing circuitry of a computing system, cause the processing circuitry to:

identify a video that has been uploaded to a video sharing service;

generate an embedding for the video using a neural network, wherein the embedding specifies a location of the video in a multi-dimensional space where a plurality of videos are located based on content of the plurality of videos;

identify from the plurality of videos a plurality of associated videos that each have an associated embedding that is located within a predetermined distance of the embedding of the video in the multi-dimensional space;

determine whether the video is likely to include a particular type of objectionable content by at least determining at least a predetermined amount of the plurality of associated videos that contain the particular type of objectionable content; and

responsive to determining that the video is likely to include the particular type of objectionable content, cause the video to be blocked from the video sharing service.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when further executed by the processing circuitry, cause the processing circuitry to determine a confidence level that the video includes the particular type of objectionable content based on the predetermined amount.

19. The non-transitory computer-readable storage medium of claim 17 , wherein to cause the video to be blocked from the video sharing service the instructions, when further executed by the processing circuitry, cause the processing circuitry to assign a viewer restriction to the video.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the instructions, when further executed by the processing circuitry, cause the processing circuitry to:

determine that the confidence level is below a predetermined threshold; and

responsive to determining that the confidence level is below the predetermined threshold, transmit an indication of the video to a user account associated with a verifier prior to causing the video to be blocked from the video sharing service, wherein causing the video to be blocked is responsive to receiving a verification from the user account associated with the verifier that the video includes the objectionable content.

Continuity (3)
Continuation 17055235
Provisional Application 62779717 · Dec 14, 2018
Related Publication 20240223825A1 · Jul 4, 2024
References Cited (41)
US 8204883B1 · Smith et al. · 2012 [cited by applicant]
US 8649613B1 · Leung et al. · 2014 [cited by applicant]
US 8706675B1 · Samaniego et al. · 2014 [cited by applicant]
US 9053416B1 · De Leo et al. · 2015 [cited by applicant]
US 11216701B1 · Sim et al. · 2022 [cited by applicant]
US 20070067258A1 · Theilmann · 2007 [cited by applicant]
US 20080168490A1 · Yu · 2008 [cited by applicant]
US 20090177670A1 · Grenier et al. · 2009 [cited by applicant]
US 20090274364A1 · Shakya et al. · 2009 [cited by applicant]
US 20130085825A1 · Davis et al. · 2013 [cited by applicant]
US 20160239493A1 · Stroganov · 2016 [cited by applicant]
US 20160350675A1 · Laks et al. · 2016 [cited by applicant]
US 20170039554A1 · Greifeneder et al. · 2017 [cited by applicant]
US 20170357896A1 · Tsatsin et al. · 2017 [cited by applicant]
US 20180012460A1 · Heitz, III et al. · 2018 [cited by applicant]
US 20180053042A1 · Philbin et al. · 2018 [cited by applicant]
US 20180121827A1 · Green · 2018 [cited by applicant]
US 20180336888A1 · Jiang et al. · 2018 [cited by applicant]
US 20180349796A1 · Gibbs · 2018 [cited by examiner]
US 20210142097A1 · Zheng et al. · 2021 [cited by applicant]
US 20210227291A1 · Wolowiec et al. · 2021 [cited by applicant]
CN 101711470 · 2010 [cited by applicant]
CN 107409238 · 2017 [cited by applicant]
CN 107566903 · 2018 [cited by applicant]
CN 108874886 · 2018 [cited by applicant]
EP 3316185A1 · 2018 [cited by applicant]
Communication pursuant to Article 94(3) EPC from counterpart European Application No. 19824039.2 dated Jun. 1, 2023, 6 pp. [cited by applicant]
Communication pursuant to Rules 161(1) and 162 EPC from counterpart European Application No. 19824039.2 dated Dec. 1, 2020, 3 pp. [cited by applicant]
Examination Report dated Jun. 1, 2023 in EP Patent Application No. 19824039.2. [cited by applicant]
International Preliminary Report on Patentability from International Application No. PCT/US2019/062787 dated Jun. 8, 2021, 7 pp. [cited by applicant]
International Search Report and Written Opinion of International Application No. PCT/US2019/062787 dated Feb. 7, 2020, 11 pp. [cited by applicant]
Office Action dated Jan. 19, 2022 in CN Patent Application No. 201980038483.5. [cited by applicant]
Office Action dated Jun. 30, 2022 in CN Patent Application No. 201980038483.5. [cited by applicant]
Office Action from counterpart Chinese Application No. 201980038483.5 dated Jan. 30, 2024, 11 pp. [cited by applicant]
Office Action, and translation thereof, from counterpart Chinese Application No. 201980038483.5 dated Nov. 10, 2023, 15 pp. [cited by applicant]
Prosecution History from U.S. Appl. No. 17/055,235, dated Nov. 13, 2020 through Jan. 10, 2024, 150 pp. [cited by applicant]
Response to Communication pursuant to Article 94(3) EPC dated Jun. 1, 2023, from counterpart European Application No. 19824039.2 filed Dec. 11, 2023, 19 pp. [cited by applicant]
Response to Communication pursuant to Article 94(3) EPC dated Jun. 1, 2023, from counterpart European Application No. 19824039.2 filed Dec. 11, 2023, 21 pp. [cited by applicant]
Response to Communication Pursuant to Rules 161(1) and 162 EPC dated Dec. 1, 2020, from counterpart European Application No. 19824039.2, filed Jun. 8, 2021, 8 pp. [cited by applicant]
Response to Office Action dated Nov. 10, 2023, from counterpart Chinese Application No. 201980038483.5 filed Dec. 25, 2023, 11 pp. [cited by applicant]
Schroff et al., “FaceNet: A Unified Embedding for Face Recognition and Clustering”, Computer Vision Foundation, Jun. 2015, 10 pp. [cited by applicant]