IP Library Granted Patent US 10,878,246
Granted Patent B1
US 10,878,246 · App. 16/102,064 · Granted Dec 29, 2020

Systems and methods for performing client-side content inference

Inventors: Narsing Krishna Vijayrao (Santa Clara, CA); Jason M. Taylor (Piedmont, CA)
Assignee: Facebook, Inc.
G06K9/00718G06N20/00H04L63/0245
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,878,246
App. No.
16/102,064
Granted
Dec 29, 2020
Kind
B1
Abstract

A method for performing client-side content inference may include (1) receiving a request to upload, from a client-side device to a server-side device, a content item that includes a first sequence of bytes and a second sequence of bytes, (2) identifying a model configured to output a classification for sequences of bytes, (3) using, at the client-side device, the model to derive a first classification for the first sequence, (4) using, at the client-side device, the model to derive a second classification for the second sequence, and (5) uploading, in response to the request, the content item to the server-side device by (a) uploading the first sequence, (b) uploading the first classification substantially contemporaneous with uploading the first sequence, (c) uploading the second sequence, and (d) uploading the second classification substantially contemporaneous with uploading the second sequence. Various other methods, systems, and computer-readable media are also disclosed.

Claims (105)

1. A method comprising:

receiving a request to upload a content item from a client-side computing device to one or more server-side computing devices, the content item comprising at least a first sequence of bytes and a second sequence of bytes;

identifying, at the client-side computing device, at least one model configured to output one or more classifications for sequences of bytes in the content item;

using, at the client-side computing device, the at least one model to derive a first classification for the first sequence;

using, at the client-side computing device, the at least one model to derive a second classification for the second sequence; and

uploading, in response to the request, the content item from the client-side computing device to the one or more server-side computing devices by:

uploading the first sequence;

uploading the first classification substantially contemporaneous with uploading the first sequence;

uploading the second sequence; and

uploading the second classification substantially contemporaneous with uploading the second sequence.

2. The method of claim 1 , wherein the second sequence and the second classification are uploaded from the client-side computing device to the one or more server-side computing devices after the first sequence and the first classification are uploaded from the client-side computing device to the one or more server-side computing devices.

3. The method of claim 1 , wherein:

the content item comprises a video file recorded and stored at the client-side computing device;

the first sequence comprises one or more frames of the video file; and

the second sequence comprises one or more additional frames of the video file.

4. The method of claim 1 , wherein:

the content item comprises a live video stream being captured by and streamed from the client-side computing device;

the first sequence comprises one or more frames of the live video stream; and

the second sequence comprises one or more additional frames of the live video stream.

5. The method of claim 1 , wherein the content item comprises at least one of:

an audio file; or

a live audio stream.

6. The method of claim 1 , wherein:

the first classification indicates that the first sequence contains objectionable content; and

the second classification indicates that the second sequence does not contain objectionable content.

7. The method of claim 1 , wherein:

the first sequence and the second sequence are overlapping sequences;

the first classification indicates that the first sequence contains an element of a known type; and

the second classification indicates that the second sequence contains the element in an objectionable context.

8. The method of claim 1 , further comprising:

identifying, at the one or more server-side computing devices, a content policy that:

maps the first classification to a first action to be performed when any sequence of bytes has been classified with the first classification; and

maps the second classification to a second action to be performed when any sequence of bytes has been classified with the second classification;

receiving, at the one or more server-side computing devices, the first sequence;

receiving, at the one or more server-side computing devices, the first classification;

performing, in response to receiving the first classification, the first action;

receiving, at the one or more server-side computing devices, the second sequence;

receiving, at the one or more server-side computing devices, the second classification; and

performing, in response to receiving the second classification, the second action.

9. The method of claim 1 , wherein uploading the first classification comprises:

uploading a first indicator of the first classification substantially contemporaneous with uploading the first byte of the first sequence; and

uploading a second indicator of the first classification substantially contemporaneous with uploading the last byte of the first sequence.

10. The method of claim 1 , wherein:

the content item further comprises a third sequence of bytes;

the method further comprises using, at the client-side computing device, the at least one model to derive a third classification for the third sequence; and

uploading the content item from the client-side computing device to the one or more server-side computing devices further comprises refraining, at the client-side computing device based at least in part on the third classification of the third sequence, from uploading the third sequence to the one or more server-side computing devices.

11. The method of claim 1 , further comprising identifying, at the client-side computing device, an upload policy that:

maps the first classification to a first action to be performed at the client-side computing device when any sequence of bytes has been classified with the first classification; and

maps the second classification to a second action to be performed at the client-side computing device when any sequence of bytes has been classified with the second classification.

12. The method of claim 11 , wherein:

the first action comprises uploading any sequence of bytes that has been classified with the first classification to a first one of the one or more server-side computing devices; and

the second action comprises uploading any sequence of bytes that has been classified with the second classification to a second one of the one or more server-side computing devices.

13. A system comprising:

one or more server-side computing devices; and

a client-side software agent, stored in physical memory of a client-side computing device and executed by a physical processor of the client-side computing device, comprising:

a receiving module that receives a request to upload a content item from the client-side computing device to the one or more server-side computing devices, the content item comprising at least a first sequence of bytes and a second sequence of bytes;

an identifying module that identifies, at the client-side computing device, at least one model configured to output one or more classifications for sequences of bytes in the content item;

a classifying module that uses, at the client-side computing device, the at least one model to:

derive a first classification for the first sequence; and

derive a second classification for the second sequence; and

an uploading module that uploads, in response to the request, the content item from the client-side computing device to the one or more server-side computing devices by:

uploading the first sequence;

uploading the first classification substantially contemporaneous with uploading the first sequence;

uploading the second sequence; and

uploading the second classification substantially contemporaneous with uploading the second sequence.

14. The system of claim 13 , wherein the uploading module uploads the second sequence and the second classification from the client-side computing device to the one or more server-side computing devices after uploading the first sequence and the first classification from the client-side computing device to the one or more server-side computing devices.

15. The system of claim 13 , wherein the one or more server-side computing devices comprise a content-management module, stored in physical memory of the one or more server-side computing devices and executed by a physical processor of the one or more server-side computing devices, that:

identifies, at the one or more server-side computing devices, a content policy that:

maps the first classification to a first action to be performed when any sequence of bytes has been classified with the first classification; and

maps the second classification to a second action to be performed when any sequence of bytes has been classified with the second classification;

receives, at the one or more server-side computing devices, the first sequence;

receives, at the one or more server-side computing devices, the first classification;

performs, in response to receiving the first classification, the first action;

receives, at the one or more server-side computing devices, the second sequence;

receives, at the one or more server-side computing devices, the second classification; and

performs, in response to receiving the second classification, the second action.

16. The system of claim 15 , wherein:

the first security action comprises transmitting any sequence of bytes that has been classified with the first classification to a first additional computing device; and

the second security action comprises transmitting any sequence of bytes that has been classified with the second classification to a second additional computing device.

17. The system of claim 13 , wherein the uploading module uploads the first classification from the client-side computing device to the one or more server-side computing devices by:

uploading a first indicator of the first classification substantially contemporaneous with uploading the first byte of the first sequence; and

uploading a second indicator of the first classification substantially contemporaneous with uploading the last byte of the first sequence.

18. The system of claim 13 , wherein:

the content item further comprises a third sequence of bytes;

the classifying module further uses, at the client-side computing device, the at least one model to derive a third classification for the third sequence; and

the uploading module uploads the content item from the client-side computing device to the one or more server-side computing devices by refraining, at the client-side computing device based at least in part on the third classification of the third sequence, from uploading the third sequence to the one or more server-side computing devices.

19. The system of claim 13 , wherein:

the first sequence and the second sequence are overlapping sequences;

the first classification indicates that the first sequence contains an element of a known type; and

the second classification indicates that the second sequence contains the element in an objectionable context;

the uploading module uploads the first classification and the second classification by:

uploading a first indicator of the first classification substantially contemporaneous with uploading the first byte of the first sequence;

uploading a second indicator of the second classification substantially contemporaneous with uploading the first byte of the second sequence;

uploading a third indicator of the first classification substantially contemporaneous with uploading the last byte of the first sequence; and

uploading a fourth indicator of the second classification substantially contemporaneous with uploading the last byte of the second sequence.

20. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a client-side computing device, cause the client-side computing device to:

receive a request to upload a content item from the client-side computing device to one or more server-side computing devices, the content item comprising at least a first sequence of bytes and a second sequence of bytes;

identify, at the client-side computing device, at least one model configured to output one or more classifications for sequences of bytes in the content item;

use, at the client-side computing device, the at least one model to derive a first classification for the first sequence;

use, at the client-side computing device, the at least one model to derive a second classification for the second sequence; and

upload, in response to the request, the content item from the client-side computing device to the one or more server-side computing devices by:

uploading the first sequence;

uploading the first classification substantially contemporaneous with uploading the first sequence;

uploading the second sequence; and

uploading the second classification substantially contemporaneous with uploading the second sequence.

Assignments (3)
CHANGE OF NAME Recorded Feb 18, 2022
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 059191/0695 →
CORRECTIVE ASSIGNMENT TO CORRECT THE COMMA OF ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 048597 FRAME: 0574. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Feb 1, 2022
From: VIJAYRAO, NARSING KRISHNA; TAYLOR, JASON M.
To: FACEBOOK, INC.
Reel/Frame 058922/0280 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2019
From: VIJAYRAO, NARSING KRISHNA; TAYLOR, JASON M.
To: FACEBOOK, INC.,
Reel/Frame 048597/0574 →
Cited By (1)
US 12,327,390